Vercel 408 Function Timeouts
Unanswered
Cuvier’s Dwarf Caiman posted this in #help-forum
Cuvier’s Dwarf CaimanOP
When I upgrade from the latest 15.x to 16.x my service begins to spew 408 Request Timeout errors. Rolling back to 15.x fixes the issue. I tried various versions of 16.x, including the latest with the same result.
- I use fluid compute
- I have a PG pool with attachDatabasePool configured
- there is no rhyme or reason to the pages or routes this effects, it’s all of them.
- my application only uses app router
My max duration is set to 5 minutes.
Middleware resolves in milliseconds.
Response finished in 100s of milliseconds.
Function timeout after 5 minutes.
What would cause this to happen? How can I fix it? I’ve posted for assistance on Vercel official support forum before and received no help. I am out of ideas, and cannot reproduce the issue in a preview environment no matter what I’ve tried, and I’ve tried hundreds of things.
Any advice would be appreciated, thank you kindly.
- I use fluid compute
- I have a PG pool with attachDatabasePool configured
- there is no rhyme or reason to the pages or routes this effects, it’s all of them.
- my application only uses app router
My max duration is set to 5 minutes.
Middleware resolves in milliseconds.
Response finished in 100s of milliseconds.
Function timeout after 5 minutes.
What would cause this to happen? How can I fix it? I’ve posted for assistance on Vercel official support forum before and received no help. I am out of ideas, and cannot reproduce the issue in a preview environment no matter what I’ve tried, and I’ve tried hundreds of things.
Any advice would be appreciated, thank you kindly.
15 Replies
@Cuvier’s Dwarf Caiman When I upgrade from the latest 15.x to 16.x my service begins to spew 408 Request Timeout errors. Rolling back to 15.x fixes the issue. I tried various versions of 16.x, including the latest with the same result.
- I use fluid compute
- I have a PG pool with attachDatabasePool configured
- there is no rhyme or reason to the pages or routes this effects, it’s all of them.
- my application only uses app router
My max duration is set to 5 minutes.
Middleware resolves in milliseconds.
Response finished in 100s of milliseconds.
Function timeout after 5 minutes.
What would cause this to happen? How can I fix it? I’ve posted for assistance on Vercel official support forum before and received no help. I am out of ideas, and cannot reproduce the issue in a preview environment no matter what I’ve tried, and I’ve tried hundreds of things.
Any advice would be appreciated, thank you kindly.
Poodle
That response finishing in 100s of ms but function timing out after 5 min means something is keeping the function alive after the response is done. Usually an open connection or listener thats not cleaning up. With fluid compute + pg pool, my guess is next 16 changed how it signals the function is done, so vercel thinks its still running. You could check if it happens on routes that dont touch the db at all. if those work fine, its definitely the pool or try explicitly calling pool.end() or releasing connections in a
finally block and see if you have any event listeners or streams that arent being closed. Also what pg client are you using? @vercel/postgres or something like node-postgres directly? The attachDatabasePool behavior might be different in 16.
finally block and see if you have any event listeners or streams that arent being closed. Also what pg client are you using? @vercel/postgres or something like node-postgres directly? The attachDatabasePool behavior might be different in 16.
Cuvier’s Dwarf CaimanOP
I have been thinking along the same lines.
Unfortunately this happens across every page and route, even pages that aren’t explicitly interacting with the database. But I can try to explicitly release and see what it does, I’ve tried it before.
I’ve also been unable to reproduce this issue in preview environments. Even happens in production during off hours when less users are online.
Unfortunately this happens across every page and route, even pages that aren’t explicitly interacting with the database. But I can try to explicitly release and see what it does, I’ve tried it before.
I’ve also been unable to reproduce this issue in preview environments. Even happens in production during off hours when less users are online.
In any case you’re the first person who has offered an opinion on this, so I appreciate your advice.
For db connections I’m using Prisma v6.19.2
I also have a read replica pool configured. So in addition to releasing, I may temporarily remove the read replica pool entirely, and see if having a single db connection for everything helps.
As for testing what I am thinking of doing is:
- deploying the change to Vercel
- immediately promoting a pre 16.x release so it’s not customer facing
- testing against the 16.x internal Vercel app
If you have ideas for how to test against a production environment that don’t require that ridiculous dance I’m all ears 👂
- deploying the change to Vercel
- immediately promoting a pre 16.x release so it’s not customer facing
- testing against the 16.x internal Vercel app
If you have ideas for how to test against a production environment that don’t require that ridiculous dance I’m all ears 👂
@Cuvier’s Dwarf Caiman In any case you’re the first person who has offered an opinion on this, so I appreciate your advice.
Poodle
Im more than happy to help, i know how frustrating it can be when you are stuck on something and no one has answers, lets hope we can figure this one out.
Prisma with read replicas adds complexity for sure. Removing the read replica pool temporarily is a good test. If the timeouts stop you know its something with how Prisma handles multiple pools in 16. Could also check if Prisma has any open issues with Next 16 specifically. For testing without that promotion dance, could you create a separate Vercel project pointing to the same repo but on a different branch? That way you get a real prod environment with prod env vars but its isolated. Or use deployment protection on the 16.x deploy so only you can access it while its live.
Cuvier’s Dwarf CaimanOP
I’ll try the deployment protection feature first, see how that goes, and get back to you 🫡
@Cuvier’s Dwarf Caiman I’ll try the deployment protection feature first, see how that goes, and get back to you 🫡
Poodle
Sounds good, let me know how it goes.
Cuvier’s Dwarf CaimanOP
The feature I needed to disable ended up being under Settings > Environments > Production
So far I am not seeing any errors. I did upgrade to the latest Prisma v6.x in this release, so it’s possible that’s the issue, but I’ll wait a bit since it takes at least 5-10m before I see the timeouts after a deploy iirc.
I’m hoping that this was just a Prisma bug this whole time.
So far I am not seeing any errors. I did upgrade to the latest Prisma v6.x in this release, so it’s possible that’s the issue, but I’ll wait a bit since it takes at least 5-10m before I see the timeouts after a deploy iirc.
I’m hoping that this was just a Prisma bug this whole time.
@Cuvier’s Dwarf Caiman The feature I needed to disable ended up being under Settings > Environments > Production
So far I am not seeing any errors. I did upgrade to the latest Prisma v6.x in this release, so it’s possible that’s the issue, but I’ll wait a bit since it takes at least 5-10m before I see the timeouts after a deploy iirc.
I’m hoping that this was just a Prisma bug this whole time.
Poodle
hopefully is just a Prisma bug, how is it holding up so far?
Cuvier’s Dwarf CaimanOP
So far so good, I don’t see any errors. I do see a ton of release event logs from @vercel/functions though. I’m on v3.3.6 of that package.
Perhaps something to do with it logging Prisma releasing the db connections?
Perhaps something to do with it logging Prisma releasing the db connections?
I won’t know for sure until I put this change live to the masses. I’ll probably do that later tonight (EST time zone) or tomorrow morning
@Cuvier’s Dwarf Caiman So far so good, I don’t see any errors. I do see a ton of release event logs from @vercel/functions though. I’m on v3.3.6 of that package.
Perhaps something to do with it logging Prisma releasing the db connections?
Poodle
good to know, yeah those release logs are probably Prisma connections cleaning up properly now. Let me know how the prod deploy goes, good luck!