whats wrong with built in node fetch (ie embedded undici)?

@risky We believe changing to another fetch may resolve an issue we are facing.

Our client has about 2600 pages. These are continually being revalidated. Pages router.

Intermittently we face issues where calls to the built in res.revalidate()function simply halts/stalls, and never returns.
We have combed through the source code of NextJS and it's sadly very hard for us to figure out where the bug exactly lies.
Due to the intermittent nature of the issue it's impossible to create an issue for it as well.

Our only hope is to "shake" the system up enough to make it work, by switching out the fetch function. We recently solved an issue for posting forms by doing exactly that, but that was not related to NextJS.

Right now there are random pages that we can't call "res.revalidate" on and the execution never even gets to the point where getStaticPropsis called. So there's a problem within NextJS.

We spent weeks to update from Next 12 to 14 with hopes of that being able to solve it.
There was actually one difference from doing that:

- v12: Failed revalidates always timed out after 300 seconds.
- v14: Failed revalidates never times out, just hangs indefinitely.

Parallell with some pages being impossible to revalidate with hanged function call, some pages can also get into a state where the browser never can get a response on a given page.

So we suspect the issue is that NextJS never handles the request on the build in router level.

Once a page is "tainted" with not being able to load or revalidate, it stays this way forever, until we restart NextJS. After that, it works fine.

Until suddenly, any random page is struck by this issue.

It seems memory related, or "promise not returning"-related. It's very strange, and impossible to debug for us.

Server is fine with plenty of CPU and memory available.

uhh this is very interesting info, sadly I dont know quite the answer to either (im more a app dir person and i know the logic behind both revalidate and things are so diferent)

i hope someone can help you with your issue, but i personally dont think fetch lib should be issue (i know in app they "patch" fetch for some extra caching)

-# i think you should update your question title to this instead though as someone may be able to help with this context

What would you recommend I update the title to?

I feel like I am spamming this forum at this point 🙂

https://nextjs-forum.com/post/1277565951990304872#message-1277565951990304872

I have no idea who to ask at this moment. We are considering a huge rewrite to use the app router instead, but the client has no incentive to pay for that :/

Oh i see that now, personally this form is 75% app dir at this point (there are still some quite knowledgeable pages dir people), but I really think this is some really random error

what are you doing rn as a solution? just not have any caching?

No, we continually revalidate all pages, from CMS webhooks basically.

All pages are dynamic, from [...slug], and are not revalidated based on users, but instead we use on demand revalidation.

So 99% of the pages work, but some pages gets this "taint" of not being able to be revalidated (or in worse cases, visited).

These pages cannot be update by the client, unless we restart the server.

It's hosted in a Docker container on Azure as a web container app.

You can simply install node-fetch and import fetch from node-fetch

Nextjs only polyfills the global fetch, not the fetch you bring by importing a library

But I can't change the inner workings of NextJS, how do we make sure NextJS uses another global fetch?

I assume it may not be possible, but we're desperate for any fix that'll save us hundreds of hours to rewrite to app router.. 😄

You can’t change the global fetch. It will be a patched fetch.

What I meant is that you simply don’t use the global fetch and instead just import node fetch and use it like any other libraries.

import fetch from "node-fetch"

const res = await fetch(…)

To make the point clearer, the code above is equivalent to

import tackleMcClean from "node-fetch"

const res = await tackleMcClean(…)

in other words, the patched fetch is not used at all

Ok I'm not following 100%, what we need to change is the fetch that is used by NextJS.

This is not within the domain of our code that we write. It is in the domain of NextJS inner workings.

It is what happens behind the call to res.revalidate(), probably related to the server router within NextJS.

I suppose we could fork NextJS and change it that way, but it takes on another dimension of tech debt for our project that way that we'd like to avoid.

So you don’t have control over the fetch being used in the code?

In that case the only way is to monkey patch nextjs using something like pnpm patch or patch-package. Or yeah, fork and publish your own nextjs clone if you are okay with that (it takes effort though)

Wait, I’m not following though. res.revalidate() means the pages router right? In the pages router, the fetch is not patched. It’s the same fetch you would use in nodejs 18+

Yes, this is correct.
Our hope is to be able to change the built in fetch, to for example node-fetch.

Assume the built in nodejs fetch has a bug, what would one do? Use another package = win.

Then I’m 99.99% certain it will fix nothing. But if you want to try, then pnpm patch or use patch-package https://www.npmjs.com/package/patch-package

Thanks.

You are probably right. It's a bug we cannot solve or report, most likely within NextJS "server router" (based on our poking around within the source code), so we're desperate to try anything.

Try a partial upgrade to app router to see if it persists?

I’m inclined to agree with @joulev here, I don’t think fetch is your issue

Is there any chance you could upload some minimal repro? Even just the dockerfile could be telling, does it only happen on prod?