How do I pinpoint this NextJS memory leak?
Answered
Irish Red and White Setter posted this in #help-forum
Irish Red and White SetterOP
NextJS 14.2.18, app router.
Our docker container Azure hosted NextJS keeps crashing due to a memory leak. It also shows up on our staging environment. In this image I am comparing two heapdumps, one is almost right after deployment, and the other is 12 hours later. There is about a 150 megabyte difference.
I am in the devtools memory heapdump compare view of the heap dumps, and I see (string) accounts for a 100 megabyte increase.
The majority of these strings are all the same string of 629896 bytes.
I'm a beginner when it comes to tracking down memory leaks in NextJS internal code. Can someone point me in the right direction here? Does the Retainers panel indicate some sort of origin for this leak?
The string in question is:
Why is this allocated over and over and never garbage collected?
Our docker container Azure hosted NextJS keeps crashing due to a memory leak. It also shows up on our staging environment. In this image I am comparing two heapdumps, one is almost right after deployment, and the other is 12 hours later. There is about a 150 megabyte difference.
I am in the devtools memory heapdump compare view of the heap dumps, and I see (string) accounts for a 100 megabyte increase.
The majority of these strings are all the same string of 629896 bytes.
I'm a beginner when it comes to tracking down memory leaks in NextJS internal code. Can someone point me in the right direction here? Does the Retainers panel indicate some sort of origin for this leak?
The string in question is:
"3:I[98087,[\"759\",\"static/chunks/759-01bcf85907a5d319.js\",\"54\",\"static/chunks/54-4e81643d16213e17.js\",\"975\",\"static/chunks/app/%5B%5B...slug%5D%5D/layout-fa302a4d416bd5b8.js\"],\"GoogleTagManager\"]\n4:I[40215,[\"759\",\"static/chunks/759-01bcf85907a5d319.js\",\"54\",\"static/chunks/54-4e81643d16213e17.js\",\"975\",\"static/chunks/app/%5B%5B...slug%5D%5D/layout-fa302a4d416bd5b8.js\"],\"default\"]\n5:I[74887,[\"759\",\"static/chunks/759-01bcf85907a5d319.js\",\"54\",\"static/chunks/54-4e81643d16213e17.js\",\"975\",\"static/chunks/app/%5B%5B...slug%5D%5D/layout-fa302a4d416bd5b8.js\"],\"default\"]\n6:I[59417,[\"759\",\"static/chunks/759-01bcf85907a5d319.js\",\"54\",\"static/chunks/54-4e81643d16213e17.js\",\"975\",\"static/chunks/app/%5B%5B...slug%5D%5D/layout-fa302a4d416bd5b8.js\"],\"default\"]\n7:I[4707,[],\"\"]\n9:I[29214,[\"381\",\"static/chunks/app/%5B%5B...slug%5D%5D/error-66d0fa22b0411c94.js\"],\"default\"]\na:I[36423,[],\"\"]\n8:[\"slug\",\"robots933456.txt\",\"oc\"]\n0:[\"HfVC6Tsh4SXfyzIqEwIwA\",[[[\"\",{\"children\":[[\"slug\",\"robots933456.txt\",\"oc\"],{\"children\":[\"__PAGE__\",{}]},",Why is this allocated over and over and never garbage collected?
Answered by Irish Red and White Setter
Managed to find the root cause of the memory leak, discussed here:
https://github.com/vercel/next.js/discussions/68636
Upgraded to Node 22.11 and it erased the issue.
The problem was memory leak in underlying Undici.
https://github.com/vercel/next.js/discussions/68636
Upgraded to Node 22.11 and it erased the issue.
The problem was memory leak in underlying Undici.
8 Replies
Irish Red and White SetterOP
For example I can see some "context in get cookies()" down that tree but I have no idea if that then indicates a connection to code where cookies is called or not.
Giant Angora
YEa you can debug those static js files. Does it tell you which js asset is the problem? It will be minified so hard to read but you can copy it into a formatter.
tip: all functions will be renamed but object values are untouched, that should help you get closer to the issue.
Can you run the same build in a docker vm with the same hardware resources to replicate locally?
tip: all functions will be renamed but object values are untouched, that should help you get closer to the issue.
Can you run the same build in a docker vm with the same hardware resources to replicate locally?
Irish Red and White SetterOP
@Giant Angora Not sure I follow, which static files should I debug?
Those filenames in those strings?
These are strings in memory, which are accumulated and not released, see this example image.
There you can see more pure "content" strings as well further down.
I don't see the logical connection to filenames in those strings to the source of a memory leak being within those files. Maybe there is but I'm just not following.
I can find those files in the server either way.
I will try to perform a more long term test on local docker to see if I can have a local proof of concept as well.
Those filenames in those strings?
These are strings in memory, which are accumulated and not released, see this example image.
There you can see more pure "content" strings as well further down.
I don't see the logical connection to filenames in those strings to the source of a memory leak being within those files. Maybe there is but I'm just not following.
I can find those files in the server either way.
I will try to perform a more long term test on local docker to see if I can have a local proof of concept as well.
Giant Angora
That is challenging.
If you do a next build you will get a similar set of static assets. Once you next start run the app and can see the mem leak, youve confirmed the bug is in your build.
Did you try any of these? https://nextjs.org/docs/app/building-your-application/optimizing/memory-usage#run-next-build-with---experimental-debug-memory-usage
Also you can try appoarch this differently. Do you know when the error started. Start going through commits to narrow down the change. git bisect can be helpful
If you do a next build you will get a similar set of static assets. Once you next start run the app and can see the mem leak, youve confirmed the bug is in your build.
Did you try any of these? https://nextjs.org/docs/app/building-your-application/optimizing/memory-usage#run-next-build-with---experimental-debug-memory-usage
Also you can try appoarch this differently. Do you know when the error started. Start going through commits to narrow down the change. git bisect can be helpful
Irish Red and White SetterOP
Good tip on the experimental debug, I'll try that!
Irish Red and White SetterOP
Managed to find the root cause of the memory leak, discussed here:
https://github.com/vercel/next.js/discussions/68636
Upgraded to Node 22.11 and it erased the issue.
The problem was memory leak in underlying Undici.
https://github.com/vercel/next.js/discussions/68636
Upgraded to Node 22.11 and it erased the issue.
The problem was memory leak in underlying Undici.
Answer
@Irish Red and White Setter Managed to find the root cause of the memory leak, discussed here:
https://github.com/vercel/next.js/discussions/68636
Upgraded to Node 22.11 and it erased the issue.
The problem was memory leak in underlying Undici.
American
Hopefully this solves my memory leak issue too 🤞 Thanks for doing the hard part!
@American Hopefully this solves my memory leak issue too 🤞 Thanks for doing the hard part!
Irish Red and White SetterOP
It was hell I tell ya!