Serving 12mil pages
Unanswered
Sun bear posted this in #help-forum
Sun bearOP
Hey! I am using fallback: "blocking" with pages router for a lot of pages that are less important but still somewhat important.
The issue I'm having is that every time Google tries to scrape these pages, my more important pages (even the prerendered ones) slow down drastically.
I use 8 instances of the nextjs app with LB so there is some scaling done already.
I suspect that this is because of the fallback blocking behavior, and maybe that nextjs runs the getStaticProps in the main event loop and not in a separate worker?
I've already done some of the low hanging fruits like moving static files to cdn.
Just curious what you would do in this situation. I suspect that one option suggested would be to host a separate nextjs server for these pages, but I would prefer not to because there are a lot of shared logic still.
Thanks in advance.
The issue I'm having is that every time Google tries to scrape these pages, my more important pages (even the prerendered ones) slow down drastically.
I use 8 instances of the nextjs app with LB so there is some scaling done already.
I suspect that this is because of the fallback blocking behavior, and maybe that nextjs runs the getStaticProps in the main event loop and not in a separate worker?
I've already done some of the low hanging fruits like moving static files to cdn.
Just curious what you would do in this situation. I suspect that one option suggested would be to host a separate nextjs server for these pages, but I would prefer not to because there are a lot of shared logic still.
Thanks in advance.
2 Replies
Are you hosting on Vercel? Given your relatively unique sounding situation + the LB mention, I get a sense that you're not.
To which, I would suggest checking out [@neshca/cache-handler](https://caching-tools.github.io/next-shared-cache). We've used it on a few apps after we learned about it (we also don't host on Vercel), and it's been great. No issues with it on relatively high scale apps, etc. And for your case, it sounds like it could be immensely helpful, at least it certainly wouldn't hurt.