Which caching strategy for pagination?
Unanswered
Birman posted this in #help-forum
BirmanOP
Hi,
I have a pretty simple, common scenario, but I can't really find the best way to construct it in Next.
I make it simple:
I have a route with a list of items that get loaded from a MySQL DB. The route also has pagination, currently with a get parameter for the pagination.
Problem: My DB is huge and the query is not super fast, I couldn't get it faster than 1-2s.
My target: I want to cache all the pages of the pagination (or the request result) and ideally they will be rendered during build so they are fast from the first request. I want it fast so crawlers don't use up much crawl budget.
There seem to be many different solutions, but I'm not sure which route is the best.
When I want to use generateStaticParams I'd need to make the current page part of the route instead of a get param afaik, but that doesn't seem recommended for pagination. And I don't know another way of rendering pages during build.
I also could use unstable_cache for the data, to at least make it fast from the second request on fast, but this seems to be deprecated soon, according to the docs.
Another option would be "use cache" directive, to cache the whole pages, which also would only speed up from the second request onwards.
Another thing I want to avoid is Suspense. This has to be awaited by crawlers like Google Bot and others don't seem to support it at all.
What is the best way to handle it? A fast, prerendered route with pagination can't be that hard right? That should be pretty common.
I also could imagine something hacked together, for example using useStaticParams to prerender the page, but instead of generating every single page, I use unstable_cache to load all results in one query and then pick the shown results in frontend based on the get param? That would also make a snapshot of the DB so pagination couldn't get inconsistent based on age of every single page. But I'm not sure if thats even possible.
I have a pretty simple, common scenario, but I can't really find the best way to construct it in Next.
I make it simple:
I have a route with a list of items that get loaded from a MySQL DB. The route also has pagination, currently with a get parameter for the pagination.
Problem: My DB is huge and the query is not super fast, I couldn't get it faster than 1-2s.
My target: I want to cache all the pages of the pagination (or the request result) and ideally they will be rendered during build so they are fast from the first request. I want it fast so crawlers don't use up much crawl budget.
There seem to be many different solutions, but I'm not sure which route is the best.
When I want to use generateStaticParams I'd need to make the current page part of the route instead of a get param afaik, but that doesn't seem recommended for pagination. And I don't know another way of rendering pages during build.
I also could use unstable_cache for the data, to at least make it fast from the second request on fast, but this seems to be deprecated soon, according to the docs.
Another option would be "use cache" directive, to cache the whole pages, which also would only speed up from the second request onwards.
Another thing I want to avoid is Suspense. This has to be awaited by crawlers like Google Bot and others don't seem to support it at all.
What is the best way to handle it? A fast, prerendered route with pagination can't be that hard right? That should be pretty common.
I also could imagine something hacked together, for example using useStaticParams to prerender the page, but instead of generating every single page, I use unstable_cache to load all results in one query and then pick the shown results in frontend based on the get param? That would also make a snapshot of the DB so pagination couldn't get inconsistent based on age of every single page. But I'm not sure if thats even possible.
4 Replies
BirmanOP
tl;dr: I want a route with pagination of a slow db query, that is fast from the first request on. Pre-rendering the pages is fine, Suspense should be avoided. What is the best solution?
@Birman tl;dr: I want a route with pagination of a slow db query, that is fast from the first request on. Pre-rendering the pages is fine, Suspense should be avoided. What is the best solution?
Poodle
Easiest way is to switch pagination to path segments like /items/page/1 instead of query params. Then generateStaticParams works natively and pages are prerendered at build. The "not recommended" advice for path based pagination is outdated, google handles it fine. Your slow query runs at build time so users never wait. Use revalidate for ISR to keep it fresh. If you need to keep query params, redirect from them to the path based URLs for backward compat.
BirmanOP
Thanks, that's very helpful. I'll switch to having the page number in the path then.
Is there a difference between just setting the revalidate time in the page file vs using "use cache" and cacheLife? The cacheLife has values for expire/stale and revalidate, while the "old" revalidate only seems to regenerate in the background but doesn't care for how old the users loaded version is for example.
If that's the case, I guess "use cache" would be better in my case, because it has this additional benefit.
Is there a difference between just setting the revalidate time in the page file vs using "use cache" and cacheLife? The cacheLife has values for expire/stale and revalidate, while the "old" revalidate only seems to regenerate in the background but doesn't care for how old the users loaded version is for example.
If that's the case, I guess "use cache" would be better in my case, because it has this additional benefit.
@Birman Thanks, that's very helpful. I'll switch to having the page number in the path then.
Is there a difference between just setting the revalidate time in the page file vs using "use cache" and cacheLife? The cacheLife has values for expire/stale and revalidate, while the "old" revalidate only seems to regenerate in the background but doesn't care for how old the users loaded version is for example.
If that's the case, I guess "use cache" would be better in my case, because it has this additional benefit.
Poodle
yeah you are right, use cache with cacheLife gives you that extra control over stale vs expire. Go with that for your use case.