ISR with Redis on initial build
Answered
Paper wasp posted this in #help-forum
Paper waspOP
Hello, I'm trying to understand what the initial build when using ISR is really doing. When running
Assuming we have an architecture with 2 Next.js container in a cluster, connected to a shared Redis, and we build the Next.js image in the CI, where are those page generated when building the image supposed to go at build time ?
npm run build
it generates static pages based on the generateStaticParams function. Assuming we have an architecture with 2 Next.js container in a cluster, connected to a shared Redis, and we build the Next.js image in the CI, where are those page generated when building the image supposed to go at build time ?
Answered by Alfonsus Ardani
I havent tried but logically, the entire application is re-initialized, including re-running the cache-handler which means that it will try to re-conenct with Redis, and will look at Redis's cache. If at build time it fails then it will re-render all the pages statically at request time (because the cache at redis is missing)
40 Replies
in each of the Next.js container
Chub mackerel
these static pages are created and stored in the .next directory. Specifically, the pre-rendered pages are placed in .next/static and server-related ISR logic in .next/server.
In your setup with 2 Next.js containers, the static pages built by npm run build are bundled into the Docker image or deployment package. They are served by Next.js when users access routes. Redis is used to handle shared cache state, ISR revalidation, and ensure consistency across instances.
In your setup with 2 Next.js containers, the static pages built by npm run build are bundled into the Docker image or deployment package. They are served by Next.js when users access routes. Redis is used to handle shared cache state, ISR revalidation, and ensure consistency across instances.
@Alfonsus Ardani to the `/.next` folder?
Paper waspOP
Ok, I'm trying to understand it with the Next.js example: https://github.com/vercel/next.js/tree/canary/examples/cache-handler-redis
so for this example, when I run my
well it seems for the example above that:
- when running
- when running
but I don't know if I'm supposed to disable the cache handler when building, or if its a normal behavior. As I understand it, since the cache-handler when the application is started only reads from redis and its LRU as fallback, its pointless to generate the static pages at build time (at least for /cet & /gmt) since you are not going to use them if they don't end up in Redis
thanks for your answer I know my question is not that clear
so for this example, when I run my
npm run build
, is the "[timezone]" page (/cet & /gmt) supposed to be generated statically (as a cache warmup or smtg) ? or is it supposed to directly connect to redis at runtime, and if redis is empty just use SSR to warmup the cache?well it seems for the example above that:
- when running
npm run build
the connection to redis is not done and it generates the pages and tries to write it in the LRUcache as a backup (which to me is useless since you are in the build process)- when running
npm run start
the cache is correctly written to redis but the first request if you start with an empty redis is SSR'dbut I don't know if I'm supposed to disable the cache handler when building, or if its a normal behavior. As I understand it, since the cache-handler when the application is started only reads from redis and its LRU as fallback, its pointless to generate the static pages at build time (at least for /cet & /gmt) since you are not going to use them if they don't end up in Redis
thanks for your answer I know my question is not that clear
@Chub mackerel these static pages are created and stored in the .next directory. Specifically, the pre-rendered pages are placed in .next/static and server-related ISR logic in .next/server.
In your setup with 2 Next.js containers, the static pages built by npm run build are bundled into the Docker image or deployment package. They are served by Next.js when users access routes. Redis is used to handle shared cache state, ISR revalidation, and ensure consistency across instances.
Paper waspOP
What you say make sense. What I fail to understand is what happens with pages that uses ISR at build time:
- for a full static page without ISR, its generated and written in file system, and in the docker image => OK
- for a page that uses ISR, is it supposed to be generated at build time as a cache warmup ? or is it supposed to connect to redis regardless of its file system and SSR the first time to init it ?
I'm using this example: https://github.com/vercel/next.js/tree/canary/examples/cache-handler-redis and I fail to understand what it is supposed to for the /cet /gmt at build time
- for a full static page without ISR, its generated and written in file system, and in the docker image => OK
- for a page that uses ISR, is it supposed to be generated at build time as a cache warmup ? or is it supposed to connect to redis regardless of its file system and SSR the first time to init it ?
I'm using this example: https://github.com/vercel/next.js/tree/canary/examples/cache-handler-redis and I fail to understand what it is supposed to for the /cet /gmt at build time
its generated at build time same like static. the ISR part is the param that is not included in the generateStaticParam. If its not included there it will generate a static page at request time
im not sure how that behavior will change if you have a custom cache handler
@Alfonsus Ardani im not sure how that behavior will change if you have a custom cache handler
Paper waspOP
yes thats where I fail to understand what it is supposed to. in the example provided by Next.js, I don't understand what it does when I run it, and what it is supposed to aswell.
but you do understand the ISR part if it isn't using custom cache handler right? :''
@Paper wasp What you say make sense. What I fail to understand is what happens with pages that uses ISR at build time:
- for a full static page without ISR, its generated and written in file system, and in the docker image => OK
- for a page that uses ISR, is it supposed to be generated at build time as a cache warmup ? or is it supposed to connect to redis regardless of its file system and SSR the first time to init it ?
I'm using this example: https://github.com/vercel/next.js/tree/canary/examples/cache-handler-redis and I fail to understand what it is supposed to for the /cet /gmt at build time
Chub mackerel
For static pages without ISR, everything is pre-rendered at build time, stored in .next/static, and bundled into the Docker image. These pages are served directly without involving Redis.
For pages with ISR, only the paths defined in getStaticPaths are pre-rendered at build time. Other paths, like /cet or /gmt if not pre-rendered, are handled dynamically on the first request. This means the page is rendered using SSR, and Redis stores it to serve future requests as a cached version. Redis ensures consistency across containers and manages ISR revalidation.
So, if /cet or /gmt aren’t pre-rendered, they’ll SSR on the first load and initialize the cache through Redis.
For pages with ISR, only the paths defined in getStaticPaths are pre-rendered at build time. Other paths, like /cet or /gmt if not pre-rendered, are handled dynamically on the first request. This means the page is rendered using SSR, and Redis stores it to serve future requests as a cached version. Redis ensures consistency across containers and manages ISR revalidation.
So, if /cet or /gmt aren’t pre-rendered, they’ll SSR on the first load and initialize the cache through Redis.
generateStaticParams*
@Chub mackerel For static pages without ISR, everything is pre-rendered at build time, stored in .next/static, and bundled into the Docker image. These pages are served directly without involving Redis.
For pages with ISR, only the paths defined in getStaticPaths are pre-rendered at build time. Other paths, like /cet or /gmt if not pre-rendered, are handled dynamically on the first request. This means the page is rendered using SSR, and Redis stores it to serve future requests as a cached version. Redis ensures consistency across containers and manages ISR revalidation.
So, if /cet or /gmt aren’t pre-rendered, they’ll SSR on the first load and initialize the cache through Redis.
Paper waspOP
/cet and /gmt are included in the generateStaticParams
const timeZones = ["cet", "gmt"];
export const revalidate = 60;
export async function generateStaticParams() {
return timeZones.map((timezone) => ({ timezone }));
}
@Alfonsus Ardani then replace /cet and /gmt with /act and /wib
Paper waspOP
what do you mean ?
Paper waspOP
The main thing I'm missing is what is supposed to happen for the /cet and /gmt pages, which are in the generateStaticParams and have a revalidate when using a custom cache.
---
as I understand it, when not using a custom pages, those page would be pre-rendered at build time and would end up in the .next/server/app/[timezone]/cet.html and gmt.html
this folder would be use as the cache folder by the ISR system and they would act as a "cache warmup", and when it is staled and visited, the background regeneration would recreate those pages
--
now with the custom cache handler, is it supposed to put the generated pages in the file system? in redis? neither?
what I seem to observe on the example is that it fails to put it in redis, and uses the fallback LRUcache at build time, but I think its not the intended behaviour because this cache is useless, since its written nowhere
---
as I understand it, when not using a custom pages, those page would be pre-rendered at build time and would end up in the .next/server/app/[timezone]/cet.html and gmt.html
this folder would be use as the cache folder by the ISR system and they would act as a "cache warmup", and when it is staled and visited, the background regeneration would recreate those pages
--
now with the custom cache handler, is it supposed to put the generated pages in the file system? in redis? neither?
what I seem to observe on the example is that it fails to put it in redis, and uses the fallback LRUcache at build time, but I think its not the intended behaviour because this cache is useless, since its written nowhere
It should put the generated pages in redis according to this -> https://nextjs.org/docs/app/building-your-application/deploying#caching-and-isr
yes if it fails then it will only be written in the memory which is basically useless when it comes to multiple different containers
but it still maintains the static caching of the site of that one particular container. so the ISR still works as long as the memory isn't purged
the example may uses LRU cache as its fallback method but you could configure it to keep storing it in a local directory
@Alfonsus Ardani yes if it fails then it will only be written in the memory which is basically useless when it comes to multiple different containers
Paper waspOP
ok so maybe I misunderstood something but to me the next build is supposed to happen when you create the image, not when you run the container, so when creating a new container from this image, you do not have access to the LRUcache used on the machine that build the image right ?
@Alfonsus Ardani the example may uses LRU cache as its fallback method but you could configure it to keep storing it in a local directory
Paper waspOP
yes that makes more sense to me
yeah that makes sense
the build process is typically in a different machine than when running the server
@Alfonsus Ardani the build process is typically in a different machine than when running the server
Paper waspOP
yes, for me that would be my CI runner and my ECS or K8S cluster respectively
Paper waspOP
ok so:
- custom cache configured as [RedisCache, FileSystemCache]
- at build time, let Redis fail and generate the cache using the filesystem
- at run time, let Redis take back the cache
but to me it is still nearly as useless since the first request will not hit redis and won't use the filesystemcache aswell, and I'll get an SSR and fill redis, ignoring the filesystemcache
- custom cache configured as [RedisCache, FileSystemCache]
- at build time, let Redis fail and generate the cache using the filesystem
- at run time, let Redis take back the cache
but to me it is still nearly as useless since the first request will not hit redis and won't use the filesystemcache aswell, and I'll get an SSR and fill redis, ignoring the filesystemcache
or maybe I'm missing a mecanism that would copy the file system cache in redis at some point when running Next?
I don't really what is supposed to happen if it misses Redis, does it try the filesystem as fallback for misses or is it just if redis is down ?
why does the first request don't hit redis or use the filesystemcache if Redis is connected at runtime? i.e nothing wrong with redis at runtime?
Paper waspOP
because redis has no cache, since the only cache I generated is in the filesystem, but maybe it does hit the filesystemcache, I don't know if fallback means it tries both when one misses, or if it means that it uses filesystem only if redis is down
I'll try it and if the filesystemcache takes over I'll mark your lastcomment as the answer
@Paper wasp because redis has no cache, since the only cache I generated is in the filesystem, but maybe it does hit the filesystemcache, I don't know if fallback means it tries both when one misses, or if it means that it uses filesystem only if redis is down
I havent tried but logically, the entire application is re-initialized, including re-running the cache-handler which means that it will try to re-conenct with Redis, and will look at Redis's cache. If at build time it fails then it will re-render all the pages statically at request time (because the cache at redis is missing)
Answer
im interested to know as well, let me know the result if it doesn't bother you 👀
@Alfonsus Ardani I havent tried but logically, the entire application is re-initialized, including re-running the cache-handler which means that it will try to re-conenct with Redis, and will look at Redis's cache. If at build time it fails then it will re-render all the pages statically at request time <:this_is_fine:770004314634321971> (because the cache at redis is missing)
Paper waspOP
so I did some digging:
- at build time, it is intended to skip redis because of a condition on top of the onCreation code because of an issue: https://github.com/caching-tools/next-shared-cache/issues/284#issuecomment-1919145094
- The cache created at build time is useless in our example
The next 15 version of the cache which should be realease at some point is different: https://github.com/caching-tools/next-shared-cache/compare/canary...next15#diff-f2ee79ef87df26d581d3562d1d835f8478dfbaffc8f0dc1c035c11ad76eb7013
the check to switch at build time is no logger present so it was not backported OR it has been fixed, we'll see when it's released. The fallback on the LruHandler and the log makes more sense here as it is just a fallback and not working side by side, in case redis handler fails
I'll try to have a date to see if I wait for the release or check out the canary version and I'll get back to you to give you the build behavior but I suppose the build will act like the background regeneration and fill up the redis cache
- at build time, it is intended to skip redis because of a condition on top of the onCreation code because of an issue: https://github.com/caching-tools/next-shared-cache/issues/284#issuecomment-1919145094
- The cache created at build time is useless in our example
The next 15 version of the cache which should be realease at some point is different: https://github.com/caching-tools/next-shared-cache/compare/canary...next15#diff-f2ee79ef87df26d581d3562d1d835f8478dfbaffc8f0dc1c035c11ad76eb7013
the check to switch at build time is no logger present so it was not backported OR it has been fixed, we'll see when it's released. The fallback on the LruHandler and the log makes more sense here as it is just a fallback and not working side by side, in case redis handler fails
I'll try to have a date to see if I wait for the release or check out the canary version and I'll get back to you to give you the build behavior but I suppose the build will act like the background regeneration and fill up the redis cache