Scaling app in kubernetes
Unanswered
English Angora posted this in #help-forum
English AngoraOP
Hey guys,
how do you manage to scale your apps in kubernetes deployments? I am wondering what setup is the best as it starts to be a pain for me.
What works for me the best right now is a resources CPU set to request 1vCPU and limit also set to 1vCPU, scaling set to 80% utilization of request value. RAM does not matter right now. When nextjs starts I run a standalone version in cluster mode but with processes count set to 1.
Now I have a problem to determine if nextjs can handle more than 1vCPU because when I ran stress tests on an instance with a limit set to 2vCPU it died at the same level as an instance with 1vCPU (no scaling).
Do you guys have some blueprints for beginners how to setup nextjs on kubernetes deployments with scaling by CPU resources
? Can nextjs handle multiple cores?
how do you manage to scale your apps in kubernetes deployments? I am wondering what setup is the best as it starts to be a pain for me.
What works for me the best right now is a resources CPU set to request 1vCPU and limit also set to 1vCPU, scaling set to 80% utilization of request value. RAM does not matter right now. When nextjs starts I run a standalone version in cluster mode but with processes count set to 1.
Now I have a problem to determine if nextjs can handle more than 1vCPU because when I ran stress tests on an instance with a limit set to 2vCPU it died at the same level as an instance with 1vCPU (no scaling).
Do you guys have some blueprints for beginners how to setup nextjs on kubernetes deployments with scaling by CPU resources
? Can nextjs handle multiple cores?3 Replies
English AngoraOP
I can add that we try to scale horizontally by adding new pods but is it a good way to go to production?
Vizsla
Autoscaling based on CPU set at 80% is pretty standard so you should be good there. I've seen this setup at some fairly large production, self-hosted NextJS infrastructures.
The only issue you run into horizontally scaling NextJS in pods is caching. Two things here:
1) Static assets
2) Serverless functions
1 and 2 will be cached only for a particular pod - so if you start horizontally scaling, calls to the newly scaffolded pods will NOT serve cached assets. You can address this in a few ways
1) On deploys, extract the static build assets and manually deploy to S3 + CloudFront CDN, update next config to point to CloudFront CDN URL
2) NextJS uses the container's filesystem to store functions cache, update your k8's config to use a durable storage (dynamoDB/redis) instead
Does that make sense? Of course all of this infra comes out of the box with Vercel, pro tier + overages can get you pretty far
The only issue you run into horizontally scaling NextJS in pods is caching. Two things here:
1) Static assets
2) Serverless functions
1 and 2 will be cached only for a particular pod - so if you start horizontally scaling, calls to the newly scaffolded pods will NOT serve cached assets. You can address this in a few ways
1) On deploys, extract the static build assets and manually deploy to S3 + CloudFront CDN, update next config to point to CloudFront CDN URL
2) NextJS uses the container's filesystem to store functions cache, update your k8's config to use a durable storage (dynamoDB/redis) instead
Does that make sense? Of course all of this infra comes out of the box with Vercel, pro tier + overages can get you pretty far
English AngoraOP
Thanks for the feedback.
Our deployment is pretty non-standard I must say. We are building one image for all our environments (so basically we do not use SSG) but when the pod is initializing we are doing revalidation on all pages to make it work for different front environment variables. The setup works perfectly, as we can use one image in various places without building the new app specially per environment. We know that it takes a little bit more time than it should for the pod to be ready for traffic (it needs to finish revalidation, and then we allow the traffic to go in).
We also use redis as a shared cache (we are still using the pages directory in nextjs v13 - we plan to move to the app directory in the near future).
The issue is for me to set correct resources for the app as the stress tests shows that it can use more that 1000mi vCPU but the performance is the same for 1000mi and 2000mi.
I was also wondering if cluster mode shouldn't spawn count of processes according to the number of vCores but it should do it dynamically somehow (on pod startup I am spawning only one process because the request resources values ale 1vCPU, when it rises to 2vCPU I dont have dynamic possibility to add another process as the function for cluster mode is executed only on pod start)
Our deployment is pretty non-standard I must say. We are building one image for all our environments (so basically we do not use SSG) but when the pod is initializing we are doing revalidation on all pages to make it work for different front environment variables. The setup works perfectly, as we can use one image in various places without building the new app specially per environment. We know that it takes a little bit more time than it should for the pod to be ready for traffic (it needs to finish revalidation, and then we allow the traffic to go in).
We also use redis as a shared cache (we are still using the pages directory in nextjs v13 - we plan to move to the app directory in the near future).
The issue is for me to set correct resources for the app as the stress tests shows that it can use more that 1000mi vCPU but the performance is the same for 1000mi and 2000mi.
I was also wondering if cluster mode shouldn't spawn count of processes according to the number of vCores but it should do it dynamically somehow (on pod startup I am spawning only one process because the request resources values ale 1vCPU, when it rises to 2vCPU I dont have dynamic possibility to add another process as the function for cluster mode is executed only on pod start)