Add s3 for store image optimization NextJS Self-hosted
Unanswered
California pilchard posted this in #help-forum
California pilchardOP
Okay based on what we tell me (https://discord.com/channels/752553802359505017/752647196419031042/1266717554169352273) to fix my "memory leak" issue from optimized image, I need to setup a S3 service. But the question is how ? There is parameter to put in
next.config.js ?33 Replies
California pilchardOP
@fuma okay sry to ping u, but something I dont understand, CLoudinary or S3 is to store image optimized from Nextjs app right ? Not for uploading picture from user for example (I already have an S3 with my backend for that) ? Because even with Cloudinary I only find tutorial how upload file in Cloudinary but isnt the purpose here, I guess I need to tell to my nextjs app to save optimized image in s3 instead of local memory storage
because they also have features to optimize images, for example, https://aws.amazon.com/developer/application-security-performance/articles/image-optimization/
you can directly upload images to S3/Cloudinary, and reference their URLs instead.
Use the URL with image versions (supported on S3 and Cloudinary) to ensure that everytime you update the image, it is also updated on Version Control like Git,
you can directly upload images to S3/Cloudinary, and reference their URLs instead.
Use the URL with image versions (supported on S3 and Cloudinary) to ensure that everytime you update the image, it is also updated on Version Control like Git,
Next.js Image component supported custom loader, please refer to the docs for more details
Another option would be, to use s3 as Origin and use a cdn to serve your (optimized) images. For that purpose I like to use bunnycdn. You enter the url to your origin (s3 bucket) and get a url to serve your files (click the „add optimization“ and you done)
@fuma because they also have features to optimize images, for example, https://aws.amazon.com/developer/application-security-performance/articles/image-optimization/
you can directly upload images to S3/Cloudinary, and reference their URLs instead.
Use the URL with image versions (supported on S3 and Cloudinary) to ensure that everytime you update the image, it is also updated on Version Control like Git,
California pilchardOP
Okay there is clearly something I dont understand. When u mean "you can directly upload image to S3". Are u talking about user upload a picture ? Or Nextjs storing image to render in S3 ? Sry its probably a dummy question but Im lost ahah
California pilchardOP
Because this tutorial for examlpe :
https://www.youtube.com/watch?v=twkImfEsnAs&ab_channel=Cloudinary
Only do image transformation right ? Like my container : https://github.com/coollabsio/next-image-transformation
But he dont store image... that's why I dont understand
https://www.youtube.com/watch?v=twkImfEsnAs&ab_channel=Cloudinary
Only do image transformation right ? Like my container : https://github.com/coollabsio/next-image-transformation
But he dont store image... that's why I dont understand
you manually upload the files to Cloudinary
from the dashboard
@fuma you manually upload the files to Cloudinary
California pilchardOP
Damn I feel so stupid ahah, what are u calling dashboard ? And manually upload the files ? What that means. Because I was thinking my memory leak are because NextJS save "rendered" image in memory (I think this is the part I missunderstood, bc doesnt make sense to save every picture rendered when these picture are already store in storage)...
I mean when using the cloud storage, you use the dashboard to upload files directly to the cloud
Because I was thinking my memory leak are because NextJS save "rendered" image in memoryYour question already said it's to use "S3" so Next.js no longer takes space in image optimization
@fuma > Because I was thinking my memory leak are because NextJS save "rendered" image in memory
Your question already said it's to use "S3" so Next.js no longer takes space in image optimization
California pilchardOP
But my image are already store in s3 (with SUpabase)
then you're just referencing your images with URLs and use Next.js Image custom loader + disable optimization by Next.js
@fuma then you're just referencing your images with URLs and use Next.js Image custom loader + disable optimization by Next.js
California pilchardOP
so I dont have any solution instead of disable optimization ?
disable it on Next.js, do it on your cloud storage
California pilchardOP
Im using S3 from Cloudflare (pretty cheap for me) I dont think there is image optimization by default, but what is difference making the optimization with my custom loader and my image transformation server directly in Nextjs. I dont even understand how Nextjs got memory leak
Next.js has their own image optimization endpoint, you can disable it from next.config and fully use S3 to optimized images (e.g. setting a width on image url)
this is all you need to know for image optimization with S3
@fuma Next.js has their own image optimization endpoint, you can disable it from next.config and fully use S3 to optimized images (e.g. setting a width on image url)
California pilchardOP
When I add
unoptimized: true in next.config.js my entire app is broken and I got so many error :[Error: EMFILE: too many open files, open '/tmp/pubkey.bc/Aa754617eb82474122609/rdrs/8ui1yyz3xro.json'] {
errno: -24,
code: 'EMFILE',
syscall: 'open',
path: '/tmp/pubkey.bc/Aa754617eb82474122609/rdrs/8ui1yyz3xro.json'
}
[Error: EMFILE: too many open files, open '/tmp/pubkey.bc/Aa754617eb82474122609/rdrs/8ui1yyz3xro.json'] {
errno: -24,
code: 'EMFILE',
syscall: 'open',
path: '/tmp/pubkey.bc/Aa754617eb82474122609/rdrs/8ui1yyz3xro.json'
}
[Error: EMFILE: too many open files, scandir '/tmp/pubkey.bc'] {
errno: -24,
code: 'EMFILE',
syscall: 'scandir',
path: '/tmp/pubkey.bc'
}make a wrapper for Next.js Image component, specify a loader for S3/Cloudinary, you may opt-out of image optimization manually to ensure the config is working
@fuma it's unrelated to image optimization, clean the .next folder and re-run
California pilchardOP
Okay, so how manage multiple image source ? My app is a movie app, every poster are not store in my S3, they are already store in public url. But user image (profile, playlist, ...) are in my s3 with my custom domain name. How the loader gonna know what to do depending of domain ?
You can just check the src url
@B33fb0n3 Another option would be, to use s3 as Origin and use a cdn to serve your (optimized) images. For that purpose I like to use bunnycdn. You enter the url to your origin (s3 bucket) and get a url to serve your files (click the „add optimization“ and you done)
California pilchardOP
I think I dont really understand what
optimized image really do in NextJS. BEcause my next-image-transformation container take an url and return the image with good size. So for Nextjs, what is the difference having an unoptimized url, or another URL optimized, he just gonna render the image returned ? I dont know why there is memory leak issue@California pilchard I think I dont really understand what `optimized` image really do in NextJS. BEcause my next-image-transformation container take an url and return the image with good size. So for Nextjs, what is the difference having an unoptimized url, or another URL optimized, he just gonna render the image returned ? I dont know why there is memory leak issue
render is render and optimization is optimization. So the "optimaztion" part is where the file will be transformed in a better format, maybe change the size of the file and so on. The browser then just renders it
@B33fb0n3 render is render and optimization is optimization. So the "optimaztion" part is where the file will be transformed in a better format, maybe change the size of the file and so on. The browser then just renders it
California pilchardOP
Yeah so why the NextJs app have memory issue, isnt the nextjs app who do the optimization, its should be my next-image-transformation (access with public domain name :
image.mydomaine.com) container no ?@B33fb0n3 U mean because of this?
California pilchardOP
Yes
@California pilchard Yes
I can only say what fuma already said, because it’s right: https://nextjs-forum.com/post/1266751561758150788#message-1266780282959171605
@B33fb0n3 I can only say what fuma already said, because it’s right: https://discord.com/channels/752553802359505017/1266751561758150788/1266780282959171605
California pilchardOP
If u talk about my errors, there are fix yeah I just forgot to rebuild the app without cache in my Coolify server mb 😅