where should I host this large json file that my web app depends on?
Answered
Bonga shad posted this in #help-forum
Bonga shadOP
Hi folks, I am building an nextjs app which has a map component and it renders a geojson layer on the base map. The geojson file is pretty big (5MB)
Currently I simply wrapped the json content as a JavaScript variable. As you can imagine, bundle size for the map component is 5MB+. This is obviously not ideal. Not only it makes the website slow to load; it also significantly increases the bandwidth consumption because each time a user loads the page, it downloads 5MB from next js server. Over time, it could lead to unnecessary host cost.
I would like some tips around how I can best serve this geojson file. I have only some general ideas and really struggle because my lack of experience in web development:
- put the geojson file in ‘public’ folder as static asset. However I think this doesn’t make much difference because it would still need to be downloaded from nextjs server.
- use SWR or useEffect() to load the file from S3 when the website is loaded. I am just afraid that downloading 4MB from S3 would significantly slow down the loading process. Should I be concerned?
- Put it in ‘public’ and use some sort of CDN solution. I never used CDN myself, my concern is that it might be expensive if my site is maliciously loaded by a bot and needs to fetch from CDN too often.
Could anyone shed some light which direction I should pursue?
Thanks.
Currently I simply wrapped the json content as a JavaScript variable. As you can imagine, bundle size for the map component is 5MB+. This is obviously not ideal. Not only it makes the website slow to load; it also significantly increases the bandwidth consumption because each time a user loads the page, it downloads 5MB from next js server. Over time, it could lead to unnecessary host cost.
I would like some tips around how I can best serve this geojson file. I have only some general ideas and really struggle because my lack of experience in web development:
- put the geojson file in ‘public’ folder as static asset. However I think this doesn’t make much difference because it would still need to be downloaded from nextjs server.
- use SWR or useEffect() to load the file from S3 when the website is loaded. I am just afraid that downloading 4MB from S3 would significantly slow down the loading process. Should I be concerned?
- Put it in ‘public’ and use some sort of CDN solution. I never used CDN myself, my concern is that it might be expensive if my site is maliciously loaded by a bot and needs to fetch from CDN too often.
Could anyone shed some light which direction I should pursue?
Thanks.
Answered by joulev
use SWR or useEffect() to load the file from S3 when the website is loaded. I am just afraid that downloading 4MB from S3 would significantly slow down the loading process. Should I be concerned?
this is what i would use. you have to download 5 MB to the browser anyway, so regardless of where you store the file the user still has to wait for the download.
you can add cache-control headers to the file response (check the storage provider for how to do that) then repeated visitors to the page will reuse the file already cached in their browser, bypassing the download.
don't store this file in
public if you host on vercel. requests to static files are counted towards the total bandwidth and in many hosting providers, the bandwidth is limited/costly. store the file in a place where read bandwidth is not expensive, like a object storage service, is a better idea.4 Replies
@Bonga shad Hi folks, I am building an nextjs app which has a map component and it renders a geojson layer on the base map. The geojson file is pretty big (5MB)
Currently I simply wrapped the json content as a JavaScript variable. As you can imagine, bundle size for the map component is 5MB+. This is obviously not ideal. Not only it makes the website slow to load; it also significantly increases the bandwidth consumption because each time a user loads the page, it downloads 5MB from next js server. Over time, it could lead to unnecessary host cost.
I would like some tips around how I can best serve this geojson file. I have only some general ideas and really struggle because my lack of experience in web development:
- put the geojson file in ‘public’ folder as static asset. However I think this doesn’t make much difference because it would still need to be downloaded from nextjs server.
- use SWR or useEffect() to load the file from S3 when the website is loaded. I am just afraid that downloading 4MB from S3 would significantly slow down the loading process. Should I be concerned?
- Put it in ‘public’ and use some sort of CDN solution. I never used CDN myself, my concern is that it might be expensive if my site is maliciously loaded by a bot and needs to fetch from CDN too often.
Could anyone shed some light which direction I should pursue?
Thanks.
use SWR or useEffect() to load the file from S3 when the website is loaded. I am just afraid that downloading 4MB from S3 would significantly slow down the loading process. Should I be concerned?
this is what i would use. you have to download 5 MB to the browser anyway, so regardless of where you store the file the user still has to wait for the download.
you can add cache-control headers to the file response (check the storage provider for how to do that) then repeated visitors to the page will reuse the file already cached in their browser, bypassing the download.
don't store this file in
public if you host on vercel. requests to static files are counted towards the total bandwidth and in many hosting providers, the bandwidth is limited/costly. store the file in a place where read bandwidth is not expensive, like a object storage service, is a better idea.Answer
@joulev > use SWR or useEffect() to load the file from S3 when the website is loaded. I am just afraid that downloading 4MB from S3 would significantly slow down the loading process. Should I be concerned?
this is what i would use. you have to download 5 MB to the browser *anyway*, so regardless of where you store the file the user still has to wait for the download.
you can add cache-control headers to the file response (check the storage provider for how to do that) then repeated visitors to the page will reuse the file already cached in their browser, bypassing the download.
don't store this file in `public` if you host on vercel. requests to static files are counted towards the total bandwidth and in many hosting providers, the bandwidth is limited/costly. store the file in a place where read bandwidth is not expensive, like a object storage service, is a better idea.
Common Murre
I agree on that. I also made some GIS pages and I put the geojson on S3 or my other server that has unlimited bandwidth but higher latency since loading the json itself doesnt need to be that snappy to keep the UX great
yeah just render the loading screen, useswr has good support for this
Bonga shadOP
Thank you both.