Next.js Discord

Discord Forum

Best practice to fetch and show big data

Unanswered
Sun bear posted this in #help-forum
Open in Discord
Sun bearOP
Hello there! I'm fetching data from api, there is array with 4K+ length, I need sort it and get info about each object inside. I can show part of array with pagination but anyway I need get all data `cose user use ui to find correct one.

36 Replies

Can I know exactly you need help with here?
Netherland Dwarf
Also is this api from your own servr? Because then you can add query params to allow pagination and sorting and do that in the backend similar to most api
@Anay-208 Can I know exactly you need help with here?
Sun bearOP
My ways are:
- Fetch data from api (not mine) on client side. It is array with 4k+ objects.

- Pass each objects in the client component. Client component has search functionality.

- When user call this component to show all 4k objects and use search to find one it took so long to fetch data and search.

It isn't problem to make pagination it will increase ux and decrease load all object but anyway user wait when all data will be available to make correct search.
I'm thinking about server side render to fetch all data.
@Anay-208 @Netherland Dwarf
Its best to search in server side, just give a fetch Req to server side
@Anay-208 Its best to search in server side, just give a fetch Req to server side
Sun bearOP
Thanks! But how be with component, because I'm using this component in client side?
in client side, fetch from api again
or server action
@Anay-208 in client side, fetch from api again
Sun bearOP
oh, you mean make my own api over external api?
within nextjs, I make my own api(or route.ts) or server action
@Anay-208 within nextjs, I make my own api(or route.ts) or server action
Sun bearOP
let me try, thank you!
Hey there @Sun bear, any updates regarding this issue?
Blue orchard bee
Just to expand on anay-208, Next.js is a fullstack framework, not only a client-side one.

You can define an API route in which, theoretically, you can request and cache the response of the data you need on the server, and thus, you can split it into chunks (say pages), so you can consume it client-side in parts.

https://nextjs.org/docs/app/building-your-application/data-fetching/fetching-caching-and-revalidating#fetching-data-on-the-server-with-fetch
@Anay-208 Hey there <@401391144786722817>, any updates regarding this issue?
Sun bearOP
Still working on it, will try tomorrow
Sun bearOP
@Anay-208 here is update, I have made server side fetching. But my fetching is just using some lib to get data, so I can't update fetched data like if I've fetched from classic api fetch('https://...', { next: { revalidate: 3600 } })
@Saint Hubert Jura Hound Maybe look into tanstack react-query? U can revalidate using it
Blue orchard bee
Yea that could work but then he'd receive 4k records client-side, if the application goes over a fast connection that's fine imo.
So if your data becomes outdated (because you created, edited or deleted an entry), you can revalidate the tag associated with that request.
@Blue orchard bee Yea that could work but then he'd receive 4k records client-side, if the application goes over a fast connection that's fine imo.
Blue orchard bee
Also depends on what these records hold, it's different between 4-5 fields to 10-15 depending of the data they're handling.
Sun bearOP
Thank you guys, let me check
Sun bearOP
so I don't use fetch, there is just lib that use it (I guess) so not sure how to revalidate data on this case. Thinking about moving getValidators to backend api route
Blue orchard bee
You can use a cache function for this
And if you need to invalidate this, you can use revalidatePath
@Blue orchard bee And if you need to invalidate this, you can use revalidatePath
Sun bearOP
hm, may I ask ? In which cases using this for invalidate data? For example
Blue orchard bee
Imagine you just edited the data, with a form, or created a new entry.
If you don't invalidate, then the data will be the same for an hour.
Sun bearOP
So thank you all! My solution is: using pagination or load more button to show all items, and inside cache use lib to get data and inside server component add revalidate time! Looks good, isn't it?
@Blue orchard bee Imagine you just edited the data, with a form, or created a new entry.
Sun bearOP
oh, it is look like google form, when I return to my form there are data in the filed that I've added
@Sun bear oh, it is look like google form, when I return to my form there are data in the filed that I've added
Blue orchard bee
I don't know much about Google Forms, but if you can add a secret key to Google Forms and call that function to your Next.js API, you can invalidate the data after new one is added.

Without it, the data would be updated depending of your revalidateTime.
If that answers your question don't forget to mark it as solved!