Scalable and performant ways would be Elasticsearch or vector search engines like Weaviate. Next.js runs on Node.js runtime, which is single thread event loop, so CPU intensive tasks should be avoided.

Let me rephrase then... As performant and scalable as you can without having to integrate with other services... So on the node.js runtime

where do u host?

So far it's still not deployed. Just localhost for now while learning how to work with next.js 🙂
Just curious what the best practice way is, thinking in node.js and not external infrastructure.

i see. Let's start with Vercel hobby plan so you will get idea.

By integrating other services you mean you can have a database though?

where are your data?

anyway option 1 seems totally fine in your context

since the filters fit an URL, you can user server-side rendering with RSC if that makes sense in your context, you don't even need a separate API endpoint

Note that requests that include auth tokens on Vercel can't be cached

so if you use an API endpoint, make sure that it's an endpoint that returns the same data for all users and do not take cookies or whatever into account, so the result for a given filter can be cached

anyway the first step would be to implement option 1, and then only see what you can improve incrementally

there is no single best practice

Thanks for trying to help, both of you.
Right now I don't even worry about where data is hosted. I made an API endpoint /products and just store it locally in a json file. This project is all about learning the "code part" about Next.js App Router and RSC.
It will eventually be protected and it could potentially be different depending on what user is logging in. Not right now but the solution should be able to handle it.
Also I don't like sending API requests to the /products endpoint every time a user is changing his filters... That relies way too much on next.js cache and its just not optimal to refetch products every time the filters changes. I know the examples and tutorials online did it this way, but isn't there a better way to do this, just with Next.js and Node.js (no elasticsearch etc.)
The database is not relevant here. I'm only concerned with fetch from the API endpoint and how changing filters should change the visible products in the smartest (best practice) way as possible. Best I can think of would be if all products was fetched from the API, they all would be accessible in memory in the server component and then the client component could ask the server component for different products depending on the filters... I would not know how to implement that solution though

By that I mean all products would be fetched from /api/products just the first time the page loads... The user would the change the filters from the client component and get a subset of products that matches the filter options he chose... The querystring params would be updated just to keep state in the url, but without reloading the page or refetching... Also the url params would not be part of the API request, but only for the client component to request s subset of the data from the server component.

Does that make sense, and is there any example of some similar solution you can point me to? Or any help to get started?

I understand your context but the thing is that the best architecture really depends on the whole picture, in your case there is no wrong choice so you can try whatever approach that you find insightful

Also I don't like sending API requests to the /products endpoint every time a user is changing his filters

I don't really get this, this is the point of an API. You can put whatever sort of cache in front of it (server-side to share values between user and local so if the user reuse the same filter they have already the result in memory) and call it a day

The database is not relevant here.

This is more related to the CPU spike, if the data are in-memory as a JSON, filtering will happen on your Node server as explained by @tafutada777 and this is not smth you want at scale
But if you use a database, it will handle the filtering so it's not the case anymore, the filtering happen on a different system

By that I mean all products would be fetched from /api/products just the first time the page loads...

That's ok for a demo or short list of product, but to me its irrelevant at scale

Does that make sense, and is there any example of some similar solution you can point me to?

If you load all the products client-side and filter there, then that's pure React and JS, you won't use the Next.js APIs much I think

I think the most relevant example is Next Commerce if you want to see an official approach