"Module not found: Can't resolve 'fs'" when calling puppeteer scraping function through client compo
Answered
Crescent-chested Warbler posted this in #help-forum
Crescent-chested WarblerOP
I'm currently building a web scraper with NextJS, and to avoid the 10s API call limit, I decided to use useState and useEffect to loop through the stores, fetching them and updating the result state after each success store scrape instead of going through all of the stores at once. However, when changing to client component, this error appears. It seems like 'fs' was needed for puppeteer to works, but somehow when using client component, it doesn't have access to it. Any guidance on how to resolve this?
Answered by Crescent-chested Warbler
My scraper function is in another file. Surprisingly putting "use server" in my scraper.ts solves the whole thing.
2 Replies
American Crow
Well, fs is file system. Client Components don't have access to your servers file system (obviously). Hard to help without any code. You most likely want to split your code in two components. Do the puppeteer fetching on the server side, pass the data to a client component and continue from there
Crescent-chested WarblerOP
My scraper function is in another file. Surprisingly putting "use server" in my scraper.ts solves the whole thing.
Answer