You can try streaming but I highly suspect that it will also suffer with timeouts but I've never seen the gpt's api take over 15s to generate response.

I can share you are repo if you would like to check

yes I would love to read so I can avoid this situation in the near future

you can also use the custom server in route handlers

Code is kinda mess cuz project was made in 2 days

But yeah

How do I do that

This was the thing in pages i think

don't know how to do it in app

This works fine locally but fails in production. Error is in "/platform" - timeout after 10s

Just skimmed through your repo. Maybe you want to look into something like https://sdk.vercel.ai/docs/ai-sdk-rsc. That way you'd get server side streaming from and to the LLM out of the box. You could move everything to RSC, don't have to call your own LLM endpoints client side anymore. From the looks of it your project might also be a great use case for generative UI. Just a friendly suggestion

Gonna try it for sure. Any feedback is welcome and i am trully grateful that you took your time. Thank you

I implemented it in a project like 2 weeks ago. The paradigms/api is different so in my case i needed two days to understand it. However having a server action which communicates with (any) LLM including function and tool calling out of the box in the case of openAI. Getting the response and the option to stream JSX (not just text) to the client is really powerful

Another topic kinda related to this one but if you might now. What is the best/easiest/performant way to convert pdf/docx to text so i can give it to LLM as prompt

Also this ai sdk is neat

Not my area of expertise sorry. I'd most likely check out how langchain does that as a first action

?