Next.js Discord

Discord Forum

How can I use edge functions here?

Unanswered
Semipalmated Plover posted this in #help-forum
Open in Discord
Semipalmated PloverOP
I have this code working with ChatGPT API:
const response = await openai.chat.completions.create({
    model: "gpt-4",
    messages: [
      {
        role: "user",
        content: prompt,
      },
    ],
  });
  return response.choices[0].message.content;


However, it takes too long. I am aware of the method using Vercel's "ai" package which features streaming, but my app is too complex for me to integrate it, and I don't need streaming.

I put export const runtime = 'edge'; in my endpoint file, but I'm still running into 405 error when deployed to vercel.

18 Replies

European sprat
if your request takes a while then you have to use streaming
or don't use edge/vercel that have limits on response time
@European sprat if your request takes a while then you have to use streaming
Semipalmated PloverOP
I don't understand why streaming is required in order to use edge functions
& yeah, all I'm trying to do is not have the response time limit
European sprat
it isn't required
if your response happens within the period of time that vercel allows for an edge function to execute then there's no problem
So, streaming is kind of like a bunch of smaller requests?
Or is it a websocket?
I’ve been trying to read the docs but this stuff is hard
European sprat
non-streaming, you wait the for the whole request to finish then receive it
streaming, you get one small piece at a time until it's done
so to get that first small piece it takes a short amount of time
Semipalmated PloverOP
Roger that
So, on my frontend, I basically show a loading icon until my backend gets the chatgpt response
Is there some kind of indicator that I can pass to my frontend to say, “All done streaming now!”
So that way I can show the chatgpt response without showing it streaming
European sprat
certainly there's ways to do it
show the loading indicator while it's streaming then hide the loading indicator when streaming is done and display the text from the variable which received all the streamed data