Vercel AI SDK code working locally but not when deployed
Unanswered
Peruvian anchoveta posted this in #help-forum
Peruvian anchovetaOP
I followed the Vercel AI SDK getting started section, made some modifications based on my use cases, and am running into a problem where when I run my next.js app locally, the request to open ai api (using vercel openai) is successful, but when deploying, the request to openai never comes back. I've tried adding try/catch blocks, console logs, and cant seem to get any error back. Im using edge runtime.
This is my code,
async function getGpt4ResponseStreamed(prompt) {
try {
console.log("Prompt in getGPT4Response: ", prompt);
let response;
try {
response = await openai.createChatCompletion({
model: "gpt-4",
stream: true,
messages: [{role: "user", content: prompt}],
});
} catch (err) {
console.log('Error while calling openai.createChatCompletion: ', err);
}
console.log("RESPONSE in getGpt4 response: ", response);
const stream = OpenAIStream(response);
console.log("STREAM: ** : ", stream);
let data = '';
for await (const chunk of stream) {
console.log("Received chunk: ", chunk);
data += new TextDecoder('utf-8').decode(chunk);
}
console.log("Data from openai api in getGPT4response: ", data);
return data;
} catch (err) {
console.log('Error while getting GPT4 response: ', err);
}
when running locally I get the open ai response successfully, but when deploying on vercel, I dont see anything in the vercel logs except that this log:
console.log("Prompt in getGPT4Response: ", prompt);
Is that last place it reaches on the code. I have no idea why this is happening, any help would be greatly appreciated. Thanks in advance!
This is my code,
async function getGpt4ResponseStreamed(prompt) {
try {
console.log("Prompt in getGPT4Response: ", prompt);
let response;
try {
response = await openai.createChatCompletion({
model: "gpt-4",
stream: true,
messages: [{role: "user", content: prompt}],
});
} catch (err) {
console.log('Error while calling openai.createChatCompletion: ', err);
}
console.log("RESPONSE in getGpt4 response: ", response);
const stream = OpenAIStream(response);
console.log("STREAM: ** : ", stream);
let data = '';
for await (const chunk of stream) {
console.log("Received chunk: ", chunk);
data += new TextDecoder('utf-8').decode(chunk);
}
console.log("Data from openai api in getGPT4response: ", data);
return data;
} catch (err) {
console.log('Error while getting GPT4 response: ', err);
}
when running locally I get the open ai response successfully, but when deploying on vercel, I dont see anything in the vercel logs except that this log:
console.log("Prompt in getGPT4Response: ", prompt);
Is that last place it reaches on the code. I have no idea why this is happening, any help would be greatly appreciated. Thanks in advance!
5 Replies
@Peruvian anchoveta according to the Vercel official doc, Open AI's createCompletion does not work in Vercel Edge Runtime.
https://sdk.vercel.ai/docs/api-reference/openai-stream
https://sdk.vercel.ai/docs/api-reference/openai-stream
so Vercel says that the said SDK, OpenAI's SDK, uses axios, which does not work in Vercel Edge Runtime. thus Vercel seems provide openai-edge package.
Note: The official OpenAI API SDK does not yet support the Edge Runtime and will only work in serverless environments at the moment. The openai-edge package is based on fetch instead of axios (and thus works in the Edge Runtime) so we suggest using it instead.
or you can use Serverless(AWS Lambada) instead of Edge Runtime.
or this would be better. you can get the most out of Cloudflare Edge streaming. https://vercel.com/blog/introducing-the-vercel-ai-sdk
disclaimer. I have not tried any of the above. I just read docs only.