Streaming langchain agent output from fastAPI
Unanswered
Giant Angora posted this in #help-forum
Giant AngoraOP
I have created this custom stream in my /api/chat (api for useChat). The code looks like this:
The CustomStream is created like so using the AIStream:
The problem I am having is that when I console log the completion it is empty. I know that the response from the API works as the response alone returns a stream. I just need to use the AIStream because I am interested in the onCompletion callback. I think the problem lies in the way I parse the data in the AIStreamParser, but not sure.
import {
StreamingTextResponse,
AIStream,
type AIStreamParser,
type AIStreamCallbacksAndOptions,
} from "ai";
const response = await fetch(
"/fastapi/agent-stream/",
{
method: "POST",
headers: {
"Content-Type": "application/json",
},
body: JSON.stringify({
messages: messages,
}),
}
);*/
// Convert the response into a friendly text-stream
if (response) {
// This does not work
const stream = CustomStream(response, {
onStart: async () => {
console.log("Stream started");
},
async onCompletion(completion) {
console.log("completion", completion); // This is empty
// Upload completion to database;
},
});
return new StreamingTextResponse(stream);The CustomStream is created like so using the AIStream:
function parseCustomStreamEvents(): AIStreamParser {
return (data: string) => {
console.log("data", data);
// Return the processed data or a specific part of it as needed.
return data; // Or return a specific part of the data if needed.
};
}
function CustomStream(
res: Response,
cb?: AIStreamCallbacksAndOptions
): ReadableStream {
return AIStream(res, parseCustomStreamEvents(), cb);The problem I am having is that when I console log the completion it is empty. I know that the response from the API works as the response alone returns a stream. I just need to use the AIStream because I am interested in the onCompletion callback. I think the problem lies in the way I parse the data in the AIStreamParser, but not sure.