Next.js Discord

Discord Forum

Streaming langchain agent output from fastAPI

Unanswered
Giant Angora posted this in #help-forum
Open in Discord
Giant AngoraOP
I have created this custom stream in my /api/chat (api for useChat). The code looks like this:
import {
  StreamingTextResponse,
  AIStream,
  type AIStreamParser,
  type AIStreamCallbacksAndOptions,
} from "ai";

const response = await fetch(
    "/fastapi/agent-stream/",
    {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
      },
      body: JSON.stringify({
        messages: messages,
      }),
    }
  );*/
  // Convert the response into a friendly text-stream
  if (response) {
    // This does not work
    const stream = CustomStream(response, {
      onStart: async () => {
        console.log("Stream started");
      },
      async onCompletion(completion) {
        console.log("completion", completion); // This is empty

        // Upload completion to database;
      },
    });
return new StreamingTextResponse(stream);

The CustomStream is created like so using the AIStream:
function parseCustomStreamEvents(): AIStreamParser {
  return (data: string) => {
    console.log("data", data);
    // Return the processed data or a specific part of it as needed.
    return data; // Or return a specific part of the data if needed.
  };
}

function CustomStream(
  res: Response,
  cb?: AIStreamCallbacksAndOptions
): ReadableStream {
  return AIStream(res, parseCustomStreamEvents(), cb);

The problem I am having is that when I console log the completion it is empty. I know that the response from the API works as the response alone returns a stream. I just need to use the AIStream because I am interested in the onCompletion callback. I think the problem lies in the way I parse the data in the AIStreamParser, but not sure.

0 Replies