Next.js Discord

Discord Forum

Decoding vercel ai sdk stream on client (onResponse via useChat())

Unanswered
bplr posted this in #help-forum
Open in Discord
Hi, trying to figure out how do i decode the chunks on the response of the vercel sdk streaming(StreamingTextResponse()).

I want to convert the Message type into another type that have more fields, any suggestion will be appreciated.
Now, when i try to get chunk by chunk and reconstruct a new object, it seems that the chunk is on non typical type, its like:

Received full data: 0:"The"
0:" name"
0:" of"
0:" the"
0:" file"
0:" mentioned"
0:" in"
0:" the"
0:" context"
0:" is"
0:" **"
0:"\""
0:"Titan"
0:"ic"
0:".xlsx"
0:"\""
0:"**"
0:".\n\n"
0:"*"
0:"Page"
0:" Number"
0:":"
0:" Not"
0:" specified"
0:".*\n"
0:"*"
0:"File"
0:" Name"
0:" Refer"
0:"enced"
0:":"
0:" Titanic"
0:".xlsx"
0:"*"


Each chunk starts with "0:'string'"

Here's my code:

const {
    input,
    stop,
    metadata,
    handleSubmit,
    setInput,
    handleInputChange,
    setMessages,
    isLoading: isChatLoading,
    messages,
    data: chatData,
  } = useChat({
    api: "/api/chat",
    body: {
      workspaceId,
      currentChatId: currentChat?.uuid,
      isContinue,
    },

    // dont call api if there are no messages
    initialMessages: data || [],
    onResponse: async (response) => {
      if (response.body) {
        const reader = response.body.getReader();
        const textDecoder = new TextDecoder("utf-8");
        let receivedLength = 0; // Length of received data
        let chunks = []; // Array to hold received chunks
        let currentMessage = ""; // Store accumulated message

        while (true) {
          const { done, value } = await reader.read();
          if (done) {
            console.log("Stream completed");
            break;
          }
          chunks.push(value);
          receivedLength += value.length;
          let currentChunk = textDecoder.decode(value, { stream: true });
          currentMessage += currentChunk;

          setCurrentMessage(currentMessage); // Assuming `setCurrentMessage` updates React state or similar
          console.log("Type of currentChunk:", typeof currentChunk);
          console.log("Is currentChunk an array?", Array.isArray(currentChunk));
          console.log("Current chunk: ", currentChunk);
          // Assuming `processChunk` processes each chunk as it arrives
          processChunk({
            id: "",
            type: "message",
            content: currentMessage,
            tool_call_id: "non-interpreter",
            activeLine: undefined,
            role: "system",
            start: false,
            end: false,
          });
        }
        const fullData = new TextDecoder().decode(
          new Uint8Array(
            chunks.reduce((acc: any, val) => acc.concat(Array.from(val)), [])
          )
        );

        // After the loop, flush the stream to get any trailing data
        currentMessage += textDecoder.decode();
        console.log("Received full data:", fullData);
        setIsStreamLoading(false); // Assuming you're controlling loading state in a React component
        setIsContinue(false); // Similarly, updating UI control states
      }
    },
  });


Thank you for your time and help!

0 Replies