Decoding vercel ai sdk stream on client (onResponse via useChat())
Unanswered
bplr posted this in #help-forum
bplrOP
Hi, trying to figure out how do i decode the chunks on the response of the vercel sdk streaming(StreamingTextResponse()).
I want to convert the Message type into another type that have more fields, any suggestion will be appreciated.
Now, when i try to get chunk by chunk and reconstruct a new object, it seems that the chunk is on non typical type, its like:
Each chunk starts with "0:'string'"
Here's my code:
Thank you for your time and help!
I want to convert the Message type into another type that have more fields, any suggestion will be appreciated.
Now, when i try to get chunk by chunk and reconstruct a new object, it seems that the chunk is on non typical type, its like:
Received full data: 0:"The"
0:" name"
0:" of"
0:" the"
0:" file"
0:" mentioned"
0:" in"
0:" the"
0:" context"
0:" is"
0:" **"
0:"\""
0:"Titan"
0:"ic"
0:".xlsx"
0:"\""
0:"**"
0:".\n\n"
0:"*"
0:"Page"
0:" Number"
0:":"
0:" Not"
0:" specified"
0:".*\n"
0:"*"
0:"File"
0:" Name"
0:" Refer"
0:"enced"
0:":"
0:" Titanic"
0:".xlsx"
0:"*"Each chunk starts with "0:'string'"
Here's my code:
const {
input,
stop,
metadata,
handleSubmit,
setInput,
handleInputChange,
setMessages,
isLoading: isChatLoading,
messages,
data: chatData,
} = useChat({
api: "/api/chat",
body: {
workspaceId,
currentChatId: currentChat?.uuid,
isContinue,
},
// dont call api if there are no messages
initialMessages: data || [],
onResponse: async (response) => {
if (response.body) {
const reader = response.body.getReader();
const textDecoder = new TextDecoder("utf-8");
let receivedLength = 0; // Length of received data
let chunks = []; // Array to hold received chunks
let currentMessage = ""; // Store accumulated message
while (true) {
const { done, value } = await reader.read();
if (done) {
console.log("Stream completed");
break;
}
chunks.push(value);
receivedLength += value.length;
let currentChunk = textDecoder.decode(value, { stream: true });
currentMessage += currentChunk;
setCurrentMessage(currentMessage); // Assuming `setCurrentMessage` updates React state or similar
console.log("Type of currentChunk:", typeof currentChunk);
console.log("Is currentChunk an array?", Array.isArray(currentChunk));
console.log("Current chunk: ", currentChunk);
// Assuming `processChunk` processes each chunk as it arrives
processChunk({
id: "",
type: "message",
content: currentMessage,
tool_call_id: "non-interpreter",
activeLine: undefined,
role: "system",
start: false,
end: false,
});
}
const fullData = new TextDecoder().decode(
new Uint8Array(
chunks.reduce((acc: any, val) => acc.concat(Array.from(val)), [])
)
);
// After the loop, flush the stream to get any trailing data
currentMessage += textDecoder.decode();
console.log("Received full data:", fullData);
setIsStreamLoading(false); // Assuming you're controlling loading state in a React component
setIsContinue(false); // Similarly, updating UI control states
}
},
});Thank you for your time and help!