Next.js Discord

Discord Forum

Help with Vercel AI SDK

Unanswered
Ross's Goose posted this in #help-forum
Open in Discord
Avatar
Ross's GooseOP
Hello! I'm working on a chat application using a custom AI agent, but I'm facing an issue where the assistant message is not being generated. I've implemented a chat function in my Agent class, which uses a streamText function to process messages. However, the streamText function doesn't seem to be returning any text, resulting in an empty assistant message. Below is the chat function code:

3 Replies

Avatar
Ross's GooseOP
async *chat(
    chatId: string,
    userId: string,
    latestMessage: string,
    tools: Tool[],
    temperature: number,
    maxTokens: number,
    csvAnalysis?: CSVAnalysis
): AsyncGenerator<StreamChunk> {
    const chatHistory = await this.fetchChatHistory(chatId)
    console.log('Chat History for chatId:', chatId, chatHistory)
    const userMessage: CoreUserMessage = {
        role: 'user',
        content: latestMessage,
    }

    const sanitizedMessages = this.prepareMessagesForAI(chatHistory)
    sanitizedMessages.push(userMessage)

    console.log('Sanitized Messages:', sanitizedMessages)

    const toolsRecord: Record<string, CoreTool<any, any>> = tools.reduce(
        (acc, tool) => {
            acc[tool.name] = tool // Map tool name to tool
            return acc
        },
        {} as Record<string, CoreTool<any, any>>
    )

    const modelClient = getModelClient(this.model, this.config)

    let currentMessage: any = null
    let accumulatedResponse = ''
    let toolCalls: ToolCallPart[] | null = null
    let toolResults: ToolResult[] | null = null

    const stream = await streamText({
        model: modelClient,
        system: this.roleDescription,
        messages: sanitizedMessages,
        maxTokens: maxTokens,
        maxSteps: 10,
        tools: toolsRecord,
        onStepFinish: async ({
            text,
            toolCalls: stepToolCalls,
            toolResults: stepToolResults,
        }) => {
            console.log('Step finished with text:', text)
            if (text) {
                accumulatedResponse += text
            }

            if (stepToolCalls && stepToolCalls.length > 0) {
                toolCalls = toolCalls || []
                toolCalls.push(...stepToolCalls)
            }

            if (stepToolResults && stepToolResults.length > 0) {
                toolResults = toolResults || []
                toolResults.push(...stepToolResults)
            }
        },
    })
    // Yield the accumulated response
    yield {
        type: 'text_chunk',
        content: accumulatedResponse.trim(),
    } as StreamChunk

    console.log('Storing Message for chatId:', chatId)
    const totalTokenCount =
        this.outputTokens + this.codeTokens + this.inputTokens
    console.log(
        'Total Token Count (output, code, input):',
        this.outputTokens,
        this.codeTokens,
        this.inputTokens
    )

    await this.storeMessage(
        chatId,
        userId,
        latestMessage,
        accumulatedResponse.trim(),
        totalTokenCount,
        toolCalls,
        toolResults
    )
}
Issues:

- The streamText function doesn't seem to return any text, resulting in an empty accumulatedResponse.

- The token count is always zero, and tool calls and results are null.

What I've Tried:

- Added logging inside the onStepFinish callback to check if it's being called.

- Verified the model client and configuration.

Could anyone help me identify what might be going wrong or suggest any debugging steps?