Streaming issue related to chunk size
Unanswered
American Staffordshire Terrier posted this in #help-forum
American Staffordshire TerrierOP
As you can see in the print, I expected that the JSON
However, when I deploy to Vercel, this issue in the print happens.
Is there any way to change the max size per token or I must adapt my code to handle when the JSON comes separately in the chunks?
{"toolName": "find_single_member_by_name", "toolInput": {"query": "Simon Jones"}} would be streamed in only one token as it happens when I run my project locally on my machine.However, when I deploy to Vercel, this issue in the print happens.
Is there any way to change the max size per token or I must adapt my code to handle when the JSON comes separately in the chunks?