Azure OpenAI and Streaming. Does anyone overcome over it?
Unanswered
bplr posted this in #help-forum
bplrOP
Hi, we're trying to move into Azure and their AI API but i'm kinda struggle with the streaming part. After a little research i found that Microsoft themself limit the stream chunks so i get a bunch of words - wait for like 10 seconds and then the next bunch of words appear - wait 10 sec... and so on.
Any ideas how can i overcome this? I tried like SSE, client-side chunking and nothing seems to be good.
If anyone have a spare time or can share a code snippet to create a fully streaming pipe for Azure - i will be greatful!
Thank you.
Any ideas how can i overcome this? I tried like SSE, client-side chunking and nothing seems to be good.
If anyone have a spare time or can share a code snippet to create a fully streaming pipe for Azure - i will be greatful!
Thank you.