I’m streaming completions with an API with SSE just as openai is streaming (fastapi python framework) and experiencing lags when ran on the cloud (Google Cloud Run) but not locally.
Anyone ever experienced similar issue, found a solution?
Thanks 
Related Topics
| Topic | Replies | Views | Activity | |
|---|---|---|---|---|
| Streaming responses - slow in web-hosted version, fast in local-development version | 2 | 385 | February 6, 2024 | |
| Strange API behaviour with streamlit vs fastapi | 0 | 649 | May 23, 2023 | |
| Azure OpenAI streaming vs. OpenAI directly | 3 | 2326 | July 7, 2023 | |
| Rate limits in middle of stream | 8 | 1115 | December 24, 2023 | |
| Significant degredation in API response time when running in GCP Cloud Run | 2 | 636 | November 1, 2023 |