We are using OpenAI’s chat API (langchain Python module) to implement a document Q&A service. While testing the service locally on my developer machine (Mac Book Pro), the API response times are much smaller (varies from 2 - 5 seconds). However, once deployed as a GCP Cloud Run service, we observe s…

Significant degredation in API response time when running in GCP Cloud Run

varun_p October 31, 2023, 4:05pm 2

@Foxalabs has some great suggestions

Topic		Replies	Views
Seeking Advises on Optimizing openAI API Calls Feedback api	2	512	November 16, 2023
Discrepancy in Response Speed between GPT-3.5-turbo API and ChatGPT UI API gpt-35-turbo , chatgpt , api	4	2905	December 24, 2023
How can I improve response times from the OpenAI API while generating responses based on our knowledge base? API chatgpt , api	3	18665	November 9, 2023
Slow Chat api responses ------ API	17	6264	December 24, 2023
Unstable speed of gpt-3.5-turbo-16k API api , gpt-35-turbo-16k , performance	6	1039	January 9, 2024