Hello everyone, I’m currently performing a performance test on my company’s product using the OpenAI API. We’re on tier-5, but we’ve encountered an issue: whenever we increase our queries to 500 per minute, the OpenAI API starts returning a ‘request timeout’ error for all subsequent queries. Has anyone else experienced this? Is there a solution to this problem? For reference, we’re using approximately 1200 tokens per query, including the system prompt and the rag data.
1 Like
Hi @wei.hong.chin!
I have a very similar observation, thus curious, did you come up with a solution?