Openai API giving timeout error between random requests using flask api

API Bugs
1

I’m using openai api with flask deployed on azure app service. The issue is I’m getting timeout errors randomly after a few requests. What can be the reason and ways to fix it?

1 Like
2

Could you provide some code snippets and some payloads you are sending? :hugs:

1 Like
3

Following is my request body
list_of_salutations = [“Thanks”]
messages = [
{“role”: “system”, “content”: f"You are an assistant tasked to remove the phrase that includes these keywords {list_of_salutations}. And don’t add anything on your own"},
{“role”: “user”, “content”: “Thanks for coming today. How can I help you?”},
]
response = self.client_openai.chat.completions.create(
model=self.openai_model_name,
messages=messages,
timeout=10,
response_format={“type”: “json_object”},
seed=1,
temperature=0.7,
max_tokens=200
)

1 Like
4

Are you getting any errors? Maybe post the exact timeout error you are getting.

Timeout errors can occur due to several reasons such as network latency, server overload, or inefficient code.

  • Ensure your Flask app is optimized and using asynchronous requests to handle multiple requests efficiently.

  • Check Azure App Service logs for any signs of resource exhaustion or errors.

  • Increase the timeout settings in your OpenAI API requests and Azure App Service configuration.

  • Consider scaling your Azure App Service plan to handle higher loads.

1 Like
5

The CPU %age is around 10 to 12% on average and Memory is around 70%. No resource exhaustion errors in the logs.

1 Like
6

Could you post the exact errors you get? :smile:

7

This is the exception type I’m getting openai.APITimeoutError

8

A quick google finds these references, please check them out:

1
2

Check Network Configuration
Check Environment Configuration
Check for Proxy Issues

It has to be an issue on your end.

Also try updating the OpenAI package.

pip install --upgrade openai

Let me know if this solved your issue. :hugs: