OpenAI API Stuck During Loop Execution, Resumes After Delay (I'm a Development Noob)

API
1

Hello Everyone,

I am very new to development, and I’m currently facing an issue while using the GPT-3.5 Turbo model to generate meta descriptions for a list of stores. My code reads store names from a CSV file, passes them to the API, and then saves the generated meta descriptions in another CSV file.

Even though the code works as expected for the first few iterations, it then seems to get stuck. After a considerable delay, it resumes processing. I have checked, and I am not exceeding the API rate limits. Additionally, there are no errors or exceptions being thrown.

Here’s a simplified snippet of my code:

# Function to generate meta description
def create_meta_description(store_name):
    # API call here
    ...

# Read CSV and generate meta descriptions
with open('store_names.csv', 'r') as readfile, open('meta_descriptions.csv', 'w', newline='') as writefile:
    csvreader = csv.reader(readfile)
    csvwriter = csv.writer(writefile)
    header = next(csvreader)
    csvwriter.writerow(['store_name', 'response'])
    
    for row in csvreader:
        store_name = row[header.index('StoreName')]
        meta_description = create_meta_description(store_name)
        csvwriter.writerow([store_name, meta_description])

Has anyone faced a similar issue? Any suggestions on how to troubleshoot this would be greatly appreciated.

Thank you in advance for your help!

2

There are times that the API call will take long. Not sure with python module but in Node.js module there is timeout parameter if you do not want to wait.

Edit: I saw the one for python.

Params
All endpoints have a .create method that supports a request_timeout param. 
This param takes a Union[float, Tuple[float, float]] and will raise an openai.error.Timeout error if the request exceeds that time in seconds

For Node.js

// Configure the default for all requests:
const openai = new OpenAI({
  timeout: 20 * 1000, // 20 seconds (default is 10 minutes)
});

// Override per-request:
await openai.chat.completions.create({ messages: [{ role: 'user', content: 'How can I list all files in a directory using Python?' }], model: 'gpt-3.5-turbo' }, {
  timeout: 5 * 1000,
});

There is also retry parameter.

// Configure the default for all requests:
const openai = new OpenAI({
  maxRetries: 0, // default is 2
});

// Or, configure per-request:
await openai.chat.completions.create({ messages: [{ role: 'user', content: 'How can I get the name of the current day in Node.js?' }], model: 'gpt-3.5-turbo' }, {
  maxRetries: 5,
});

Reference:

3

Thank you for the detailed information! I wasn’t aware that the OpenAI API provided a request_timeout parameter for Python as well. This will be very useful in handling delays more efficiently.

I’ve updated my code to include the request_timeout parameter, setting it to 20 seconds for now. This should help in cases where the API takes longer than expected to respond.

As for the retry mechanism, I’ve manually implemented a loop in Python to retry the API call a few times with a delay, which should serve the same purpose as the maxRetries parameter in the Node.js version.

Thanks again for your guidance!

4

I found this thread here. With more and more people with the same problem.