Assistant calls through Python SDK freezes intermittently

API Bugs
1

I’ve a setup with OpenAI assistant threads being created as part of python socketio connection creation with our client (UI). As more connections are opened, intermittently a new thread creation call freezes. Here is my thread creation code:

def create_session():
    print("Before thread creation")
    thread = None
    try:
        thread = client.beta.threads.create(timeout=2)
    except:
        print("Failure to create thread!")
    print("After thread creation")
    return thread.id

Interestingly the timeout argument also seem to be not honored and no exceptions are raised either. The server is hosted on flask, socket implementation used is eventlet, openai SDK version is 1.4.0.

Thanks in advance for any help!

2

One potential solution is to modify your code to handle timeouts more effectively and possibly investigate the root cause of intermittent freezing. Here’s a revised version of your code:

def create_session():
    print("Before thread creation")
    thread = None
    try:
        with eventlet.Timeout(2, False):  # Set a timeout for the thread creation
            thread = client.beta.threads.create()
    except eventlet.Timeout as e:
        print(f"Timeout Error: {e}")
    except Exception as ex:
        print(f"Error creating thread: {ex}")
    print("After thread creation")
    return thread.id if thread else None

In this version, I’ve introduced the eventlet.Timeout context manager to handle timeouts more effectively. If the thread creation takes longer than 2 seconds, it will raise a Timeout exception, allowing you to catch and handle it appropriately. Also, I added a generic exception handling block to catch any other potential errors during thread creation.

Make sure to import eventlet at the beginning of your script:

import eventlet

This adjustment may help in addressing the freezing issue and provide more insight into what’s causing the problem. If the problem persists, further investigation into the broader system architecture, Flask, and OpenAI SDK interactions may be needed.