API ChatCompletion.Create is taking too long

API
1

GPT API with ChatCompletion.create is taking too long, more than 5 minutes in just one query, I make the question from python code and never ends.

1 Like
2

Welcome to the Forum! Please can you post your code so we can see if you have any errors that may be causing the problem, also please include a typical prompt.

1 Like
3 
for symbol in symbols:
    name = symbol['name']
    short_percentage = symbol["shortPercentage"]
    long_percentage = symbol["longPercentage"]
    short_volume = symbol["shortVolume"]
    long_volume = symbol["longVolume"]
    long_positions = symbol["longPositions"]
    short_positions = symbol["shortPositions"]
    total_positions = symbol["totalPositions"]
    avg_short_price = symbol["avgShortPrice"]
    avg_long_price = symbol["avgLongPrice"]

    stringData = f"For asset {name}, the short Percentage is {short_percentage}%, long percentage is {long_percentage}%, short volume is {short_volume}, long volume is {long_volume}, the number of long positions is {long_positions}, the number of short positions is {short_positions}, the total positions are {total_positions}, the average short price is {avg_short_price} and the average long price is {avg_long_price}\n"

    timeBeforeResponse1 = time.time()
    response = openai.ChatCompletion.create(
        model="gpt-3.5-turbo",
        messages=[
            {"role": "user",
                "content": "I need to know the sentiment of data, only return one of this values [VERY POSITIVE, POSITIVE, NEUTRAL, NEGATIVE, VERY NEGATIVE]: with the following data get the sentiment:'''"+stringData+"'''"},
        ],
        temperature=0,
    )
    time.sleep(1)

    timeBeforeResponse2 = time.time()
    responseKeywords = openai.ChatCompletion.create(
        model="gpt-3.5-turbo",
        messages=[
            {"role": "user",
                "content": "I need just 5 keywords in format '''keyword1, keyword2, keyword3, keyword4, keyword5''', splitted by comas, keywords gotten from the following data, :'''"+stringData+"''', please don't include words that are written in the previous data, I need any keyword that is related in financial terms, like markets, countries, exchange and currecies affected"},
        ],
        temperature=0,
    )

    time.sleep(1)

    timeBeforeResponse3 = time.time()
    responseMarket = openai.ChatCompletion.create(
        model="gpt-3.5-turbo",
        messages=[
            {"role": "user",
                "content": "I need to know the market affected by the data, only return one of this values [FOREX, FUTURES, STOCKS, CRYPTO, BONDS, COMMODITIES, OTHER]: with the following data get the market:'''"+stringData+"'''"},
        ],
        temperature=0,
    )```

It is a code that brings symbol information and ask gpt 3.5 about keywords, sentiment and markets
4

You might get a better idea if your request is being serviced at all by adding a streaming loop, even if it just prints to the shell.

Then robust error handling with cases for all the exceptions that can be thrown by the openai module.

Threading with your own timeout will prevent a call that never returns.

1 Like