Connection error when trying to call the API every X minutes

API
,
1

Hi,

I’m trying to run the following code:

while True:
    execute_prompt() # Uses ChatCompletion to create the call
    time.sleep(600) # 10 mins

But I am getting the following error after waiting the 10 minutes in the time.sleep() function.

openai.error.APIConnectionError: Error communicating with OpenAI: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

Is there a clear explanation of why this is happening? Or a way to solve it?

Note: I need to use the while loop since I am trying to run the code in a specific thread of a separated program. This is just an example to try to replicate the problem. The original program aims to provide a summary of a specific context every 24 hours.

Extra information: I’m using gpt-3.5-turbo, and I have not exceeded any rate limits.

Thanks in advance.

1 Like
2

It really depends on what is happening in the rest of your code.

If it is what I assume then you are opening a connection to the api, then send a request and then after 10!!! minutes try to reuse the same connection.

Why do you need to reserve that connection?


execute_prompt(prompt)
  curl open
  curl exec
  curl close

and not


curl open

execute_prompt(prompt)
  curl exec

...

But as long as you don’t tell us which programming language you use (i mean ok, that’s python, but still that is something you should always add in the beginning of such questions) or show us the code of that execute_prompt function that’s just fishing in a dark hole in the floor.

2 Likes
3

Thanks for the answer, and sorry for the lack of information, i will add the rest of the code here.

I don’t need to reserve the connection, i believe that closing it and opening again when i execute execute_prompt() will be ideal, but i do not know if that is possible. I’m thinking about this since my idea is to call the API every 24 hours.

This is the complete code, in case you can give me some more help. It is written in Python.


def execute_prompt():
    prompt = "PROMPT"

    response = openai.ChatCompletion.create(
        model="gpt-3.5-turbo", 
        messages=[{
            "role": "system",
            "content": f"{prompt}"
        }]
    )

    # Print the answer
    print(response["choices"][0]["message"]['content'])

interval = 600

while True:
    execute_prompt()
    time.sleep(interval)

Thanks again for your help @jochenschultz

4

Ah, ok that doesn’t really make any sense to me.

I had the same problem on azure.

In my case I was sending a prompt to gather some information and then used the whole chat in the next request including the answer from the first request to split the tasks per requests.

The microsoft guys said they’d need the prompt to figure that out.

Do you mind sharing the prompts?

Or else you just may try this solution:


from multiprocessing import Process

def execute_prompt():
    # Your existing code

def run_execute_prompt():
    p = Process(target=execute_prompt)
    p.start()
    p.join(30)  # Timeout in seconds
    if p.is_alive():
        p.terminate()
        p.join()

interval = 600

while True:
    run_execute_prompt()
    time.sleep(interval)

It starts a new sub process for each request as if you were running the program again without any prior actions.

I mean if that makes any sense in your usecase also depends on where you deploy it. On AWS lambda or Azure functions you could just push a message in a message queue and let it also have a retry logic in a fresh container (you can even prepare the containers, so it doesn’t take time to run them).

Or why not create an entry in crontab and let the python function run every 10 minutes

5

Thanks again for the answer.

I copied the exactly same code you provided and i am still getting the same error:

Traceback (most recent call last):
  File "/Users/tadeo/anaconda3/lib/python3.10/site-packages/urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "/Users/tadeo/anaconda3/lib/python3.10/site-packages/urllib3/connectionpool.py", line 449, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/Users/tadeo/anaconda3/lib/python3.10/site-packages/urllib3/connectionpool.py", line 444, in _make_request
    httplib_response = conn.getresponse()
  File "/Users/tadeo/anaconda3/lib/python3.10/http/client.py", line 1374, in getresponse
    response.begin()
  File "/Users/tadeo/anaconda3/lib/python3.10/http/client.py", line 318, in begin
    version, status, reason = self._read_status()
  File "/Users/tadeo/anaconda3/lib/python3.10/http/client.py", line 287, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/tadeo/anaconda3/lib/python3.10/site-packages/requests/adapters.py", line 489, in send
    resp = conn.urlopen(
  File "/Users/tadeo/anaconda3/lib/python3.10/site-packages/urllib3/connectionpool.py", line 787, in urlopen
    retries = retries.increment(
  File "/Users/tadeo/anaconda3/lib/python3.10/site-packages/urllib3/util/retry.py", line 550, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/Users/tadeo/anaconda3/lib/python3.10/site-packages/urllib3/packages/six.py", line 769, in reraise
    raise value.with_traceback(tb)
  File "/Users/tadeo/anaconda3/lib/python3.10/site-packages/urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "/Users/tadeo/anaconda3/lib/python3.10/site-packages/urllib3/connectionpool.py", line 449, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/Users/tadeo/anaconda3/lib/python3.10/site-packages/urllib3/connectionpool.py", line 444, in _make_request
    httplib_response = conn.getresponse()
  File "/Users/tadeo/anaconda3/lib/python3.10/http/client.py", line 1374, in getresponse
    response.begin()
  File "/Users/tadeo/anaconda3/lib/python3.10/http/client.py", line 318, in begin
    version, status, reason = self._read_status()
  File "/Users/tadeo/anaconda3/lib/python3.10/http/client.py", line 287, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/tadeo/anaconda3/lib/python3.10/site-packages/openai/api_requestor.py", line 516, in request_raw
    result = _thread_context.session.request(
  File "/Users/tadeo/anaconda3/lib/python3.10/site-packages/requests/sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/tadeo/anaconda3/lib/python3.10/site-packages/requests/sessions.py", line 701, in send
    r = adapter.send(request, **kwargs)
  File "/Users/tadeo/anaconda3/lib/python3.10/site-packages/requests/adapters.py", line 547, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/tadeo/Documents/Code/limoncito/test.py", line 47, in <module>
    ejecutar_prompt()
  File "/Users/tadeo/Documents/Code/limoncito/test.py", line 17, in ejecutar_prompt
    response = openai.ChatCompletion.create(
  File "/Users/tadeo/anaconda3/lib/python3.10/site-packages/openai/api_resources/chat_completion.py", line 25, in create
    return super().create(*args, **kwargs)
  File "/Users/tadeo/anaconda3/lib/python3.10/site-packages/openai/api_resources/abstract/engine_api_resource.py", line 153, in create
    response, _, api_key = requestor.request(
  File "/Users/tadeo/anaconda3/lib/python3.10/site-packages/openai/api_requestor.py", line 216, in request
    result = self.request_raw(
  File "/Users/tadeo/anaconda3/lib/python3.10/site-packages/openai/api_requestor.py", line 528, in request_raw
    raise error.APIConnectionError(
openai.error.APIConnectionError: Error communicating with OpenAI: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

Also, thanks for the explanation about the code you included.

I can not use it in crontab since i need to run this code in a thread, because i am running other code in the background that is running 24/7 on a VPS as part of an Slack bot.

Hope to see if there’s a way to solve it, thanks for your help @jochenschultz

6

Hi @imtdb

Welcome to the OpenAI community.

This error could be due to the sleep implementation.

The error message you’re seeing, RemoteDisconnected('Remote end closed connection without response') , often indicates that the server closed the connection unexpectedly.

If your connection is idle for too long, the server might close it due to inactivity. This is common behavior for many web servers to save resources. If you have a sleep duration of 10 minutes or more, the connection might be getting closed in the meantime.

Also I see in your code that you’re only sending the system message. Ideally the system message is meant for instruction to the model on how to respond to the user message which would contain the question/query.

7

Thank you for the warm welcome @sps,

I understand that the connection is closing due to inactivity, but how can I open it again to use it in the example I provided? I was searching the docs, but i was unable to find any information about how to close and open a connection in a scenario like this.

I will also consider your comments on the system message.

Thanks in advance.

1 Like
8

One way would to implement sleep within the function.

if(response["choices"].length):
    print(response["choices"][0]["message"]['content'])
    time.sleep(interval)

UPDATE: @RonaldGRuckus reached out to me and shared this issue from GitHub. It is a common issue with the OpenAI library. It maintains a session once it’s created so that calls aren’t closing and re-opening new connections when called in rapid succession.

Here’s the workaround for this issue

2 Likes
9

I think in your case after reviewing the library, I would say it has to do with this:

Maybe try 599 as interval or close the session explecitely and also implement a retry logic.

import time
import openai

def execute_prompt(retries=3, initial_delay=5):
    prompt = "PROMPT"
    delay = initial_delay
    for i in range(retries):
        try:
            response = openai.ChatCompletion.create(
                model="gpt-3.5-turbo", 
                messages=[{
                    "role": "system",
                    "content": f"{prompt}"
                }]
            )
            print(response["choices"][0]["message"]['content'])
            return
        except openai.error.APIConnectionError as e:
            print(f"Attempt {i+1} failed: {e}")
            time.sleep(delay)
            delay *= 2  # Exponential backoff

        # Close the session to ensure a new connection
        openai.api_requestor._session.close()

interval = 600 # maybe change this to 599

while True:
    execute_prompt()
    time.sleep(interval)
10

I don’t think this is a problem with your code. I think there are genuinely network issues or else OpenAI is somehow ejecting it. You can even ask ChatGPT to generate a function that does this and it will generate the same kind of thing.

11

The code looks OK. (import time).

I would try with Python 3.8.15, and also pip install --upgrade openai in the same administrator or user context as the python “install for all users”=administrator or root.

An upgrade just gave me openai-0.27.10 → openai-0.28.0

A parameter you can add alongside others like model is request_timeout=120, along with a try/except, to ensure the library connection is released and unblocked before it is called again. Another parameter max_tokens=500 (or what is appropriate) will keep AI from accidentally generating endless text for minutes.

1 Like
12

Thanks for the reply @_j .

After updating the OpenAI lib from 0.27.10 to 0.28.0 the problem is gone.

Have a great day, and thanks to everyone else who contributed!

1 Like
13

As an addendum to this discussion, has anyone tried using the “with” statement to scope and let Python control and
cleanup the lifecycle of the prompt and/or request objects?

14

Have you only updated the OpenAI library or have you added the “request_timeout” parameter in the code to solve this problem?

15

FWIW, we sometimes see errors even when we’re not idling out. The OpenAI platform has been seeing a lot of growth in load, it seems, and sometimes just returns 429 or 503.
What works for us is to make each request use retry. We re-try five times, with a sleep time of 0.5 seconds times two to the power of number of retries.
An easy way to do this in python is a simple wrapper function, although you could also use some pip library like tenacity or maybe retry.

def retry(f):
  for i in range(6):
    try:
      if i > 0:
        time.sleep((2**i)*0.5)
      return f()
    except Exception as e:
      print(f're-trying after exception: {e}', flush=True)
  raise Exception('re-try count exhausted')

Usage:

result = retry(lambda: do_the_thing(args))
16

I have only updated the OpenAI library.

1 Like