Hi, I’m receiving a strange "{"rate_limit_usage": {\ in completion stream, which breaks everything as is it not in json format. Started happening around an hour ago. Never happened before. I’m within my rate limit, so that shouldn’t be an issue.
It is in a JSON format, just not one you’re looking for, and with an error you’re not handling with a try/except.
I’ll go over to software of mine and just give you an example of handling that would display the error on the console and not barf.
try:
response = openai.Completion.create(
model=self.aimodel.currentText(), prompt=full_prompt,
temperature=softmax_temperature, max_tokens=response_tokens,
top_p=top_p_set, frequency_penalty=0.0, presence_penalty=0.0
)
except openai.error.Timeout as e:
# Handle timeout error, e.g. retry or log
print(f"OpenAI API request timed out: {e}")
pass
except openai.error.APIError as e:
# Handle API error, e.g. retry or log
print(f"OpenAI API returned an API Error: {e}")
pass
except openai.error.APIConnectionError as e:
# Handle connection error, e.g. check network or log
print(f"OpenAI API request failed to connect: {e}")
pass
except openai.error.InvalidRequestError as e:
# Handle invalid request error, e.g. validate parameters or log
print(f"OpenAI API request was invalid: {e}")
pass
except openai.error.AuthenticationError as e:
# Handle authentication error, e.g. check credentials or log
print(f"OpenAI API request was not authorized: {e}")
pass
except openai.error.PermissionError as e:
# Handle permission error, e.g. check scope or log
print(f"OpenAI API request was not permitted: {e}")
pass
except openai.error.RateLimitError as e:
# Handle rate limit error, e.g. wait or log
print(f"OpenAI API request exceeded rate limit: {e}")
pass
except Exception as e:
error_message = f"Error: {str(e)}"
print(error_message)
self.status_update(False, error_message)
# CustomApplication.processEvents()
pass
1 Like
I’m also having the same issue. The JSON is not valid - so it fails to decode. I have been forced to add specific error handling for this
It has only started today. The code has not changed - so it is a change to the stream format
5 Likes
Same, this is some bug last hour or so on their API. It is breaking their own python library. Invalid JSON in a streaming response throws an APIError(message=‘HTTP code 200 from API ("{\“rate_limit_usage\”: {\)’, http_status=200, request_id=None)
6 Likes
Same here. It shows rate limit but it should not exceed. Not sure why…
4 Likes
I have encountered the same problem as well.
3 Likes
We are facing same issue since past more than 2 hours. Looks like open api has done some kind of breaking change. This needs to be fixed immidiately our prod applications are down.
We are well within limits
3 Likes
We have noticed that even if you drop invalid packets, it doesn’t send [DONE] at the end of the stream
1 Like
If you aren’t using functions, you can see if anything about this behavior changes by switching to the -0301 models. Unlikely but possible.
Unlikely because there are several other accounting systems that can track and inject besides just the model, like verifying keys per organization, or the very model selection itself.
2 Likes
Same issue for me. I’m using gpt-r-0314. It seems that there is a bug when sending the last chunk of a stream from the server side.
+1 here.
when I use gpt-3.5-turbo，it appears, when change to other model, it disappers.
Same issue occurred with gpt-3.5-turbo-0613
Facing the same issue as well in our prod service. We noticed it about an hour back - everything was working perfectly fine before
same here!!! please keep me posted! 
1 Like
+1 here.
when I use gpt-3.5-turbo，it appears, when change to other model, it disappers.
1 Like
same here!!! please keep me posted!
1 Like
I am also seeing this issue. My production site is broken and has been down for a number of hours now.
I have tried new keys & multiple models, all the same response.
Also seeing the same problem and our production site has been down for several hours. Please rollback the breaking change 
I face the same problem, I am using requests to stream and this is my temporization fix:
def extract_streaming_chunk(line):
# https://github.com/openai/openai-python/blob/5453a19efe6fa4395673782e5e3bd161572d383c/openai/api_requestor.py#L106
if line and line.startswith(b"data: "):
# SSE event may be valid when it contain whitespace
line = line[len(b"data: ") :]
# DONE
if line.strip() == END_OF_STREAM.encode("utf-8"):
# return here will cause GeneratorExit exception in urllib3
# and it will close http connection with TCP Reset
return END_OF_STREAM
else:
try:
# Sometime, OpenAI send an invalid chunk. This seem a bug from OpenAI
# OpenAI streaming chunk: b'"{\\"rate_limit_usage\\": {\\'
# https://community.openai.com/t/receiving-rate-limit-usage-in-completion-stream/427476/
return json.loads(line.decode("utf-8"))["choices"][0]["delta"].get(
"content"
)
except:
logger.warning(f"OpenAI streaming chunk: {line}")
return None
return None