Error 500 with structured outputs using Azure

API
,
1

My first resquest using structured outputs with gpt-4o version 2024-08-01-preview went succesfully. However, now, whenever I change my schema I get a openai.InternalServerError: Error code: 500 - {‘error’: {‘message’: ‘The server had an error while processing your request. Sorry about that!’, ‘type’: ‘server_error’, ‘param’: None, ‘code’: None}}. The old schema still works. I tried to change just a little by swapping int with str and still get the exact error.
Anyone have the same issue? How can I solve it?

4 Likes
2

I am encountering a similar issue while trying to implement the example provided by Microsoft as following:

from pydantic import BaseModel
from openai import AzureOpenAI

client = AzureOpenAI(
  azure_endpoint = os.getenv("AZURE_OPENAI_ENDPOINT"), 
  api_key=os.getenv("AZURE_OPENAI_API_KEY"),  
  api_version="2024-08-01-preview"
)


class CalendarEvent(BaseModel):
    name: str
    date: str
    participants: list[str]

completion = client.beta.chat.completions.parse(
    model="MODEL_DEPLOYMENT_NAME", # replace with the model deployment name of your gpt-4o 2024-08-06 deployment
    messages=[
        {"role": "system", "content": "Extract the event information."},
        {"role": "user", "content": "Alice and Bob are going to a science fair on Friday."},
    ],
    response_format=CalendarEvent,
)

event = completion.choices[0].message.parsed

print(event)
print(completion.model_dump_json(indent=2))

It seems that the implementation only works when I keep the variable names and types exactly as specified in the example. Any changes to the names or data types of the response format lead to this error:

openai.InternalServerError: Error code: 500 - {'error': {'message': 'The server had an error while processing your request. Sorry about that!', 'type': 'server_error', 'param': None, 'code': None}}

Example of change: just simply remove s in participants

class CalendarEvent(BaseModel):
    name: str
    date: str
    participant: list[str]
3 Likes
3

I’m also encountering the same error from yesterday(25th sept 2024), but the workaround you suggested is working for me. I don’t have a solution for it but I would be curious to know since when are seeing this error? Thanks

1 Like
4

I just tried to switch from using OpenAI directly to using Azure OpenAI yesterday, and the issue occurred

5

This has been happening to me since yesterday. Seems to be an issue with the OpenAI API. I bet we’re hitting the cache when the API returns a response using one of the known examples. Otherwise, it’s throwing a 500 error. @OpenAI - please look into this!

1 Like
6

It appears to be working for me now. :magic_wand:

1 Like
7

Yeah, it’s working now :heart: :heart: :heart:

8

Using structured output (response_format) is returning 500 for me. Same exact scenario worked perfectly fine yesterday, but since today it’s failing. Dummy tests of structured format work fine, but my scenario (with a long prompt, complex response format) returns 500.
I’m using gpt-4o, version, 2024-08-06, through Azure.
Any idea…?