Structured Output not giving schema compliance on gpt-4o-mini

API Bugs
,
1

Example: https://platform.openai.com/playground/chat?models=gpt-4o-mini&preset=preset-i3LiIoW9aaFGVgAp7TGBRXqG

The function definition for record_availability says it should take 2 parameters:

  • routes: string // a stringified JSON of {“carrier”: string, “available_on”: string|null, “origin”: string, …}
  • available_on: string // date formatted as MM/DD/YYY

(I’m aware I’m “holding it wrong” by asking for "routes" to contain stringified JSON. I coded myself into a corner and discovered this by accident.)

What I get

record_availability({
  "routes": "Springfield/Oakland, Chicago, or nearby areas to Arizona or Mexico",
  "offered_on_date": "08/15/2024",
  "carrier": "John Smith",
  "available_on": null,
  "origin": "Springfield/Oakland, Chicago, or nearby areas",
  ...
})

What’s wrong

The top-level record_availability() call has additional parameters that are supposed to go into the "routes" parameter’s JSON.stringified content.

"additionalProperties": false is set, so these extra parameters are invalid.

Looking closely, "routes" and "offered_on_date" are members of the top-level params. So technically this meets the function call schema requirements if we ignore the additional members. (The "routes" string isn’t the stringified JSON requested, but we expect no guarantees from structured outputs about it.)

Model

This misbehavior happens with gpt-4o-mini. At least for this sample, gpt-4o is conforms to schema correctly. (In fact, gpt-4o fills “routes” with correct stringified JSON, though again we don’t expect it.)

Structured Outputs are supposed to work with gpt-4o-mini as well as gpt-4o, and not rely on model quality

1 Like