Structured Ouput - Not respecting the nullable() / nullable : true constraint - with `model : gpt-4o` (thus gpt-4o-2024-08-06)

API Bugs
1

Summary: model : gpt-4o (thus gpt-4o-2024-08-06) does not respect the nullable() / nullable : true constraint for structured outputs at the time of writing (14th and 15th of February 2025). Which it did previously. Explicitly using model: "gpt-4o-2024-11-20" may resolve the issue (at least it did for me).

It is quite easy to reproduce using the playground example for structured outputs. Just switch the models to observe the behavior. Reponses to expect when using the model : gpt-4o (thus gpt-4o-2024-08-06) for the “date” property are values like "date": ":null", "date": "/" or "date": ". Using the model: "gpt-4o-2024-11-20" “date” is set to null as expected.

// OPENAI_API_KEY=YOUR_API_KEY
import 'dotenv/config';

import OpenAI from "openai";
import { zodResponseFormat } from "openai/helpers/zod";
import { z } from "zod";

// or manually add { apiKey : 'YOUR_API_KEY' } within the parentensies
const openai = new OpenAI();

const CalendarEvent = z.object({
  name: z.string(),
  date: z.string().nullable(), // .nullable() not resepected
  is_date_weekend: z.boolean().nullable(),
  participants: z.array(z.string()),
});

const completion = await openai.beta.chat.completions.parse({
//  model: "gpt-4o-2024-08-06", // .nullable() not respected
//  model: "gpt-4o-2024-11-20", // switch between models to reproduce the issue
  messages: [
    { role: "system", content: "Extract the event information. Do not infer missing data. If a property has no relevant parsed information, set it explicitly to null. Do not guess values, assume defaults, or attempt to derive meaning from context." },
    { role: "user", content: "Alice and Bob went to a science fair." },
  ],
  response_format: zodResponseFormat(CalendarEvent, "event"),
});

const event = completion.choices[0].message.parsed;

console.log( JSON.stringify( event, null, ' ') );
// responses contain values like "date": ":null", "date": "/", "date": "."`
3 Likes
2

We came across this issue, too. We came up with two possible fixes:

  1. Specifying "type": ["string", "null"]" works as a fix - this ensures correct handling of nullable values.
  2. Reverting to an earlier version of GPT-4o (e.g., gpt-4o-2024-11-20 ) prevents this issue , meaning this is likely a new behavior introduced in the most recent version.