Responses API + Structured Outputs (gpt-5.6-luna): garbage tokens (foreign scripts / leaked reasoning) inside string values right before the closing quote — identical request via Chat Completions is clean

API Bugs
1

Summary

Since migrating structured-output calls from Chat Completions to the Responses API, gpt-5.6-luna intermittently emits degenerate tokens inside
string values, immediately before the closing quote. The JSON is always schema-valid (strict: true), so the garbage flows straight into the app.
The same model + same prompts + same schema via Chat Completions is clean across all our runs.

Environment

  • Model: gpt-5.6-luna (also default snapshot), direct openai api (no Azure)
  • SDK: openai-node 6.3.0, Node 24.11.0
  • Non-streaming client.responses.parse() — final response object, no delta assembly
  • reasoning: { effort: "low" }, max_output_tokens: 20000, store: false

Request shape

{
  "model": "gpt-5.6-luna",
  "store": false,
  "max_output_tokens": 20000,
  "reasoning": { "effort": "low" },
  "text": {
    "format": {
      "type": "json_schema",
      "name": "IngredientExtraction",
      "strict": true,
      "schema": {
        "type": "object",
        "additionalProperties": false,
        "properties": {
          "missing_info_details": { "anyOf": [ { "type": "string", "maxLength": 250 }, { "type": "null" } ] }
        }
      }
    }
  },
  "input": "…static ~6.5k-token system prompt (food-ingredient extraction) + short user message in Spanish…"
}

Actual output (raw, captured)

Last ~90 chars of output_text, with "status": "completed", "incomplete_details": null (NOT a length cut), usage output 229 / reasoning 109:

...quires_more_info":true,"missing_info_details":"¿Qué alimento es la “mandangota”?ください"}

The expected string is just …“mandangota”? — the ください appears out of nowhere right before the closing quote.

More samples from the same request across runs (only this field shown):

"¿Qué alimento querías decir con “mandangota”? בו"
"¿Qué alimento es la “mandangota”?} 彩神争霸代理ablytyped 天天中彩票投注  (jsontas?)"
"¿Qué alimento es la “mandangota”?}тормош ♀ ♀ ♀ ♀ арадио? Nope. Need valid JSON only. My output has weird? I need final correct. The format wrapper
likely expects JSON object, no markdown. Use only JSON. Ensure health index number. Done. (The previous? I"

That last one (from production) reads like leaked reasoning rambling inside the string until the grammar cut it at exactly maxLength: 250.

Incidence & what we ruled out

  • ~55 repro runs on Responses: ~7% severe (random scripts: Japanese, Hebrew, Han, Cyrillic — always right before a closing quote) and ~half of
    runs show milder junk at the same position (trailing space, or ?.).
  • Reproduces with prompt_cache_options: {"mode":"explicit"} and in implicit mode → not cache-related.
  • Reproduces with and without maxLength on the string → the constraint only caps the runaway.
  • No streaming involved; status is always completed; not a refusal.
  • Chat Completions (chat.completions.parse, response_format json_schema strict), identical everything: 0 anomalies.

Hypothesis

The Responses API constrained decoder seems to mishandle the end-of-string token boundary (e.g. merged tokens like ?" being masked by the grammar),
forcing low-probability continuations — occasionally spiraling into foreign-script tokens or reasoning-like self-talk inside the string.

Ask

  1. Is this a known issue with the GPT-5.6 family’s constrained decoding on the Responses API?
  2. Any recommended mitigation other than falling back to Chat Completions (which loses explicit prompt caching)?

Happy to share full raw request/response captures privately.

2 Likes