4.1 mini Model : Escapes Quotes/Dashes

API Bugs
1

Hi OpenAI team and community,

I ran into an issue with GPT‑4.1‑mini when using structured JSON outputs.
Whenever the model generated curly quotes (“ ”), plus signs (+), or dashes (–), the JSON output contained escaped Unicode or hex codes such as:

Select \x002B New\x0022 in the Command Bar to open the FB \x002D Account using Google.

Instead of:

Select  "+ New" in the Command Bar to open the FB - Account using Google.

Regex Solution That Works

Before passing the model input, I normalize the text by replacing curly quotes and dashes with ASCII equivalents:

import re

# Replace curly quotes with straight quotes
text = re.sub(r'[“”]', '"', text)

# Replace en-dash/em-dash with hyphen
text = re.sub(r'[–—]', '-', text)

:white_check_mark: Result:

Select "+ New" in the Command Bar. This action will open the FB - Account using Google

Has anyone else run into this? Any official recommendation to prevent the escaping at the model output level?

2

With a json schema, and “You are an AI who creates patch files as your response, of any destination non-binary file, simply that response will apply a diff to alter a file.” sent to gpt-4.1-mini:

The image shows a user requesting code to demonstrate four varied file path examples, and an assistant responding with a Python snippet illustrating absolute, relative, and parent directory paths. (Captioned by AI)
The image shows a user requesting code to demonstrate four varied file path examples, and an assistant responding with a Python snippet illustrating absolute, relative, and parent directory paths. (Captioned by AI)608×475 12.1 KB

Try some more:

The image shows a programming-related conversation, where a user instructs to use directional double quotes instead of escaped double-quotes, and the assistant responds with a code diff that updates a Python list of file paths, as well as a line count change. (Captioned by AI)
The image shows a programming-related conversation, where a user instructs to use directional double quotes instead of escaped double-quotes, and the assistant responds with a code diff that updates a Python list of file paths, as well as a line count change. (Captioned by AI)610×330 9.81 KB

No Unicode code points seen, nor hex.

Certainly a plus or hyphen is odd, as they are simple ASCII. Language is generated in one direction, though: the AI could be predicting the \x (or a /u is also a common fault) because it predicts “being fancy” is possible in that type of instruction to a user, or that it discusses code. A token that can be demoted if you log the chunk string.

If you have the same task input to re-run, I would indicate in the response format schema that the output field with the symptom is unescaped native Unicode UTF-8, except for escaping double-quote and newline/tab.

Are you unwrapping the AI’s JSON contents in the JSON API response also as JSON with a JSON library for parsing?

1 Like