It appears that there may be a bug related to error-handling when the API is processing checks for JSON schema lengths. I am getting schema length-related issues for a JSON schema when requesting a structured output from the API (“strict”: True) in the “json_schema” object.
Here’s some example output:
Error: 400 - {
"error": {
"message": "Invalid schema for response_format '<removed_schema_name>': Maximum schema size exceeded. Expected the total schema size (including property names, definition names, enum values, and const values) to be less than 15000 characters, but your schema has more than 15031 characters. Consider reducing the length or count of string literals in your schema, or use strict: false to opt out of structured outputs.",
"type": "invalid_request_error",
"param": "response_format",
"code": null
}
}
And the corresponding Python code that produces this output is:
if response.status_code == 200:
# Parse and print the response content
result = response.json()
print(result['choices'][0]['message']['content'])
else:
print(f"Error: {response.status_code} - {response.text}")
I’ve tried removing components of my schema (removing more and more to try to meet the max character requirement to no avail, unfortunately).
Assuming the issue isn’t on my end with the API call, I think it maybe could have to do something with how the schema is tracked the first time it hits the API? Speculation here of course…
Anyone have any other ideas or things to try (either to validate that this is indeed a bug or to fix an issue on my end)? Thanks in advance!
Impressive. 15k ≈ 4k tokens: a lot for an AI to follow and fill out fields with quality when it gets that whole schema as input language.
I would follow that to the letter: see how you can shorten those field names. The error doesn’t mention “description”, so you are at a point of shaving off 31 letters somewhere, in terms of what affects the output and the output enforcement, and watching the error return decrease.
An error sending to the API should cause no persistence.
Further shortening names and rewriting the received output in code can shave cost off what you send to the AI, if you must match some external API you are otherwise sending to directly.
There appears to be mention of that maximum schema size in characters in documentation or API reference:
BTW: It is odd that they would mention “const”, as I found it performs no action on functions at least - enum is its substitute, when giving only one value (although pydantic usage may be adding const to what is sent). Other keywords also may have only descriptive effect.
Your initial comment regarding my token count triggered me to end up reexamining some assumptions regarding my schema definition and use case. I ended up really cutting down on my JSON schema definition through the use of subschema definitions to avoid replicating some costly (in terms of character count) enumeration values. After I adjusted my schema through the use of subschemas I was able to avoid any further errors regarding my schema sizes.
The OpenAI docs for structured outputs describe subschemas that helped me are here. I used the “math steps” example as an illustrative example to build upon for my use case.
I think you’re right that there didn’t end up being any state-based issues with the API processing a schema for the first time and this issue was due to needing to drastically reduce my schema size. For others encountering this issue, I’d recommend starting with a working basic schema and building up from there (vs. starting with a too-large schema and trying to cut down). This will also have the side-effect of helping you concretize some intuition for where the limits are for structured output schema sizes.
