Structured Outputs -- invalid schema error (only when deployed)

Error

I’m developing an agent that uses Structured Outputs to generate inputs for a function. I already tested it locally & made sure everything works. But when I deployed it, I got this error on my cloud logs:

Error code: 400 - {'error': {'message': "Invalid schema for function 'CreateFilter': In context=('properties', 'filter_object'), schema must have a 'type' key.", 'type': 'invalid_request_error', 'param': 'tools[8].function.parameters', 'code': 'invalid_function_parameters'}}

Responsible Code

My schema includes 3 instances of this filter_object param. As you see though, the param includes the type key. I made sure all 3 instances include the key
image

What I tried so far

  • re-run locally → works no issues
  • re-deploy → still the same error
  • log the schema in the deployed version → shows the correct schema with the type key in every instance of filter_object
1 Like

The following types are supported for Structured Outputs:

String
Number
Boolean
Integer
Object
Array
Enum
anyOf

We try to see what you are doing with a basic function:

{
  "name": "set_unit",
  "description": "angle unit for all functions",
  "strict": true,
  "parameters": {
    "type": "object",
    "properties": {
      "unit": {
        "const": "Deg",
        "type": "string",
        "title": "Can I use a title?",
        "description": "I make the AI only send degrees",
        "enum": [
          "Deg"
        ]
      }
    },
    "additionalProperties": false,
    "required": [
      "unit"
    ]
  }
}

Does the “const” serve any purpose? Present or removed, the AI receives the same text:

namespace functions {

// angle unit for all functions
type set_unit = (_: {
// Can I use a title?
//
// I make the AI only send degrees
unit: "Deg",
}) => any;

} // namespace functions

Remove enum, and any purpose of “const” is not there.

namespace functions {

// angle unit for all functions
type set_unit = (_: {
// Can I use a title?
//
// I make the AI only send degrees
unit: string,
}) => any;

} // namespace functions

Title is placed the same as description, and description is used more universally in functions since inception. Other JSON keywords that seem useful or constraining also just are written as description to the AI.

Therefore I would see if cleaning up the function to just what is useful fixes what you don’t entirely show.

Does “deploy” imply Azure? Does gpt-4o mean 2024-05-13?

1 Like

The const is a Pydantic feature. Here’s the original model:

class ContactFilter(BaseModel):
    filter_object: Literal["contact"]

The purpose here is to force filter_object == "contact" so the LLM doesn’t hallucinate other values.

The model is the default gpt-4o which as of Oct 3 I think refers to 08-12 i.e., structured outputs are enabled otherwise I wouldn’t get this error message.

Deployment is on Google Cloud Run.

1 Like

const also has no effect when sent to the API to meet that goal.

If it works for your python and not another python platform, then your new mention of pydantic used in code and needing compatible support by another’s platform may be the issue. Here’s an “upgrade” with the max versions of requirements for OpenAI and the dependencies going down:

pip install --upgrade --upgrade-strategy eager regex "charset-normalizer<4" "idna" "urllib3<3" "certifi" "requests" "anyio<5" "distro<2" "sniffio" "h11<0.15" "httpcore==1.*" "httpx<1" "annotated-types" "typing-extensions<5" "pydantic-core==2.20.1" "pydantic<3" "jiter<1" "tqdm" "colorama" "openai" "tiktoken"

I removed the minimum as getting from pypi will get the latest supported and which the SDK may have been developed against.

Python 3.9-3.11 would be a good thing, newer for more inbuilt typing and hinting.

1 Like