Calling chat completion parse with structured output and previous tool calls in the message history throws a 500

API
1

Hi, using the latest 1.40.0 version of python SDK and the new gpt-4o-2024-08-06 model, when I pass a previous message history that contains tool calls and the tool responses alongside a response format pydantic model, it throws a 500 error. When I remove the tool call it seems to work fine.

1 Like
2

I have a minimal reproduceable example:

from pydantic import BaseModel
from openai import OpenAI

client = OpenAI()

class Step(BaseModel):
    file_path: str
    repo_name: str
    type: str
    diff: str
    description: str
    commit_message: str


class CodingOutput(BaseModel):
    steps: list[Step]


completion = client.beta.chat.completions.parse(
    model="gpt-4o-2024-08-06",
    messages=message_dicts,
    response_format=CodingOutput,
)

print(completion.choices[0].message.parsed)

Works:

message_dicts = [
    {
        "content": "You are an exceptional principal engineer that is amazing at finding and fixing issues in codebases.",
        "role": "system",
    },
    {
        "content": 'We need to enhance error handling in the `from_str` method.',
        "role": "assistant",
    },
    {
        "content": "Break down the task of fixing the issue into steps.",
        "role": "user",
    },
]

Does not work, throws a 500 HTTP error:

message_dicts = [
    {
        "content": "You are an exceptional principal engineer that is amazing at finding and fixing issues in codebases.",
        "role": "system",
    },
    {
        "content": "To address the issue of the malformed input string causing a `KeyError` in the `PriorityLevel` enum, we need to implement a comprehensive solution.",
        "role": "assistant",
        "tool_calls": [
            {
                "id": "call_wHhKUvGq8xMyB78PYQrqIqTp",
                "function": {
                    "name": "expand_document",
                    "arguments": '{"input": "some/document.py", "repo_name": "owner/repo"}',
                },
                "type": "function",
            },
        ],
    },
    {
        "content": 'class PriorityLevel(IntEnum):\n    LOW = 25\n    MEDIUM = 50\n    HIGH = 75\n\n    def to_str(self) -> str:\n        return self.name.lower()\n\n    @classmethod\n    def from_str(self, name: str) -> "PriorityLevel":\n        return self[name.upper()]\n',
        "role": "tool",
        "tool_call_id": "call_wHhKUvGq8xMyB78PYQrqIqTp",
    },
    {
        "content": 'We need to enhance error handling in the `from_str` method.',
        "role": "assistant",
    },
    {
        "content": "Break down the task of fixing the issue into steps.",
        "role": "user",
    },
]
3

from my experience, you do not add previous tool calls in the context.

4

You need to add the previous tool calls results in the context no? Our use case is to format only the last output.

5

hmm, you only add the tool call response and tool call output when you are processing it. but i believe (maybe i am wrong) that you do not add it to the main context itself. so in the sample code you gave, it is already processed and already behind two messages.

6

Each call with the context is independent from oneanother, it’s not processed and stored in a memory somewhere. You need to keep sending longer and longer message chains.

This works perfectly fine outside of the json schema mode.

7

In the assistant message that contains the “tool_calls” the content needs to be None for some reason.

Updated version:

message_dicts = [
    {
        "content": "You are an exceptional principal engineer that is amazing at finding and fixing issues in codebases.",
        "role": "system",
    },
    {
        "content": None,
        "role": "assistant",
        "tool_calls": [
            {
                "id": "call_wHhKUvGq8xMyB78PYQrqIqTp",
                "function": {
                    "name": "expand_document",
                    "arguments": '{"input": "some/document.py", "repo_name": "owner/repo"}',
                },
                "type": "function",
            },
        ],
    },
    {
        "content": 'class PriorityLevel(IntEnum):\n    LOW = 25\n    MEDIUM = 50\n    HIGH = 75\n\n    def to_str(self) -> str:\n        return self.name.lower()\n\n    @classmethod\n    def from_str(self, name: str) -> "PriorityLevel":\n        return self[name.upper()]\n',
        "role": "tool",
        "tool_call_id": "call_wHhKUvGq8xMyB78PYQrqIqTp",
    },
    {
        "content": "We need to enhance error handling in the `from_str` method.",
        "role": "assistant",
    },
    {
        "content": "Break down the task of fixing the issue into steps.",
        "role": "user",
    },
]

I don’t think it should be like this, it must be a bug.

1 Like
8

Hmm this doesn’t fix it for me.

9

Is it because the “required” and “additionalProperties” fields are needed? Docs say that this is the case.