Hi, using the latest 1.40.0 version of python SDK and the new gpt-4o-2024-08-06 model, when I pass a previous message history that contains tool calls and the tool responses alongside a response format pydantic model, it throws a 500 error. When I remove the tool call it seems to work fine.
I have a minimal reproduceable example:
from pydantic import BaseModel
from openai import OpenAI
client = OpenAI()
class Step(BaseModel):
file_path: str
repo_name: str
type: str
diff: str
description: str
commit_message: str
class CodingOutput(BaseModel):
steps: list[Step]
completion = client.beta.chat.completions.parse(
model="gpt-4o-2024-08-06",
messages=message_dicts,
response_format=CodingOutput,
)
print(completion.choices[0].message.parsed)
Works:
message_dicts = [
{
"content": "You are an exceptional principal engineer that is amazing at finding and fixing issues in codebases.",
"role": "system",
},
{
"content": 'We need to enhance error handling in the `from_str` method.',
"role": "assistant",
},
{
"content": "Break down the task of fixing the issue into steps.",
"role": "user",
},
]
Does not work, throws a 500 HTTP error:
message_dicts = [
{
"content": "You are an exceptional principal engineer that is amazing at finding and fixing issues in codebases.",
"role": "system",
},
{
"content": "To address the issue of the malformed input string causing a `KeyError` in the `PriorityLevel` enum, we need to implement a comprehensive solution.",
"role": "assistant",
"tool_calls": [
{
"id": "call_wHhKUvGq8xMyB78PYQrqIqTp",
"function": {
"name": "expand_document",
"arguments": '{"input": "some/document.py", "repo_name": "owner/repo"}',
},
"type": "function",
},
],
},
{
"content": 'class PriorityLevel(IntEnum):\n LOW = 25\n MEDIUM = 50\n HIGH = 75\n\n def to_str(self) -> str:\n return self.name.lower()\n\n @classmethod\n def from_str(self, name: str) -> "PriorityLevel":\n return self[name.upper()]\n',
"role": "tool",
"tool_call_id": "call_wHhKUvGq8xMyB78PYQrqIqTp",
},
{
"content": 'We need to enhance error handling in the `from_str` method.',
"role": "assistant",
},
{
"content": "Break down the task of fixing the issue into steps.",
"role": "user",
},
]
from my experience, you do not add previous tool calls in the context.
You need to add the previous tool calls results in the context no? Our use case is to format only the last output.
hmm, you only add the tool call response and tool call output when you are processing it. but i believe (maybe i am wrong) that you do not add it to the main context itself. so in the sample code you gave, it is already processed and already behind two messages.
Each call with the context is independent from oneanother, it’s not processed and stored in a memory somewhere. You need to keep sending longer and longer message chains.
This works perfectly fine outside of the json schema mode.
In the assistant message that contains the “tool_calls” the content needs to be None for some reason.
Updated version:
message_dicts = [
{
"content": "You are an exceptional principal engineer that is amazing at finding and fixing issues in codebases.",
"role": "system",
},
{
"content": None,
"role": "assistant",
"tool_calls": [
{
"id": "call_wHhKUvGq8xMyB78PYQrqIqTp",
"function": {
"name": "expand_document",
"arguments": '{"input": "some/document.py", "repo_name": "owner/repo"}',
},
"type": "function",
},
],
},
{
"content": 'class PriorityLevel(IntEnum):\n LOW = 25\n MEDIUM = 50\n HIGH = 75\n\n def to_str(self) -> str:\n return self.name.lower()\n\n @classmethod\n def from_str(self, name: str) -> "PriorityLevel":\n return self[name.upper()]\n',
"role": "tool",
"tool_call_id": "call_wHhKUvGq8xMyB78PYQrqIqTp",
},
{
"content": "We need to enhance error handling in the `from_str` method.",
"role": "assistant",
},
{
"content": "Break down the task of fixing the issue into steps.",
"role": "user",
},
]
I don’t think it should be like this, it must be a bug.
Hmm this doesn’t fix it for me.
Is it because the “required” and “additionalProperties” fields are needed? Docs say that this is the case.