Hi,
I am unable to use gpt-4o or any other models for my use because of 1024 character limit on tool call descriptions. We auto generate the descriptions and are advertently verbose because its valuable for the work we are doing.
Are you sure that is the case? Iām able to send much larger descriptions without any issues.
import tooldantic as td
import typing
import openai
client = openai.OpenAI()
class Test(td.OpenAiBaseModel):
test: str = td.Field(description="A test field." * 3000)
r = client.chat.completions.create(
model="gpt-4o-mini",
tools=[Test.model_json_schema()],
messages=[
{"role": "user", "content": "thanks"},
],
)
What is the version of your open ai client?
File "/Users/prannayk/Library/Caches/pypoetry/virtualenvs/sentinel-gjiWBBWm-py3.10/lib/python3.10/site-packages/openai/_base_client.py", line 1049, in _request
raise self._make_status_error_from_response(err.response) from None
openai.BadRequestError: Error code: 400 - {'error': {'message': "Invalid 'tools[1].function.description': string too long. Expected a string with maximum length 1024, but got a string with length 1090 instead.", 'type': 'invalid_request_error', 'param': 'tools[1].function.description', 'code': 'string_above_max_length'}}
Aha, your model json schema is setting the description of the parameter of the tool, not the description of the tool itself. I tried your test case and your description of the tool name itself is empty.
In [27]: Test.model_json_schema()['function']['description']
Out[27]: ''
1 Like
Are you sure thatās the case? Not using home-made libraries and sending so it actually is structured-activating:
import os, httpx, json
api_key = os.getenv("OPENAI_API_KEY")
headers = {
"Authorization": f"Bearer {api_key}",
"Content-Type": "application/json",
}
data = {
"model": "gpt-4o-2024-08-06", "max_tokens": 200,
"messages": [
{"role": "user", "content": "Who's a good bot? YOU!"},
],
"tools": [
{
"type": "function",
"function": {
"name": "get_current_weather",
"description": "Get the current weather on a planet. " * 100,
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The planet, e.g. Jupiter"
},
"unit": {
"type": "string",
"enum": ["celsius", "kelvin"],
"description": "Temperature unit"
}
},
"required": ["location", "unit"],
"additionalProperties": False
},
"strict": True
}
}
],
"tool_choice": "auto"
}
try:
response = httpx.post("https://api.openai.com/v1/chat/completions",
headers=headers,
data=json.dumps(data))
response.raise_for_status()
except httpx.HTTPStatusError as exc:
print(f"Error response {exc.response.status_code} while requesting {exc.request.url!r}.")
print(f"Response body:\n{exc.response.text}")
else:
print(response.json())
Error response 400 while requesting URL('https://api.openai.com/v1/chat/completions').
Response body:
{
āerrorā: {
āmessageā: āInvalid ātools[0].function.descriptionā: string too long. Expected a string with maximum length 1024, but got a string with length 3700 instead.ā,
ātypeā: āinvalid_request_errorā,
āparamā: ātools[0].function.descriptionā,
ācodeā: āstring_above_max_lengthā
}
}
2 Likes
Yes. Tool descriptions should be succinct and provide a brief directive for the ai. Example, āUse this tool to do x when yā, which is important for tool selection. As for the parameters, this is where you want to get deep into the embedding of system instructions into the tool.
1 Like
Thanks for the clarification, @_j . I see where I made the mistakeāyouāre absolutely right about the 1024-character limit being on the function description itself, while I was thinking about the parameter descriptions, which donāt have that same restriction. So, I appreciate you pointing that out.
But this brings up a bigger questionāis the 1024-character limit on the function description really a limitation? From everything Iāve seen, especially in OpenAIās examples, the model seems tuned to use the function description as a short directive for tool selection, while the parameters are where the real detailed guidance happens. If you truly need more tool description verbosity, you could always supplement the function description with a more detailed system message. But Iād argue that the most important thing is having well-described parametersāthose are what the model uses to drive its generation through the constrained outputs.
So, Iād challenge the assumption that the 1024-character limit is a hard barrier. If you can show me a case where a function description over 1024 characters is necessary, Iād be happy to counter with an example where a combination of a system message and thorough parameter descriptions could yield even better results.
Iād push back on the āhome-madeā label. Tooldantic is not some cobbled-together solution. Itās a thin, targeted extension of pydantic, which is the gold standard for validation in Python. All it does is make pydantic more suited to handling LLM outputs in a clean and structured way. Every part of it runs through pydantic, so itās built on a foundation thatās already widely trusted and used in the Python community. Far from being āhome-made,ā itās just adding a specific layer of utility to a tool thatās already deeply ingrained in the ecosystem.
At the end of the day, pydantic is the core hereāso if you trust pydantic (and most Python devs do), tooldantic is just an extension that makes it easier to work with LLMs.
1 Like
Yes, it is an odd case to expect that beyond 1000 characters is required for āReturns OpenAI developer support solutionsā, when you are guiding the production of language in the properties. The description should be a ācall or noā decision point. If you want to double or triple that 250 English tokens placed into AI context, write it in Chinese, though.
However, client libraries shouldnāt truncate input without error, and must also operate correctly and transparently to activate the desired modality.
1 Like