Getting 500 with gpt-4o and tools

API
1

I am trying to use gpt-4o with tools but I am Consistently Getting:

Failed: Error code: 500 - {'error': {'message': 'The server had an error processing your request. Sorry about that! You can retry your request, or contact us through our help center at help.openai.com if you keep seeing this error. (Please include the request ID req_da312b4f9976465709784838d88fdcdf in your email.)', 'type': 'server_error', 'param': None, 'code': None}}

{'model': 'gpt-4o',
 'temperature': 0,
 'max_tokens': 512,
 'top_p': 1,
 'n': 1,
 'tools': [{'type': 'function',
   'function': {'name': 'google_search_insights',
    'description': 'You are required to summarize the context provided along with citation for the required brand by only considering relevant resources',
    'parameters': {'type': 'object',
     'properties': {'is_any_relevant_info': {'type': 'boolean',
       'description': 'Return False if No valid point can be retrieved from the resources provided'},
      'all_citations': {'type': 'array',
       'items': {'type': 'string'},
       'description': 'List of all the urls/citations used in build up the relevant insights for the provided brand'},
      'agent_response': {'type': 'string',
       'description': 'Agent response containing all the relevant insights for provided brand'}}}}}],
 'messages': [{'role': 'user',
   'content': 'Not giving the actual text as it contains ~ 9913 tokens'}]}

Note: the User Message contains around 9913 tokens.
The same request is giving a response without tools.

Please suggest the required steps to resolve this.

2

Try: temperature and top_p at 0.1

Try: max_tokens at 2000

The server error is usually when the AI has written something as a tool output that is not a valid tool call or JSON. With the poor quality of this model, an additional 10k may be enough that it cannot still understand how to use the function. You can provide a much longer function description, elaborating on how to use it, what it will return, the types of queries to write, how characters should be escaped or written in URL encoding, etc. to re-focus attention on using the tool properly.

3

Thanks.
It is working with maxToken: 2000 but generating a response with tokens < 500
with maxToken: 1024, it failed on the 9/10 request.
But with model: gpt-4-32k and maxToken: 1024, the same request passes with a valid tools response.

is it the problem with gpt-4o and not the tools?

4

It very well could be the model. Tool calls, once invoked by the first token the AI produced, need to be generated in a manner that will validate.

OpenAI blocks you from seeing what the AI produced as a tool output with their server error, so you have little hope of debugging what the AI did wrong, where their training went wrong, or holding it accountable.

GPT-3.5-turbo-0613 can use tools with no problems, especially in the original version before it also got hit with degradation. 10x the cost for one that can’t is silly.

Did you try my particular temperature and top_p?

5

yes.
but it seems like the focus is not on temperature and top_p, but rather on validating whether the schema of the tools is correct, which is getting controlled by increasing the maxTokens.

6

I’m getting the exact same 500 error consistently. It’s around 6 tools defined and expecting multiple tools called (ideally in parallel). It’s only GPT-4o failing

GPT-4-turbo works, GPT-3.5-turbo works (although super low quality, doesn’t follow prompt). I’ve triple checked my tool schema definitions, upgraded maxTokens to 2048, and tried everything else in this thread ^. Nothing worked so far. Stuck with GPT-4-Turbo w/ the higher latency.