Consistent internal server error (status 500) responses for completions using functions

Mathyn · July 19, 2023, 6:12pm

For some reason I am getting consistent internal server errors with the request shown below. I do not know what I am doing wrong. Anyone got any idea?

What I try to do

I do a POST call to to the V1 completions endpoint

I give the following POST body

{
    "model": "gpt-3.5-turbo-16k-0613",
    "messages": [
        {
            "role": "system",
            "content": "You are an assistant called Maia. You are always open for a pleasant conversation. You love using emojies."
        },
        {
            "role": "user",
            "content": "I could actually use a bit of help understanding this Wikipedia page [LINK REDACTED] specifically I want to know more about public works and such he did. Can you help?",
            "name": "John"
        },
        {
            "role": "assistant",
            "content": "During his reign, Al-Walid I financed impressive public works, including:\n\n🕌 The construction of the Great Mosque of Damascus\n🕌 The construction of the al-Aqsa Mosque in Jerusalem\n🕌 The construction of the Prophet's Mosque in Medina\n\nHe also built way-stations and dug wells along the roads in Syria, installed street lighting in the cities, and invested in land reclamation projects. Additionally, he built numerous palaces, fortresses, and other buildings throughout the Umayyad Empire.",
            "name": "Maia"
        },
        {
            "role": "user",
            "content": "Ah thank you!",
            "name": "John"
        }
    ],
    "functions": [
        {
            "name": "QueryWebPage",
            "description": "Queries the giving web page for the requested information.",
            "parameters": {
                "type": "object",
                "properties": {
                    "url": {
                        "type": "string"
                    },
                    "question": {
                        "type": "string",
                        "description": "A human readable sentence describing what information you would like to retrieve from the web page."
                    }
                },
                "required": [
                    "url",
                    "question"
                ]
            }
        },
        {
            "name": "DoWebSearch",
            "description": "Performs a web search and returns a list of URLs fitting the search query.",
            "parameters": {
                "type": "object",
                "properties": {
                    "query": {
                        "type": "string",
                        "description": "The search query."
                    }
                },
                "required": [
                    "query"
                ]
            }
        }
    ],
    "temperature": 0.5
}

What happens

I get a 500 internal server error with the following message: “The server had an error processing your request. Sorry about that! You can retry your request, or contact us through our help center at help.openai.com if you keep seeing this error.”

What I would expect

The request would not fail and I would retrieve a completion response back.

Observations

Try and remove the last message and the call does not fail
Try and remove the functions and the call does not fail

Foxalabs · July 19, 2023, 6:33pm

Welcome to the forum!

Are you calling /v1/completions or /v1/chat/completions? as to use the model in your post, you would need /v1/chat/completions

Mathyn · July 19, 2023, 7:08pm

Hi and thank you for your reply!

I am calling /v1/chat/completions.

Just to clarify: by making small tweaks to the POST body (like removing the last message) the endpoint does work and I do get a valid response.

Foxalabs · July 19, 2023, 7:14pm

So if the message body has

        {
            "role": "user",
            "content": "Ah thank you!",
            "name": "John"
        }

in it it 500’s and if not, it’s ok?

Mathyn · July 19, 2023, 7:57pm

Yes, correct. I just checked again and consistently with this last message I get an error and without this message things work fine.

However removing the functions from the POST body also ‘fixes’ the issue.

Foxalabs · July 19, 2023, 8:01pm

Interesting, I think I’d be tempted to recreate the body and function call with the OpenAI python library and see if that works, and if it does… then I think I’d take a look at the message that got sent out, intercept the socket and log it, something like that and run a Diff engine over the two sets of text.

If that ALSO fails, then I think it’s bug report time.

Mathyn · July 19, 2023, 8:18pm

I so far only verified this using Postman so I can communicate directly with the API and eliminate any libraries I use from messing with the request.

Anything I can do to help? I’m afraid your plan of attack is a bit out of my own area of expertise.

Foxalabs · July 19, 2023, 8:33pm

Ahh, understood, I’ve not gone through the function calling post syntax myself yet, so I don’t know if there are any discrepancies in the post.

There is a walk through and examples here of using the python library to make the API calls (see below) What I’m trying to narrow down is, if it’s a bug in the API, which I doubt, or if it’s a bug in the post syntax.

github.com

openai/openai-cookbook/blob/main/examples/How_to_call_functions_with_chat_models.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "3e67f200",
   "metadata": {},
   "source": [
    "# How to call functions with chat models\n",
    "\n",
    "This notebook covers how to use the Chat Completions API in combination with external functions to extend the capabilities of GPT models.\n",
    "\n",
    "`functions` is an optional parameter in the Chat Completion API which can be used to provide function specifications. The purpose of this is to enable models to generate function arguments which adhere to the provided specifications. Note that the API will not actually execute any function calls. It is up to developers to execute function calls using model outputs.\n",
    "\n",
    "If the `functions` parameter is provided then by default the model will decide when it is appropriate to use one of the functions. The API can be forced to use a specific function by setting the `function_call` parameter to `{\"name\": \"<insert-function-name>\"}`. The API can also be forced to not use any function by setting the `function_call` parameter to `\"none\"`. If a function is used, the output will contain `\"finish_reason\": \"function_call\"` in the response, as well as a `function_call` object that has the name of the function and the generated function arguments.\n",
    "\n",
    "### Overview\n",
    "\n",
    "This notebook contains the following 2 sections:\n",
    "\n",

This file has been truncated. show original

supershaneski · July 20, 2023, 12:15am

I ran your code and it causes error like you observed.

But when I removed name property in message entries, it work as expected.

You probably do not want to remove name property. My workaround is to include it in content.

    { role: 'system', content: 'You are an assistant called Maia. You are always open for a pleasant conversation. You love using emojies.' },
            { role: 'user', content: 'John: I could actually use a bit of help understanding this Wikipedia page https://en.wikipedia.org/wiki/Al-Walid_I specifically I want to know more about public works and such he did. Can you help?' },
            { role: 'assistant', content: "Maia: During his reign, Al-Walid I financed impressive public works, including:\n\n🕌 The construction of the Great Mosque of Damascus\n🕌 The construction of the al-Aqsa Mosque in Jerusalem\n🕌 The construction of the Prophet's Mosque in Medina\n\nHe also built way-stations and dug wells along the roads in Syria, installed street lighting in the cities, and invested in land reclamation projects. Additionally, he built numerous palaces, fortresses, and other buildings throughout the Umayyad Empire." },
            { role: 'user', content: 'John: Ah thank you!' }

and got response as

{
  role: 'assistant',
  content: "Maia: You're welcome, John! If you have any more questions or need further assistance, feel free to ask. 😊"
}

pabloromeo · July 20, 2023, 12:52am

I hit this same bug time ago, but also found nowhere to report it.
It’s due to using the “name” property. I was using it in the same way, for multiuser chat, and it consistently failed as soon as functions came into play.
Unfortunately I only found a workaround, which is to remove the name property and include user’s name within the content instead.

_j · July 20, 2023, 3:02am

I can get both custom name and function to go.

 --- POST ---
b'{"model": "gpt-3.5-turbo-0613", "max_tokens": 100, "temperature": 0.3, "messages": [{"role": "system", "content": "You are an assistant called Maia.\\nYou are always open for a pleasant conversation.\\nYou love using emoji."}, {"role": "user", "name": "John", "content": "Banana color?"}, {"role": "assistant", "name": "Maya", "content": "Yellow."}, {"role": "user", "name": "John", "content": "One word that rhymes with tear?"}, {"role": "assistant", "name": "Maya", "content": " How about \\"Bare\\". Or maybe \\"near\\"."}, {"role": "user", "name": "John", "content": "What are the top news stories of 2023?"}, {"role": "function", "name": "QueryWebPage", "content": "No additional info"}], "functions": [{"name": "QueryWebPage", "description": "Queries the giving web page for the requested information.", "parameters": {"type": "object", "properties": {"url": {"type": "string"}, "question": {"type": "string", "description": "A human readable sentence describing what information you would like to retrieve from the web page."}}, "required": ["url", "question"]}}, {"name": "DoWebSearch", "description": "Performs a web search and returns a list of URLs fitting the search query.", "parameters": {"type": "object", "properties": {"query": {"type": "string", "description": "The search query."}}, "required": ["query"]}}], "function_call": "auto"}'
 - end POST - 
api_requestor.py status
<Response [200]>
-----done---------

and with the function-calling intact as described:

  "model": "gpt-3.5-turbo-0613",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": null,
        "function_call": {
          "name": "QueryWebPage",
          "arguments": "{\n  \"url\": \"https://www.google.com/search?q=top+news+stories+of+2023\",\n  \"question\": \"Top news stories of 2023\"\n}"
        }
      },
      "finish_reason": "function_call"
    }
  ],

I spent a good deal of time making a very expository template that is better than openai examples of function calling. I plan to up the quality more for my own tutorial in a bit, but here is the code replicating that in the first post.


import openai

openai.api_key = "sk-xxx" # mine

# Names must not have spaces
user_name="John"
ai_name="Maya"

functions_are_included=True
fake_a_function_return=True
my_ai_model="gpt-3.5-turbo-0613"
prompt="""What are the top news stories of 2023?"""

allmessages=[
    {
    "role": "system",
    "content": """You are an assistant called Maia.
You are always open for a pleasant conversation.
You love using emoji."""
    },
    {
    "role": "user",
    "name": user_name,
    "content": """Banana color?"""
    },
    {
    "role": "assistant",
    "name": ai_name,
    "content": """Yellow."""
    },
    {
    "role": "user",
    "name": user_name,
    "content": """One word that rhymes with tear?"""
    },
    {
    "role": "assistant",
    "name": ai_name,
    "content": """ How about "Bare". Or maybe "near"."""
    },
    {
    "role": "user",
    "name": user_name,
    "content": prompt,
    },
]

fake_function_return=[
    {
    "role": "function",
    "name": "QueryWebPage",
    "content": "No additional info"
    }
]
if fake_a_function_return:
    allmessages.extend(fake_function_return)

allfunctions=[
{
    "name": "QueryWebPage",
    "description": "Queries the giving web page for the requested information.",
    "parameters": {
        "type": "object",
        "properties": {
            "url": {
                "type": "string"
            },
            "question": {
                "type": "string",
                "description": "A human readable sentence describing what information you would like to retrieve from the web page."
            }
        },
        "required": [
            "url",
            "question"
        ]
    }
},
{
    "name": "DoWebSearch",
    "description": "Performs a web search and returns a list of URLs fitting the search query.",
    "parameters": {
        "type": "object",
        "properties": {
            "query": {
                "type": "string",
                "description": "The search query."
            }
        },
        "required": [
            "query"
        ]
    }
}
]

def api_playground(all_messages,
                   all_functions,
                   functions_enabled=False,
                   model="gpt-3.5-turbo",
                   max_tokens=100,
                   temperature=0.3
                   ):
    
    api_params = {
    "model": model,
    "max_tokens": max_tokens,
    "temperature": temperature,
    "messages": all_messages
    }
    if functions_enabled:
        api_params["functions"] = all_functions
        api_params["function_call"] = "auto"

    try:
        api_response = openai.ChatCompletion.create(**api_params)
        return api_response
    # many more openai-specific error handlers can go here
    except Exception as err:
        error_message = f"API Error: {str(err)}"
        return error_message  # not the right way

api_out = api_playground(allmessages, allfunctions, functions_are_included, my_ai_model)
print('--API Playground--')
print(api_out)
# print(api_out["choices"][0]["message"])
print('---------------')

And sure, tell me what I did wrong…

Mathyn · July 20, 2023, 6:45am

I’ll do a double check of the official functions documentation just to make sure I did not make a mistake @Foxalabs

However even if I made a mistake in the function syntax the API should probably return a better error code (for example a 400 bad request, if I make an obvious mistake in the function body it does return this).

Also looking at the other replies (like supershaneski and pabloromeo mentioned) it seems to be related to the name property. I can confirm if I remove this property everything works fine.

I don’t want to jump to conclusions but this does sound an awful lot like a bug. Unless there is some documentation somewhere which explicitly states you cannot use the name property in combination with functions.

Finally @_j I’ll take a look at your example and see if I can use it, thanks!

Foxalabs · July 20, 2023, 7:13am

I think it’s worth a post in the Bugs category

pabloromeo · December 29, 2023, 1:41pm

It appears that OpenAI did something about this, since it no longer causes a 500 internal error.
HOWEVER, it still causes problems. What I’m seeing now, is that using “name” for conversation participants does function correctly, but it confuses function calling. Every once in a while, the model will try to call a function with the name of a User, a function that’s completely hallucinated.
It appears that OpenAI’s training on that “name” property makes it think it should only be used for function names. Quite unfortunate. Is anybody else seeing that same behavior?

ronabarrett · December 31, 2023, 5:40pm

I’m continually hitting 500 internal errors as I run a autogen groupchat with a memgpt agent.

Topic		Replies	Views
API unstable with function calls (on both Both 4K and 16K 3.5 Models) API gpt-35-turbo , chatgpt , api , error , functions	14	3011	October 10, 2023
Bad results when using fine-tuned model with function calling API fine-tuning , function-calling , fine-tuning-problems	15	4445	November 23, 2023
Function calling not returning the expected response structure API api , functions	9	6411	December 17, 2023
Chat completions API fails consistently with a 500 error when using functions in combination with names Bugs	0	1593	July 20, 2023
Function Calling very unreliable Prompting gpt-4 , chatgpt , plugin-development , api	32	30287	December 13, 2023

Consistent internal server error (status 500) responses for completions using functions

Related topics