I developed a small chat app that uses functions like Google search, URL download, file access, etc. Just basic implementation to experiment with tools. The mechanism works beautifully, but the OpenAI API response is (very) unreliable, failing completely about 50% of the times randomly when functions are used. Digging into the code I found that as soon as a function is defined in the request, (and not necessarily when the model uses it) the API becomes sketchy and responds error messages.
Here’s a screenshot of how things looks when it works, and below some examples of when id doesn’t.
"data":{
"error":{
"message":"The server had an error processing your request. Sorry about that! You can retry your request, or contact us through our help center at help.openai.com if you keep seeing this error. (Please include the request ID 62f54e5eefffc42225377c7b75098ecc in your email.)",
"type":"server_error",
"param":null,
"code":null
}
}"data":[
"DONE"
]
I contacted openAI and they suggested I hit the rate limit - but I’m the only one using the account as a developer so I know that’s not the case, and besides removing the function definitions solves the issue immediately. Here’s an example of the full request and response:
Request Body:
{
"messages":[
{
"role":"system",
"content":"\u0022You are Chat GPT, a versatile AI assistant specializing in Agent with Tools.\r\n .\u0022"
},
{
"role":"user",
"content":"hi",
"name":"user"
},
{
"role":"assistant",
"content":"Hello! How can I assist you today?",
"name":"assistant"
},
{
"role":"user",
"content":"how are you?",
"name":"user"
},
{
"role":"assistant",
"content":"Thank you for asking! As an AI, I don\u0027t have feelings, but I\u0027m here and ready to help you with any questions or tasks you have. How can I assist you today?",
"name":"assistant"
},
{
"role":"user",
"content":"tell me about your team",
"name":"user"
},
{
"role":"user",
"content":"hi",
"name":"user"
}
],
"functions":[
{
"name":"googleSearch",
"description":"Search the web with Google",
"parameters":{
"type":"object",
"properties":{
"SearchTerm":{
"type":"string",
"description":"Search term to look for"
}
},
"required":[
"SearchTerm"
]
}
},
{
"name":"sendEmail",
"description":"Send an Email to an Email address",
"parameters":{
"type":"object",
"properties":{
"EmailAddress":{
"type":"string",
"description":"Email Address of Recepient"
},
"Subject":{
"type":"string",
"description":"Subject of Email "
},
"EmailBody":{
"type":"string",
"description":"Content of Email "
}
},
"required":[
"EmailAddress",
"Subject",
"EmailBody"
]
}
}
],
"stream":true,
"model":"gpt-3.5-turbo-16k",
"temperature":0.1,
"function_call":"auto"
}
Response Headers:
Date : wed, 21 jun 2023 00:26:51 gmt content-type : text/event-stream transfer-encoding : chunked connection : close access-control-allow-origin : * cache-control : no-cache, must-revalidate openai-organization : XXXXX openai-processing-ms : 26 openai-version : 2020-10-01 strict-transport-security : max-age=15724800; includesubdomains x-ratelimit-limit-requests : 3500 x-ratelimit-limit-tokens : 180000 x-ratelimit-remaining-requests : 3499 x-ratelimit-remaining-tokens : 179895 x-ratelimit-reset-requests : 17ms x-ratelimit-reset-tokens : 35ms x-request-id : XXX cf-cache-status : dynamic server : cloudflare cf-ray : XXX alt-svc : h3=":443"; ma=86400
Response Body:
"data":{
"error":{
"message":"The server had an error processing your request. Sorry about that! You can retry your request, or contact us through our help center at help.openai.com if you keep seeing this error. (Please include the request ID 62f54e5eefffc42225377c7b75098ecc in your email.)",
"type":"server_error",
"param":null,
"code":null
}
}"data":[
"DONE"
]
Does any of you encounter similar issues? Any ideas? the only reference I could found was on Izmu’s blog: 'Function calling', OpenAI's API new feature.
