The submit_output_tools returns an error where the type is "server_error" but the message is ""

API
,
1

The assistant/thread/run process is generally working fine. But i find a consistent error that happens after the 4th round of call backs (that is it calls my functions, I react, it calls my function, I react, it calls my functions, i react, and then on the forth round, I get this error). On this round, when i respond via the submit_output_tools method (as I did in the prior rounds) I get back this error:

The remote server returned an error: (400) Bad Request.
{
“error”: {
“message”: “”,
“type”: “server_error”,
“param”: null,
“code”: null
}
}

Note: The submitted JSON is valid. The total length of the JSON is 5KB or less. The lack of a message makes it rather hard to know what this is about.

Here’s an example of the exact payload I’m sending back:

{
  "tool_outputs": [
    {
      "tool_call_id": "call_svRXz1urN7pFG9ghxLN6QOtN",
      "output": "Content For 'https://www.investopedia.com/articles/personal-finance/100214/retirement-strategies-low-income-seniors.asp', Page 1 of Estimated 77\r\nINVESTING\nSIMULATOR\nBANKING\nPERSONAL FINANCE\nNEWS\nREVIEWS\nACADEMY\nLIVE\nTable of Contents\nRetiring with Low Income\nThe Programs You Know\nOther Programs for Retirees\nLow-Cost Lifestyle\nControlling Your Finances\nThe Bottom Line\nPERSONAL FINANCE  RETIREMENT\r\n\r\nThere is more content to be read for this item, call ReadNextPage(117153) to get next page\r\n"
    },
    {
      "tool_call_id": "call_W1sRJXbzdtZBEW65m5WsXxH2",
      "output": "Content For 'https://www.newretirement.com/retirement/retirement-income-strategies/', Page 1 of Estimated 146\r\nPlanning\n\nFinancial Planning\nFinancial planning tools and services to put you on the path to the future you want\n\nOUR PLANNER\n Get Started\n How It Works\n Customer Stories\nSERVICES\nClasses\nCoaching\nFinancial Advisor\nCALCULATORS\nSimple Retirement Calculator\r\n\r\nThere is more content to be read for this item, call ReadNextPage(117154) to get next page\r\n"
    },
    {
      "tool_call_id": "call_pfDySStDxywuQZfiRblRm9Na",
      "output": "Content For 'https://smartasset.com/retirement/top-11-retirement-strategies', Page 1 of Estimated 105\r\nHOME BUYING\nTAXES\nRETIREMENT\nFINANCIAL ADVISORS\nBANKING\nINVESTING\nSMARTREADS\nMORE\nI'm an Advisor\nFind an Advisor\nSEARCH\nTop 10 Retirement Strategies You Need to Know\nUpdated on August 28, 2023\nWritten by Javier Simon, CEPF®\nSHARE\n\nWhether you’re in your\r\n\r\nThere is more content to be read for this item, call ReadNextPage(117155) to get next page\r\n"
    },
    {
      "tool_call_id": "call_A5Bjxlcd795i2arRFx2YNATu",
      "output": "Content For 'https://www.investopedia.com/articles/personal-finance/110315/4-phases-retirement-and-how-budget-them.asp', Page 1 of Estimated 87\r\nINVESTING\nSIMULATOR\nBANKING\nPERSONAL FINANCE\nNEWS\nREVIEWS\nACADEMY\nLIVE\nTable of Contents\nPre-Retirement\nEarly Period of Retirement\nMiddle Retirement\nLate Retirement\nFrequently Asked Questions (FAQs)\nThe Bottom Line\nBUDGETING & SAVINGS  BUDGETING\nBudgeting\r\n\r\nThere is more content to be read for this item, call ReadNextPage(117156) to get next page\r\n"
    },
    {
      "tool_call_id": "call_Z05zsG2jjkJLyLHrWroewK1D",
      "output": "Content For 'https://www.fidelity.com/viewpoints/retirement/budgeting-in-retirement', Page 1 of Estimated 89\r\nSkip to Main Content.\nFidelity.com Home\nCUSTOMER SERVICE\tPROFILE\tOPEN AN ACCOUNT\tVIRTUAL ASSISTANT\tLOG IN\nAccounts & Trade\nPlanning & Advice\nNews & Research\nProducts\nWhy Fidelity\nLearn\nLearn\nFavorites\nTopics\nEvents\nSearch\nAugust 02, 2022\nSave\nShare\nPrint\r\n\r\nThere is more content to be read for this item, call ReadNextPage(117157) to get next page\r\n"
    },
    {
      "tool_call_id": "call_POlbY0ROKMjhH7ixmjeAhfAH",
      "output": "Content For 'https://www.blackrock.com/us/individual/education/retirement/building-a-retirement-budget', Page 1 of Estimated 59\r\nSkip to content\nBlackRock\niShares\nAladdin\nOur company\nIndividual Investors \nUnited States \nSign In\nFunds\nInvestment strategies\nInsights & education\nAbout us\nUSING YOUR SAVINGS\nDetermining your budget in retirement\nBuilding your budget Considerations\r\n\r\nThere is more content to be read for this item, call ReadNextPage(117158) to get next page\r\n"
    },
    {
      "tool_call_id": "call_aprwfHjTUCShSxz3xkXE69o8",
      "output": "Content For 'https://www.nerdwallet.com/article/investing/retirement-planning-an-introduction', Page 1 of Estimated 240\r\nSkip to content\nCredit cards\nBanking\nHome\nLoans\nInsurance\nPersonal finance\nInvesting\nSmall business\nSIGN IN\nSIGN UP\nINVESTING\n5 STEPS TO RETIREMENT PLANNING IN 2024: AN INTRODUCTION & HOW-TO GUIDE\nAdvertiser disclosure\n5 Steps to Retirement Planning in\r\n\r\nThere is more content to be read for this item, call ReadNextPage(117159) to get next page\r\n"
    },
    {
      "tool_call_id": "call_eUduXT4iHVNTjypafGue5KaO",
      "output": "Content For 'https://www.empower.com/the-currency/life/guide-to-retirement-planning', Page 1 of Estimated 74\r\nSkip to main content\nWM 2.0 - Blogs\nThe Currency\nPress center\nSearch\n\nWednesday, January 10, 2024\n\nWM 2.0 Blog - Currency Categories\nMoney\nLife\nWork\nPlay\nBreadcrumb\nThe Currency  Life  Seven essential steps for retirement planning\n\nSeven essential steps\r\n\r\nThere is more content to be read for this item, call ReadNextPage(117160) to get next page\r\n"
    },
    {
      "tool_call_id": "call_B9GMkLPL9J5FlELfJZJgO2bS",
      "output": "Content For 'https://www.consumerfinance.gov/consumer-tools/retirement/', Page 1 of Estimated 32\r\nSkip to main content\nAn official website of the United States government\nEspañol ?? Ti?ng Vi?t ??? Tagalog P?????? ??????? Kreyòl Ayisyen (855) 411-2372\n Submit a Complaint\n Search\nConsumer Education\n\t\nRules & Policy\n\t\nEnforcement\n\t\nCompliance\n\t\nData &\r\n\r\nThere is more content to be read for this item, call ReadNextPage(117161) to get next page\r\n"
    }
  ]
}

Ideally, I wish it would not make these requests in parallel but I did not see any way to prevent it, and it doesn’t seem to honor a prompt request not to do this.

Has anyone run into this issue and better yet, anyone know a way to find out what the error is?

Thanks,

Ben

2

Hi and welcome to the Developer Forum!

The issues around assistants is known, there is ongoing development of the assistants subsystems and we should start to see updates in due course. I don’t have any timelines as yet, but it is being looked at and worked on.

2 Likes
3

That’s excellent. Thank you. I did try to see if I could find the issue in the forum first, but I didn’t see anything that matched. I’m just glad to know there’s awareness.

4

Reading through your posting, it seems as although the AI emitted four parallel tool calls, you are only returning one at a time, making the AI retry against your error in fulfillment. You should fulfill the tools in parallel and return all at once.

This link has code of mine that demonstrates returning back to the AI the assistants tools it emitted in parallel along with the tool return that contains all parallel responses.

5

I am returning the calls all at once, per the documentation. What I was saying is that ideally I’d have a way to tell it not to parallelize that, because you can end up with a huge payload. But you can see I have each ID and they are in an array.

I think you misunderstood my description. I was showing that in general the back and forth works properly. the AI calls I response, the AI calls I respond (those calls might have 10 function requests in them each, but as a “round” I am responding). When I get to the 4th round, something internal in openai seems to go bad. In the example I showed, the AI was not making 4 calls it was making 9 in parallel and I was responding in-kind)

6

In the tool description would be the place to document the correct procedures for the tool method operation.

OpenAI has turned writing function descriptions of theirs into multiline meandering stores. You can give the AI a whole bunch of lines of numbered lists and nested bullet points about exactly what the function is doing and how many iterations AI can make before answering, etc. And that multi-tool must contain one method per call.

7

I have not found a prompt that effectively tells the AI to call this function only one at a time. It will parallel what it thinks it can parallel and in fact, on this call, it is correct, there’s no reason it can’t run 5 at a time or 20 at a time, but the payload is large in response for each of these. It would make more sense to have the AI process one at a time. Its possible the answer to one could make seeking the next one irrelevant. Another solution is that I could dump the results from these calls into a new thread. And then summarize the activity so far into that new thread and start a new Run. However, i hesitant to do that, because I don’t know the nature of this error. Since I paged the content being returned, the total size is roughly 1500 tokens give or take. I would not expect that to cause issue. Since there’s no “message” associated with the error, until I understand the nature of the error better, I am reluctant to change my process.

8

I wonder if this is a case where the tool ids might have been reused in the thread. You’d think with the long random-looking characters that would never be an issue, but it’s been reported more than once. The endpoint refuses on “non-matching” ids if they were also seen earlier in model context.

9

I’m not sure but I think its too consistent for it to be an ID collision issue. Happens at round 4, that is, when the status has gone:
queued->in_progress->required_action->queued->in_progress->required_action->queued->in_progress->requried_action->queued->in_progress->requried_action

then it errors on the call to the submit_output_tools end point. I have tried a few tricks like cancelling the run, and starting another on the same thread at the point the error occurs, but that immediately gets the same answer on the subsequent requires_action status. This implies that the thread itself is in a bad state.

I do appreciate your help in trying to solve it. Without the message, its like having canned vegetables without labels. Hard to tell what’s inside. The assistant system as a whole works great most of the time.