Assistants API often generates function calls with invalid JSON

API Bugs
1

Did anyone else notice recently any issues with Assistants API where it outputs invalid function calls? It usually happens when the arguments JSON is quite long.

What I have seen so far:

  • 3 dots at the end,
  • special text that there are more chars (example: ,{“symb… 250 more characters,”:}]} ),
  • some very long text at the end (example: Assistant AI:6.1 Assistant AI:6.1 Assistant AI:6.1 Assistant AI:6.1 Assistant AI:6.1 Assistant AI:6.1 Assistant AI:6.1 Assistant AI:6.1 Assistant AI:6.1 Assistant AI:6.1 Assistant AI:6.1 Assistant AI:6.1),
  • lots of \n or \t.

When I then say that the JSON is invalid, it does not correct itself.

This was not happening in the past and started recently. It happens very often (more than 50% of time).

1 Like
2

Some other cases:
"w… MULTIPART_TRUNCATED _True Dueling Duals 2023-04-07 17:53:47 2023-04-07 17:54:15 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

or:
{\“symb…
[See more]
\ucd5c\uc2e0\ubc84\uc804
\uc0c8\ub85c\uc6b4
urllib
\uc73c\ub2c8
\uba70, incidentally

It looks as if the temperature is set to high and it outputs gibberish.

Also, sometimes it starts with a correct JSON and then continues outputting function schema instead.

1 Like
3

I don’t think the assistants API as it exists is long for this world. It has never really made sense, it takes longer than normal chat calls, and there are other issues including your own. I’m sure some people are making it work but overall when it comes to just chat, Assistants doesn’t hold up. Assistants using RAG and code interpreter are cool features but still.

4

To me, it looks like some internal stuff is leaking. Just look at this:

{\"symb...alborg/fwlink/functions.save_portfolio/dd567c54-5e4a-4e7a-aeb9-053473f6a95f`_METADATA_RemoteContent-Truncated`_RETRY_LATER` \uc785\ub2c8\ub2e4.  `_RETRY_`  `_METADATA_`  `_TRUNCATED_`  `_METADATA_`  `_TRUNCATED_`  `_METADATA_`  `_TRUNCATED_`  `_METADATA_`  `_TRUNCATED_`  `_METADATA_`  `_TRUNCATED_`  `_METADATA_`  `_TRUNCATED_`  `_METADATA_`  `_TRUNCATED_`  `_METADATA_`  `_TRUNCATED_`  `_METADATA_`  `_TRUNCATED_`  `_METADATA_`  `_TRUNCATED_`  `_METADATA_`  `_TRUNCATED_`  `_METADATA_`  `_TRUNCATED_`  `_METADATA_`  `_TRUNCATED_`  `_METADATA_`  `_TRUNCATED_`  `_METADATA_`  `_TRUNCATED_`  `_METADATA_`  `_TRUNCATED_`  `_METADATA_`  `_TRUNCATED_`  `_METADATA_`  `_TRUNCATED_`  `_METADATA_`  `_TRUNCATED_`  `_METADATA_`  `_TRUNCATED_`  `_METADATA_`  `_TRU...  `_METADATA_`  `_TRUNCATED_`  `_METADATA_`  `_TRUNCATED_`  `_METADATA_`  `_TRUNCATED_`  `_METADATA_`  `_TRUNCATED_`  `_METADATA_`  `_TRUNCATED_`  `_METADATA_`  `_TRUNCATED_`  `_METADATA_`  `_TRUNCATED_`  `_METADATA_`  `_TRUNCATED_`  `_METADATA_`  `_TRUNCATED_`  `_METADATA_`  `_TRUNCATED_`  `_METADATA_`  `_TRUNCATED_`  `_METADATA_`  `_TRUNCATED_`  `_METADATA_`  `_TRUNCATED_`  `_METADATA_`  `_TRUNCATED_`  `_METADATA_`  `_TRUNCATED_`  `_METADATA_`  `_TRUNCATED_`  `_METADATA_`  `_TRUNCATED_`  `_METADATA_`  `_TRUNCATED_`  `_METADATA_`  `_TRUNCATED_`  `_METADATA_`  `_TRUNCATED_`  `_METADATA_`

The beginning is suspicious. The function that is being called is save_portfolio.

5

It looks like only the latest model is affected. I have not yet see this issue on gpt-4-1106-preview.

Could someone from OpenAI team take a look at this? Using gpt-4-0125-preview leads to way more processed tokens as usually the function call arguments start with a proper json and then some gibberish is generated, followed by a lot of whitespaces at the end. Similar task with 1106 consumes only 11659 in, 2404 out tokens while 0125 uses 19463 in, 8321 out tokens.

1 Like