JSON erros: "Invalid control character" and "Unexpected character found when decoding array value (2)"


I’m building a GPTs assistant API that receives data in JSON that the user uploaded (as csv) to GPT-4. I’m positive that everything on my side is implemented correctly and sometimes this API call succeeds. However, the success-rate is maybe 20%. Most of the time I get either a client or server-side error about the JSON being formatted incorrectly (it isn’t, I checked):

  • “Invalid control character”
  • “Unexpected character found when decoding array value (2)”

GPT-4 can decode the JSON it created just fine, only when sending it and unjsonifying it on the server side does the error occur, but only sometimes. If I send a shorter payload, it seems to work more reliably. I’m staying under the (100,000 char) limit.
The data itself is a pandas DataFrame that GPT jsonifies via:
data.to_json(orient='split', index=False)
and on the server is read via:
pd.read_json(io.StringIO(data_json), orient="split")

Here is an example of the JSON:

So I ruled out everything I can on my side, the only explanation left is that OpenAI is somehow messing with the JSON before it gets sent.

Anyone run into this or has ideas for how I can debug this further?