Hi,
I’m building a GPTs assistant API that receives data in JSON that the user uploaded (as csv) to GPT-4. I’m positive that everything on my side is implemented correctly and sometimes this API call succeeds. However, the success-rate is maybe 20%. Most of the time I get either a client or server-side error about the JSON being formatted incorrectly (it isn’t, I checked):
- “Invalid control character”
- “Unexpected character found when decoding array value (2)”
GPT-4 can decode the JSON it created just fine, only when sending it and unjsonifying it on the server side does the error occur, but only sometimes. If I send a shorter payload, it seems to work more reliably. I’m staying under the (100,000 char) limit.
The data itself is a pandas DataFrame that GPT jsonifies via:
data.to_json(orient='split', index=False)
and on the server is read via:
pd.read_json(io.StringIO(data_json), orient="split")
Here is an example of the JSON:
'{"columns":["date_week","sales","x1","x2"],"data":[["2018-04-02",3984.66,0.32,0.0],["2018-04-09",3762.87,0.11,0.0],["2018-04-16",4466.97,0.29,0.0],["2018-04-23",3864.22,0.07,0.0]...
So I ruled out everything I can on my side, the only explanation left is that OpenAI is somehow messing with the JSON before it gets sent.
Anyone run into this or has ideas for how I can debug this further?