Here is the case. I’m using an Action that uses GET API endpoint and gets a text in response. Everything works fine, but if there is too much text, there is an error saying: “The response size appears to be too large for processing”
How to deal with these limits? What are they exactly for a single API response?
Have anybody struggled with such a problem and do you know if GPT could manage partial response retrieval?
For example, for one single API request, the response will have multiple file/text segments in response. Has anybody tried this?
Keep in mind, that it’s not a token limit or anything, because if I manually upload the same text to GPT, the retrieval and other stuff work as intended.
Will appreciate any relevant help and knowledge to solve this. Thanks!
My custom API returns a lot of text and I finally just ran into this limit myself. In the failed return, the data size is about 109k and 21k tokens. I believe the poster above is correct, the limit is 100k bytes.
I am going to be doing a number of tests today to determine if that is correct.
If you are hitting a custom written API, you can modify it to allow for server side pagination of data, i.e. viewing smaller chunks at a time. You’ll also have to incorporate instructions to utilize the pagination when a request is made. These instructions would be added to the GPT’s Action description.
I use the Actions GPT to write both my API script and Action description in tandem so they integrate with each other properly. Just like any prompt, ensuring you are quite verbose in describing what you want to achieve is important when these requests are made to return data via your API.
For instance, my API pulls data from a Google Sheet that is ordered by date. I have to specify that I want data retrieved starting from the most recent date, and also that at times I want all data to be viewed, instead of just X amount of pages until my GPT believes it has what it wants.
