I’m creating a GPT calling a custom API. All requests to the API fail with a “ResponseTooLargeError” status.
These same calls were working yesterday. The API response doesn’t seem huge to me (< 1000 tokens).
Is this a bug or is there a new limit on the size of the response GPTs can handle ?
Seeing that as a common issue right now, thanks for reporting it.
It would be awesome if the actions could take parameters to omit fields on the response to “sanitize” it before being returned. In many cases a lot of the data is redundant.
I had a thought that it would be cool to use hardcoded code interpreter scripts for action post-processing here Response Filtering for Actions .
Though the response too large error I get there is probably more legitimate, because the Github API I was using includes a lot of extra data.
Got the same error. Looks like the github action api returns a lot of additional info. another way to analyse via gpt3.5-turbo-16k? or another cheap model with a large context? or use simple xpatch parser
Try to use json.dumps(portrait, separators=(“,”, “:”)) for serizliation. It removes spaces in result json str. Worked for me.
I have the same issue.
My JSON scheme includes multiple paths, each path to another entity in the datamodel. Some paths within the same JSON scheme work, most don’t and give a ResponseTooLargeError".
When I ask the GPT to return to me the HTTP Endpoint URL of the erroneous query and I execute that URL in a browser the result is correct. When I copy that result and paste it in ChatGPT, ChatGPT states that the result is just approx. 300 tokens. The length shouldn’t be a problem.
For the sake of it, I tried limits and offsets, it didn’t help.
Tried to limit the amount of return fields, didn’t help.
Tried json.dump to the response: didn’t help.
But still, a few paths work properly.
Hello, do you need an example? I can provide one.
Examples are always useful, please list any you have.
The server is owned by the Dutch parlement. All parlement data, motions, documentation, etc. is available via an OpenAPI interface. It’s a complex datamodel, I’m using this datamodel to experiment how my GTP would implement an execution plan when asking information using multiple entities with their relationships.
So the example JSON scheme includes 2 paths to 1) /Persoon (Dutch for Person) and 2) /Fractie (Dutch for Group / Party).
So the question “Who is Geert Wilders?” (the politician who won the elections last week) will result into the error. But the question "Give information about the political party ‘Partij voor de Dieren’ will execute and return the correct answers from the /Fractie path.
The documentation about the OpenAPI interface can be found here https://opendata.tweedekamer.nl/documentatie/odata-api The documentation is in Dutch, a translator is your friend.
{
"openapi": "3.1.0",
"info": {
"title": "Politics API",
"description": "API for accessing data from gegevensmagazijn.tweedekamer.nl",
"version": "v1.0.0"
},
"servers": [
{
"url": "https://gegevensmagazijn.tweedekamer.nl/OData/v4/2.0"
}
],
"paths": {
"/Fractie": {
"get": {
"operationId": "getFractieData",
"parameters": [
{
"name": "Nummer",
"in": "query",
"required": false,
"description": "Filter factions by number",
"schema": {
"type": "integer"
}
},
{
"name": "Afkorting",
"in": "query",
"required": false,
"description": "Filter factions by abbreviation",
"schema": {
"type": "string"
}
},
{
"name": "NaamNL",
"in": "query",
"required": false,
"description": "Filter factions by Dutch name",
"schema": {
"type": "string"
}
},
{
"name": "NaamEN",
"in": "query",
"required": false,
"description": "Filter factions by English name",
"schema": {
"type": "string"
}
},
{
"name": "AantalZetels",
"in": "query",
"required": false,
"description": "Filter factions by number of seats",
"schema": {
"type": "integer"
}
},
{
"name": "AantalStemmen",
"in": "query",
"required": false,
"description": "Filter factions by number of votes",
"schema": {
"type": "integer"
}
},
{
"name": "DatumActief",
"in": "query",
"required": false,
"description": "Filter factions by active date",
"schema": {
"type": "string",
"format": "date"
}
},
{
"name": "DatumInactief",
"in": "query",
"required": false,
"description": "Filter factions by inactive date",
"schema": {
"type": "string",
"format": "date"
}
}
],
"responses": {
"200": {
"description": "Successful response"
}
}
}
},
"/Persoon": {
"get": {
"operationId": "getPersoonData",
"parameters": [
{
"name": "Id",
"in": "query",
"required": false,
"description": "Filter persons by ID",
"schema": {
"type": "string"
}
},
{
"name": "Nummer",
"in": "query",
"required": false,
"description": "Filter persons by Nummer",
"schema": {
"type": "integer"
}
},
{
"name": "Titels",
"in": "query",
"required": false,
"description": "Filter persons by Titels",
"schema": {
"type": "string"
}
},
{
"name": "Initialen",
"in": "query",
"required": false,
"description": "Filter persons by Initialen",
"schema": {
"type": "string"
}
},
{
"name": "Tussenvoegsel",
"in": "query",
"required": false,
"description": "Filter persons by Tussenvoegsel",
"schema": {
"type": "string"
}
},
{
"name": "Achternaam",
"in": "query",
"required": false,
"description": "Filter persons by last name",
"schema": {
"type": "string"
}
},
{
"name": "Voornamen",
"in": "query",
"required": false,
"description": "Filter persons by Voornamen",
"schema": {
"type": "string"
}
},
{
"name": "Roepnaam",
"in": "query",
"required": false,
"description": "Filter persons by Roepnaam",
"schema": {
"type": "string"
}
},
{
"name": "Geslacht",
"in": "query",
"required": false,
"description": "Filter persons by Geslacht",
"schema": {
"type": "string"
}
},
{
"name": "Functie",
"in": "query",
"required": false,
"description": "Filter persons by Functie",
"schema": {
"type": "string"
}
},
{
"name": "Geboortedatum",
"in": "query",
"required": false,
"description": "Filter persons by Geboortedatum",
"schema": {
"type": "string",
"format": "date"
}
},
{
"name": "Geboorteplaats",
"in": "query",
"required": false,
"description": "Filter persons by Geboorteplaats",
"schema": {
"type": "string"
}
},
{
"name": "Geboorteland",
"in": "query",
"required": false,
"description": "Filter persons by Geboorteland",
"schema": {
"type": "string"
}
},
{
"name": "Overlijdensdatum",
"in": "query",
"required": false,
"description": "Filter persons by Overlijdensdatum",
"schema": {
"type": "string",
"format": "date"
}
},
{
"name": "Overlijdensplaats",
"in": "query",
"required": false,
"description": "Filter persons by Overlijdensplaats",
"schema": {
"type": "string"
}
},
{
"name": "Woonplaats",
"in": "query",
"required": false,
"description": "Filter persons by Woonplaats",
"schema": {
"type": "string"
}
},
{
"name": "Land",
"in": "query",
"required": false,
"description": "Filter persons by Land",
"schema": {
"type": "string"
}
},
{
"name": "Fractielabel",
"in": "query",
"required": false,
"description": "Filter persons by Fractielabel",
"schema": {
"type": "string"
}
},
{
"name": "ContentType",
"in": "query",
"required": false,
"description": "Filter persons by ContentType",
"schema": {
"type": "string"
}
},
{
"name": "ContentLength",
"in": "query",
"required": false,
"description": "Filter persons by ContentLength",
"schema": {
"type": "integer"
}
},
{
"name": "GewijzigdOp",
"in": "query",
"required": false,
"description": "Filter persons by GewijzigdOp",
"schema": {
"type": "string",
"format": "date-time"
}
},
{
"name": "ApiGewijzigdOp",
"in": "query",
"required": false,
"description": "Filter persons by ApiGewijzigdOp",
"schema": {
"type": "string",
"format": "date-time"
}
},
{
"name": "Verwijderd",
"in": "query",
"required": false,
"description": "Filter for not deleted persons",
"schema": {
"type": "boolean"
}
},
{
"name": "offset",
"in": "query",
"required": false,
"description": "Offset for pagination",
"schema": {
"type": "integer",
"default": 0
}
}
],
"responses": {
"200": {
"description": "Successful response",
"content": {
"application/json": {
"schema": {
"type": "array",
"items": {
"type": "object",
"properties": {
"Id": {
"type": "string"
},
"Achternaam": {
"type": "string"
},
"Fractielabel": {
"type": "string"
}
}
}
}
}
}
}
}
}
}
}
}
The GPT can be found here:
https://chat.openai.com/g/g-lKwBGi0FE-dutch-politics
Successful query.
HTTP Endpoint URL: https://gegevensmagazijn.tweedekamer.nl/OData/v4/2.0/Fractie?$filter=NaamNL%20eq%20%27Partij%20voor%20de%20Dieren%27
Unsuccessful query
Functioning HTTP Endpoint URL: https://gegevensmagazijn.tweedekamer.nl/OData/v4/2.0/Persoon?$filter=Achternaam%20eq%20%27Wilders%27
How do I fix this error message I am getting where the "ResponseTooLargeError? Thx
I also run into the ‘ResponseTooLargeError’ problem when trying to retrieve a csv file from a custom API endpoint. It works fine for small csv:s (<100 rows & limited content) files but not large csv:s.
What is the max limit for a http response?
The problem is that it sees the CSV file not as a structured file, but as a set of text and tries to write the whole file in the code, because of this it gets an error.
From my observations the limit is 100k characters, if the response is larger, it throws the ResponseTooLargeError
Is this problem going to be fixed? First mentioning of this problem was in November last year. It appears that the problem still exists.
REST APIs are not designed with LLM/context sizes in mind. Often, API endpoints return too much data and cause problems like these, clogging the context window.
If you want an imminent solution, you need reduce the response size before it gets to the model (or in this case, GPT). If you can’t modify the API implementation then one way is deploy a proxy in front of the target API and modify the response there. Then, use the proxy in your GPT.
Other approach how to use APIs with large responses in GPTs is to use superface – crate a tool that trims the API response and then use the tool in your GPT.
Of course, this only works when you don’t need all the data in the GPT. If you do need all the data from the API accessible there, I’d consider RAG or sliding window but that will take a bigger effort.
DISCLAIMER: I work at superface
