Hello everyone, I’m attempting to retrieve a response from the chat completion API in JSON format using the GPT-3.5 Turbo model. However, I’m encountering an issue where if the JSON response is too lengthy, it gets truncated due to a maximum token length constraint.
I’m considering a couple of unconventional approaches to address this. Firstly, could I potentially request the AI to minify the JSON in some way, allowing me to expand it later? Alternatively, is it feasible to ask the AI to generate the JSON and provide it in an encrypted hash format, which I could later decrypt to access the information? Is either of these options possible? If getting an hashed output possible than how can i decode it I tried but not able to decrypt as I don’t know what hashing algo it used.
First, capture the value of “finish_reason” from the API return object.
If it is “stop”, then the AI decided to end the output itself.
If it is “length” then the max_tokens limit you specified was hit (or if you had not specified max_tokens at all, the maximum of 4k tokens or the remaining space was hit after providing a large input).
The AI doesn’t write the API response JSON. Only the message content.
The most efficient “compression” for the AI to write in terms of tokens is natural English language, direct to you as a response. If you were to have the AI send the language to a code interpreter session for it to run an encrypting or zip compression python script, the AI still had to send the same language, plus the additional overhead of the code it writes.
Your connection between OpenAI and your code is encrypted by HTTPS.