Perhaps one way could be to identify the type of prompts that cause errors due to too large messages from the token limits being surpassed, and then try to revise the prompt to fetch results that will be slightly lower than that limit, in addition to upgrading your models to the newer ones that can support 32k tokens. This worked for me where I had no control of the database, but realize it might not be applicable in your situation but thought to share just in case.
Related topics
| Topic | Replies | Views | Activity | |
|---|---|---|---|---|
| 16k Input vs Output: Edit and token strategies for long input texts | 2 | 1943 | December 17, 2023 | |
| How can I adjust the length of the prompt so that it does not exceed the max tokens? | 4 | 3508 | December 18, 2023 | |
| Optimizing Token Utilization for GPT-4 with Vector Database: Overcoming 1000-Token Limit Challenges | 2 | 379 | October 9, 2024 | |
| Help Needed: Tackling Context Length Limits in OpenAI Models | 8 | 15945 | February 8, 2024 | |
| Optimizing Input Token Usage in API Conversations | 2 | 1016 | March 10, 2024 |