Feeback Plugins - Workaround Solution for Tokens Limit

GPT builders Plugins / Actions builders
, , ,
1

Hello,

I’ve been working on a plugin to sort through a public government API. However, I’ve run into the issue of hitting the token limit because this API has numerous paths, all of which are interesting to users. Right now, the only solution seems to be limiting the available paths in the plugin. But if I wanted to offer access to all paths, I would have to create multiple plugins (like plugin1, plugin2), which I believe would result in a poor user experience.

I wanted to share this problem here because I believe others might also encounter this issue, and it would be great to find a workaround solution. One idea I have is to create a multi-plugin system where we separate domains into mini-plugins and have one central plugin serve as the orchestrator for calls and other operations.

Thanks!!

2 Likes
2

Maybe you can share some more details? At which point is it hitting the token limit? When you register the plugin or when ChatGPT calls your API? It hits the limit because the request to the API is too big or the response is too big?

1 Like
3

On api call, whe user makes a request, like the api spec + ai-plugin spec + user request goes over the limit.

4

Interesting… As far as I know, when ChatGPT sends a request to the plugin, it would not send the API spec or the plugin spec to the plugin. ChatGPT only reads these spec once when the plugin is installed. Are you sure they are causing the token limit? Or is it the response from the plugin that’s too big?