Or will an API let you use as many tokens as you want?
Not at this time.
Some (for chat apps) are using GPT-3 to “summarize” the back-text when it gets too long.
What are you trying to accomplish?
One use is to have GPT to ingest customer utterances from a chat bot to then ask it how to summarize then into specific intents. We have very large stores of utterances (200k+) … so obviously running into basic limitations to sample down those … would be nice to avoid the limits or at least increase them.
I would probably suggest fine-tuning a model. Use the chat history as the prompt and write out the summaries…
As I understand it, the limitation is mostly compute/hardware at this point. I imagine the limitations will be eased as technology improves. For now, though, fine-tuning might be able to help you…
Unlimited length summarization using AI