Hi,
I have a task that involves sending to API the same large amount of prompts but only with a few new sentences at last. I wonder if OpenAI saves previous chat in some way maybe as embedding, so I can send that same pre-prompts to it once, and resume every time with new questions. In that way, I wish to save a lot of token usage.
Use playground if you can, can go back and delete or keep any history in the chat
Yeah I am looking for basically the same thing but with API, because I do have many requests at the same time
In short, no.
That’s just not how things work.
The neural network is stateless even OpenAI with ChatGPT must re-send the chat history each message, there’s just no way around this.
Anything you want it to “know” while it’s generating the completion it has to process again on that run, there is no free lunch.