Every API call needs to process whole conversation again.
So yes, you need to pay for the whole thing again.
My guess is even if they added some way to store conversation, I dont see why you would not have to pay for it anyways. Since it needs to be processed within the models token limit. And is going to use the same resources.
Its just how it works.
If you dont process the “duplicates” again, how should the model know about it?
Otherwise with your every API call, it would basically have to create a completely new GPT model where your past conversation is inserted into the model.
The good thing is that you have full control over how the past conversation/memory is handled.
ChatGPT is just an app using the API
You can build an app storing conversation ids just like chatGPT
And you have to store it somewhere
When you use that conversation again, you again send the whole conversation with next prompt
ChatGPT most likely also sends the whole conversation with every prompt. (although it might be doing some summarization, hence losing control)
I think some of what you are asking is a bit similar to fine-tuning a model. But thats not available for 3.5 or 4
But I am not sure if it was possible to use it to quickly create fine tuned model within conversation and then switch to it every prompt.
I think its more about giving some instructions and role and then creating fine tuned model where you dont have to send that info again. But then you still have to send the whole conversation again with each prompt, just not the instructions or whatever you fine tuned.
(fine tuned is more costly than regular too)
I think many are using embeddings instead, as they mention its better than fine tuning