Is possible OpenAI API caching the conversation?

[zavadzki72]

@_j

There is a way, but it cannot mitigate the cost. “Assistants” will maintain a chat session in a thread on the API - and resend it to the model every time.

With your own code, you are able to manage your budget more effectively, selecting your own tradeoff between memory quality and expense when you choose how many past chat turns the AI should know about.

About managing this on our side, we are already doing it here, we really don’t need to send the context again and ‘lose’ tokens with this, perhaps due to some request id or context id, you know?