How can I reduce API costs with repeated prompts?

[_j]

Great minds…ask the same questions?

Is it possible to preload context in open ai?

Semantic similarity search is terrible on tabular data like a CSV, and then you don’t even have a header to inform what the AI is looking at. Every API call is inherently stateless. As you’ve likely discovered, if you want to receive inference based on an input, even if that input is employing the same system message or context such as a document, you have to provide that input again with your alternate “question” about it. There are a few mechanisms you can exploit because of the repetitive…

To reduce costs: use the batch API. You can submit chat completions jobs to be run within a 24hr turnaround, and receive a 50% discount on all.