Semantic similarity search is terrible on tabular data like a CSV, and then you don’t even have a header to inform what the AI is looking at.
Every API call is inherently stateless.
As you’ve likely discovered, if you want to receive inference based on an input, even if that input is employing the same system message or context such as a document, you have to provide that input again with your alternate “question” about it.
There are a few mechanisms you can exploit because of the repetitive nature of the task.
1. Cache
OpenAI offers a discount on that part of input that has been recently used before. This means that if you have an unchanging document and system task, repeated queries against that in a short time can have the input discounted by 50%, or on just the gpt-4.1 series models, by 75%.
Since this relies on a first “hit” that is not discounted, and the cache also has minimum matching requirements (where the start of at least 1k tokens is not altered), and then that the server-side cache is not 100% guaranteed and expires, you can also simply submit the job to “batches” or “service level: flex” and also receive a 50% discount on the whole job, input and output, guaranteed.
2. Stateful storage with Responses
The responses endpoint allows you to store and reuse a previous response ID, and this can be used multiple times. It doesn’t actually change how the AI model works, but can be a mechanism you use to form a “chat”, and then not need to send that over the network again. It doesn’t have any cost benefit, and you’d also have to offer some initial input, like “Here is my CSV, just acknowledge you are ready to answer questions about it by answering ‘OK’”
Without full context…
It sounds like you could want full observation of the CSV file, and not some search done on it. Providing the full CSV to the AI can do that, where it can answer any question across all the knowledge at once. However, you might consider loading the actual knowledge into a database directly, with the fields intact. Then you can present a tool to the AI, where it can actually make queries, such as “customer: startswith(“smith”)” + date(after: 30 days ago).