hi ,
Looking for option to keep the data (prompt data / content) which we pass along with user questions. currently, expecting to pass the data at every request along with user questions. also the usage cost is getting increased.
any other alternate way to keep / store our custom data in api or other place s and can be used for further requests ?
you can use assistants api. the thread will last until you decide to delete it.
see the data rentention docs. check the note at the bottom.
** Objects related to the Assistants API are deleted from our servers 30 days after you delete them via the API or the dashboard. Objects that are not deleted via the API or dashboard are retained indefinitely.
Hi @muruga.b , use assistant api for your use case. Opt for GPT-4o-mini, it will be quite inexpensive. https://platform.openai.com/docs/assistants/overview
You could also use GPT-4o for completions but use mini to summarize your conversation history just before your next call. That would reduce the cost of long running conversations at the expense of slightly slower query times. For long conversations that could drop your cost from $0.20 per query down to around $0.08.
We do something similar and we see a consistent 60% - 80% cost reduction with no perceived loss of answer quality.
Hi @bhagyesh , @supershaneski Thank you very much for your responses.
I use OpenAI services / endpoints via Azure AI services. AI assistant is available only on specific locations (via azure portal). Also the custom data i was taking about is not the static one. it will be different for each user / login and the data keeps evolving. i don’t think storing it as a file in assistant and make use of it is a correct approach.
currently , passing my data to openai endpoint along with the Prompt messages and getting response from openAI model for our questions. but the answers / responses are not always correct and same for the same questions. looking for the solution or alternate option to handle such in-accuracy in openAI responses.
if you are already attaching the data in your context and still not always getting correct response, there might be issue in the format of your data, the way the info are written and organized.
Hi @muruga.b, I pondered upon what you’re saying. It is difficult to understand your workflow now. If you can share some flowchart/schematic of what you’re trying to accomplish, that would help me come up with suitable suggestion.