My goal is to replicate the workflow of a custom GPT through the API, by keeping the conversation history as lengthy as the models context window allows me too. It would be good to be able to enable the internet retrieval feature, but I do not need to use functions features, only give a starting context and use the chat completion.
When creating an assistant through OpenAI assistants API, we get an assistant id that we can store in order to access the assistant again later. If I am not mistaking, the assistant stores the convo history so that when a user calls it again, he can take the conversation from where he left it. However, I do not need any of the assistant’s features in my usecase, I only need chat completion.
I am wondering if by using Langchain, there is a way to store a kind of convo id and store it to be able to take the convo from where I left it.
If the answer is no, I imagine I could create a uuid per convo and store every messages in db and somewhat be able to take back from where I left it by reinjecting the whole convo history but this looks heavy to me, both in db storage and in costs when I will reinject the convo and pay for all the tokens again.
So here are my 2 questions:
- What technique would you recommend to store convos for raw text completion at efficient cost ?
- Is it possible using Langchain or do I need to use OpenAI API ?