Dear community, I am new to the world of coding and am very excited about the capabilities offered by OpenAI. I have a question now.
I want to practice building a chat app using Flutter, with a local SQLite database, Firestore for the app backend, and the RevenueCat package for handling payments, etc.
After browsing the community’s records and asking ChatGPT about this structure, I learned that OpenAI’s API does not store chat history like https://chat.openai.com/ does. So, I need to store chat history locally and manage user payments and token usage through Firestore. Users will send messages to OpenAI, and the responses received from OpenAI will be stored in the local SQLite database. At the same time, I will update Firestore with the number of tokens used by the users. Given OpenAI’s excellent context linking capabilities, I may need to send a large amount of chat history to OpenAI each time to allow it to return valuable information based on historical records. (However, this would waste a lot of tokens; it would be great if OpenAI could store a separate “New Chat” for each end-user based on user_id, just like https://chat.openai.com/ does).
Is my architecture reasonable, or are there any optimizations or points to be aware of? Thank you very much!
Your architecture isn’t clear from your description. In chat completions, from my experience, chat history is used to maintain context in an ongoing conversation. Chat Completion Architechture
It sounds like, in your use case, your “chat history” is actually order history which, as you describe, is maintained in your SQL database. Chat history is, by definition, user questions and assistant responses, which you say you are storing in your SQL database as well.
In my mind, what you need is a plugin to have the AI search the SQL database for chat and/or order history rather than trying to cram a user’s history into the chat history.
I wouldn’t hold my breath on OpenAI storing API users chat history. You can mimic ChatGPT on your own through the API, with more freedom and bells and whistles. I’d rather be in control of the chat history (but use more tokens, oh well). There are lots of reasons for this.
Maybe look into embeddings to inject relevant topics into the conversation. But not sure of the purpose of your app. But overall seems reasonable to me (at a high level). The only thing is when you say “store local”. Not sure if this is an on-prem server or what this refers to. But make sure it has good uptime!