- Allow each user to have an ongoing conversation. So when they come back to the app, day after day, the Chatbot remembers the conversation, who they are (based on logged in UID) what they’ve talked about etc.?
Either create your own memory solution, or use something like Zep, or use Assistants API (in beta, not very stable).
- Put some sort of limit on the amount of use per user, so I don’t get some users that overspend on the API so my costs don’t go through the roof?
Save usage linked to the user id and check before each request if the limit has not been reached.