I have a chat bot that is specific to the user. The user information changes over time. Currently, I pack all the relevant user information into each prompt, which is quite tedious and makes the prompts fairly long.
Is there a better, more efficient way, to do this? I basically want a deeply personalised chatbot for each user.
The more efficient way would be to offload the work of telling the AI what it should know about the user to the user themselves. Give them a user interface box.
You could also have a memory-like function the AI can emit, where it can silently capture the user’s own private revelations and place those into a prompt injection for the future.
I think you’ll find that unless the personalization is specifically written as instructions to alter the responses of the AI, this will matter little in shaping AI behavior.
You might have one, but I can’t think of any reason why an administrator would be making personal chatbots, unless it was for their own mom or a business application.
There is actually a ton of highly personalised use cases (medial, nutrition, education, etc.). Here is a concrete example: Post-surgery recovery progress. User logs updates about recovery multiple times a day, additional data from healthcare providers (i.e. physio, doctor,…) is added frequently. The user asks the chatbot about which exercises she can/can’t do today. Response needs to be based on the user’s latest status.
The memory function approach doesn’t really work I guess since the user related data changes frequently.
I think the use case you are describing is quite interesting and relevant. Just taking this as an example, I think you‘d need a hybrid approach. There may be baseline/static information about the user that is always required and which you therefore always need to inject in the prompt. Besides this you‘d need to dynamically retrieve information based on a user‘s specific request/question. For the latter a RAG pipeline would likely be the most efficient approach. The dynamically retrieved data would then be placed alongside the static data in the prompt to respond to the user.