What are best practices for handling / inserting actions taken into context and memory?
As a loose example, if you have a function that checks the weather and then tells the user what the weather is, I want the bot to know that it checked the weather for the user at 10pm so that when the user asks again I can pull in those actions that it took into the context window.
Would I label these actions as assistant or system? Its not explicitly something that it told the user and I don’t want it to tell the user about how it checked the weather I just want it to know that it checked the weather as part of handling the conversation.
There is a very specific way to provide the past turns of what the assistant wrote to emit a function call, and then the return value that is placed, to then have the AI answer using the function information it obtained.
It is then up to you how long you want to have function calls remain part of the chat history, if you’d want to expire them faster than the rest of the surrounding chat. You can have the type of function inform the expiration you program. Most assistant/function exchanges are somewhat short compared to full AI responses to a user.
So the function call is inserted with the assistant role and then the result of the call is inserted afterwards with the function role makes sense.
Would this extend then to just telling the LLM what actions were taken instead of a strict function call? Like passing the LLM a summary of what actions it just took under the role assistant? Or would that be better served as the function role?