Past responses are not retained in memory

API
1 
messages = []

while True:  #input != "quit()"
    
    
    usr_content = input("User: ")
    messages.append({"role": "user","content": usr_content})
    response = client.chat.completions.create(

        model="gpt-3.5-turbo",
        
        messages=[{"role": "system", "content": 
             
             """
I need you to analyse the content to output a maximum of five variables: board_name, item_name, update_column, update_value, follow_up_action. 

  Output in the form: 
  
                      board_name =  'value' 
                      item_name =  'Firstname Surname'
                      update_column = 'update_column'
                      update_value = 'update_value'
                      follow_up_action = 'update_item' or 'create_item' method of MondayClient class.


For example: 

'Add John Smith to the customer CRM'. Here board_name='Customer CRM', item_name='John Smith', email=None, update_column=None and follow_up_action = 'create_item'  

'Update Abdul Mohammed's email to abdul@facebook.com on the Customer CRM board. Here board_name='Customer CRM', item_name='Abdul Mohammed', email=abdul@facebook.com, update_column=email and follow_up_action = 'update_item'                   

After printing each output variable, output a natural language response confirming you have completed action.

Bear in mind, it is also possible the user asks follow up questions, in which case you will not need to output the variables above. In such cases just answer using natural language accordingly.
             """},
                  
            
             {"role": "user","content": usr_content}])
    
    
    reply = response.choices[0].message.content
    messages.append({"role": "assistant", "content": reply})
    parts = reply.split('\n')

The above is the first half of a Python script I have written. The issue I am having with the code above is that even though I am appending each message to messages the memory is still not retained. The below is what I get:

User: Add James Blunt to the Customer CRM
System: Action completed. Item ‘James Blunt’ has been added to the ‘Customer CRM’ board.
User: What did you just add?
System:I apologize, but I couldn’t find any information on what I just added. Could you please provide more details or clarify your question?
User: Who did you just add to the Customer CRM?
System: I have recently added John Smith to the Customer CRM.

Even though I asked a follow up question right after the system’s response, it still did not understand that “James Blunt” was the contact added. Instead, it failed the first attempt then failed again by resorting to the example in the instructions.

How can I fix this so memory is retained?

2

You need to send the entire context every time.

The models have no innate memory.

3

ChatCompletions do not maintain state, meaning they don’t remember the context or history of the conversation. If you require a stateful conversation, where the context is preserved, you should use the Assistants API instead or use tools like langchain for stateful.

4

These suggestions are fine, though perhaps a bit overkill for someone just starting out.

One simply needs to maintain state themselves by concatenating additional messages in the messages array.

While the tools you are recommending will handle all that for you, it’s important to learn how to manage context oneself. This is critical for being able to control costs and having any idea whatsoever why the models do what they do.

1 Like
5

Hi @muddzy ,

The problem in this code is that the API call isn’t using the variable named messages, where you you’re storing the reply.

This should help you get started:

2 Likes