Adding incremental data to an Assistant

Hey everyone,

I’d like to build an assistant based on emails from specific people.
If you want to add incremental emails to the corpus of data with which to query, would you have to re-upload all the data or can you add smaller increments?

1 Like

Welcome to the forum. I’ve not messed with Assistants a lot myself yet, but here’s what GPT-4 has to say… You’ll basically need to add the new emails into the thread as a message likely? You might also look into RAG to just grab appropriate emails and insert them…

Incremental Updates to Data Corpus

  1. Adding Incremental Emails:
    • The Assistants API is designed for real-time interactions and does not inherently accumulate data like an email corpus.
    • For adding emails incrementally, manage this dataset externally and feed relevant data into the API during interactions.
    • The API does not store or build upon past interactions as part of a dataset.

Building the Assistant

Assistant Creation

  • Create an Assistant with specific instructions and choose a model. Tools like Code Interpreter and Retrieval can be enabled.

Starting a Conversation

  • A ‘Thread’ represents a conversation. Create a new Thread when a user initiates a conversation.

Adding Messages

  • Messages, possibly including new email data, are added to the Thread.
  • The API manages the model’s context window, so you don’t need to manage it.

Running the Assistant

  • When you run the Assistant on a Thread, it generates responses based on the Messages.
  • The Assistant decides which previous Messages to include in the context.

Flexibility in Instructions

  • You can change instructions with each Run, allowing for dynamic responses based on the current context or updated data.

Considerations for Email-Based Assistant

  • Email Data Management: You’ll need a system to manage and update your email corpus, feeding relevant emails or email-derived data into the Assistant API during each interaction.
  • Privacy and Security: Ensure you handle email data securely, respecting privacy and confidentiality.

Testing and Iterating

  • Experiment with Scenarios: Test various scenarios in the Assistants playground to understand the Assistant’s handling of incremental data updates.
  • Feedback and Refinement: Use the feedback from tests to refine your Assistant’s instructions and the structuring of Threads and Messages.
1 Like