Best way to auto improve assistant performance

Prompting
, ,
1

Hello, community,

I am not sure if this topic is fitting for prompting…

I have automation that creates social media posts on zapier, and what I build into some of the automation is feedback to assistants some I share back the final post that was published, and on weekly basis I share with assistant the post stats for it to see the performance; now in the assistant instructions I am asking it to use those to modify what posts are developed.

my question is two parts:
1- are the assistants actually learning when I share such info?
2- if not is their a way to automate the prompt instructions for the assistant through an API request? as I can fit a separate assistant to look at performance and generate updated instructions that would change the post writing assistant operation based on how stats are going.
ex: Post results > anayze and create new assistant instructions (another assistant or normal request) > edit Post Writing API Assistant Instructions

Basically is there an endpoint to change assistant instructions (not request or thread level instruction but the main ones for assistant as shows on openai dashboard)?

Did anyone else do something like this successfully?

2

Part 1: Learning from Shared Info

  1. Are the assistants actually learning when I share such info?
  • No, the current OpenAI models do not have the capability to learn or retain information between sessions. Each interaction with the assistant is stateless, meaning it does not remember previous conversations or learn from the data shared with it. Therefore, sharing post stats and final published posts will not improve the assistant’s performance over time.

Part 2: Automating Prompt Instructions

  1. Is there a way to automate the prompt instructions for the assistant through an API request?
  • Yes, you can automate and dynamically adjust prompt instructions using an API request. While you cannot change the main instructions for an assistant directly via the OpenAI dashboard, you can structure your automation to modify the prompt context before each API call. Here’s how you can achieve this:

Approach:

  1. Collect Post Performance Data:
  • Use your existing automation to gather post performance stats on a weekly basis.
  1. Analyze Performance Data:
  • Use a separate script or automation (potentially another assistant or a custom-built analysis tool) to analyze the performance data and generate new instructions.
  1. Update Assistant Instructions Dynamically:
  • Create a system where these newly generated instructions are appended to or replace parts of the prompt for the post-writing assistant. This can be done by including these instructions in the context of each API call.

Example Workflow:

  1. Post Performance Data Collection:
  • Use Zapier to collect performance data from social media platforms and store it in a database or a spreadsheet.
  1. Analysis and Instruction Generation:
  • Develop a script (in Python or another language) that analyzes this data and generates updated instructions. This script can be triggered automatically after the data collection step.
  1. Dynamic Prompting:
  • When making an API call to the assistant for creating a new post, include the updated instructions in the prompt. For example:
import openai

# Updated instructions generated from performance analysis
updated_instructions = "Based on recent post performance, focus on creating engaging headlines and using more visuals."

# Original prompt
original_prompt = "Create a social media post about our new product launch."

# Combine both for the API call
final_prompt = f"{updated_instructions}\n\n{original_prompt}"

response = openai.Completion.create(
  engine="davinci-codex",
  prompt=final_prompt,
  max_tokens=150
)

print(response.choices[0].text.strip())

Conclusion:
While OpenAI’s models do not learn from previous interactions, you can automate and dynamically update the context provided to the model for each request. This ensures that the assistant’s output evolves based on the latest performance data without needing persistent learning capabilities.

1 Like
3

Thanks a lot
I will try it out!

4

Henry, about your first point, I am pretty sure that when I use ChatGPT, it can remember things between sessions. The only thing I have to do is to tell it: Please remember this fact (or similar).
The weird thing is that it can remember only if I use the web version, not from the app.

5

I think assistants particularly are a bit behind in the sense they don’t have internet access and to my knowledge no memory