Agents SDK streamed message_output_created events don't align with final_output of run result

Coding with ChatGPT
, , , ,
1

I don’t understand the relationship between streamed message output events and the final_output of the run result. I would expect that the final_output is composed of all the aggregated streamed message output events, but that is not the case. I believe this is a conceptual misunderstanding, but I can’t find any thorough, in-depth documentation on how streaming works in the agents SDK with multiple tools involved.

For example, I have this basic workflow:

  1. User asks agent a question
  2. Agent uses web_search_preview tool to search some information
  3. A message output is created that answers the question using the information from the web search
  4. Per my prompt instructions, the agent then calls another tool to save this output to my database
  5. Another message output is created, which is very similar to the one in step 3 but also includes the fact that the agent saved the info. This is what’s contained in the final_result of the run result object.

My question is, in a streaming context, I don’t see a way to identify the message output that’s generated in step 3 as not being the “real final output”.

For example, if I’m just passing along message output deltas to the user, they would first see the entire output of step 3, then the entire output of step 5. However, only the output of step 5 is the real “final output” that would be returned if I ran this in a sync context, and presumably the response I would want to save in a DB say for conversation persistence. However, inspecting both the raw & item stream events, I see no logical way to distinguish between the fact that the streamed message output events created in step 3 are actually NOT part of what will ultimately be the final output of the agent. What am I missing?

2

are u payloading and how are u payloading? what index are you using

This image shows a Python code snippet defining request headers and payload parameters for making an API call to Vertex AI, including comments and example values for video generation options. (Captioned by AI)
This image shows a Python code snippet defining request headers and payload parameters for making an API call to Vertex AI, including comments and example values for video generation options. (Captioned by AI)1503×575 39.1 KB

cuz payloading like this you said streaming, i depreciated my obs → python code, bu by doing this, you can achieve concurrent continuous payloads

especially since i think you want like long form outputs? id poll that bro bro

   print(f"⏳ Polling operation status from: {operation_poll_url}")
    while elapsed_wait_seconds < max_poll_wait_seconds:
        poll_response = requests.get(operation_poll_url, headers=headers)
        
        if not poll_response.ok: # Handle errors during polling
            print(f"❌ Polling failed with status: {poll_response.status_code}")
            try:
                print(f"   Polling Response JSON: {poll_response.json()}")
            except json.JSONDecodeError:
                print(f"   Polling Response Text: {poll_response.text}")
            poll_response.raise_for_status() # This will raise an HTTPError

        operation_status_data = poll_response.json()

but fyi i believe its more effective to vector the data first, you are already using agents so just aquire stream metadata and rebuild it using a ruleset.

3

I’m using the openai agents python SDK and streaming results, like result = Runner.run_streamed(agent...). Unsure why I can’t link to docs.

My question is specific to the behavior of streaming with the agents SDK. I’m wondering why sometimes intermediate tool calls within the orchestration path produce message output events which are not included in the agent’s final output. I’m not sure how I should interpret or understand these, and there’s not a clear logical way I see to distinguish message output events from intermediate tool calls from the “final” message output events whose deltas actually contribute to the agent’s final output.

4

DUDE I had that issue before

I Got you

have u upgraded ur pydantic ruleset? and schema enforcement so that they model cant just randomly disobey?

5

Yeah I don’t think it’s a pydantic issue as I’m up-to-date and not using structured outputs. I think it’s more a fundamental question about how the agent works and when/why it produces message outputs. Going back to the example:

  1. User asks a question
  2. Agent invokes my custom tool, and the tool returns the raw string “Success”.
  3. I receive “message output” events in the streamed response which contain what the agent has to say about this tool use. For example, something like “I called this tool and the invocation was successful”. So note this is NOT the tool output; this is the agent generating a message about based on the current context.
  4. The agent then calls another tool, per the system instructions.
  5. The agent then produces another message output, same as (3). This message output is the final_output of the Runner results.

So my question is, how should I understand the message output created in 3? Why is the agent producing message output when it’s still not done within this orchestration cycle? Why is that message output not included in the final_result? Since it’s not in the final_result, I might want to not show it to the user, but I see no way to identify the fact it’s not the “final_output” as I’m reading the stream in real time (ie I only know it’s not the “final_result” at the very end of the stream once (5) is complete

6

bro bro - pydanci allows u to set the rules… for orcehstration

so you just create a rule schema that enforces delivery

the orchestrator controls that, if u want it sequentially u just payload it and make sure ur pipeline logic is sound,

if u dont want to pay load it, then parse it from a jsonl or metadatastore