Function call output not available in Assistants API messages list

With the Chat API, the array of messages could include the user’s, the assistant’s, and any function call’s outputs. This was good because then you could save a list of messages, re-load it later, and pick up where you left off. The entire context, including from function call outputs, would be visible.

With the Assistants API, only user and assistant messages appear in the messages list, and I cannot figure out how to retrieve tool output to use as part of continuing a thread.

It doesn’t seem to be possible. Will it be possible? Is the output visible when I do a run even if I can’t see it in the thread history?

Am I missing something obvious? :frowning:

Have you looked at the Run Steps object? You should be able to see the function-related details there.

Let me know if this addresses what you’re looking to do.

Thanks @nikunj!

It does look like runs contain the information I need, but it would seem overly difficult to use that in the way I’m describing.

I’m keeping a history of users’ threads (in a database currently—would love to just store them with OpenAI and search them via metadata, e.g. by user_id).

When a user clicks on one, I want to load the thread and all its messages.

I can get all the messages for a thread very easily and load them. But they don’t include the tool content.

I can do this:

messages = client.beta.threads.messages.list("thread_abc123")
for message in message.data:
   # Show the messages here, including the tool content messages

But that doesn’t include tool content.

Messages do have run_id and thread_id however.

So I guess I could check the run for every assistant message and see what tool calls preceded it, then show those. Perhaps that’s what I’ll do, unless you have a better suggestion. Something like this:

messages = client.beta.threads.messages.list("thread_abc123")
for message in message.data:
   if message.role = "assistant":
      # Get the run using message.run_id and message.thread_id
      # See if the run had a tool call
      # If so, show the tool call here
   # Show the message content here

Is that the best way to do it?

Maybe you can try using assistant logger in your setup? Here is an example of an assistant function call to check stocks. When logger is set to DEBUG, the output is pretty verbose.



code below runs on colab

#@title Added Logger DEBUG
import openai
import time
import yfinance as yf
import logging
import json

# Step 1: Configure the logger
logging.basicConfig(filename='assistant.log', level=logging.DEBUG)
logger = logging.getLogger("Assistant")
logger.setLevel(logging.DEBUG)  # Set the logger's level to DEBUG

def get_stock_price(symbol: str) -> float:
    stock = yf.Ticker(symbol)
    price = stock.history(period="1d")['Close'].iloc[-1]
    return price

tools_list = [{
    "type": "function",
    "function": {
        "name": "get_stock_price",
        "description": "Retrieve the latest closing price of a stock using its ticker symbol",
        "parameters": {
            "type": "object",
            "properties": {
                "symbol": {
                    "type": "string",
                    "description": "The ticker symbol of the stock"
                }
            },
            "required": ["symbol"]
        }
    }
}]

# Initialize the client
client = openai.OpenAI()

# Step 2: Create an Assistant
assistant = client.beta.assistants.create(
    name="Data Analyst Assistant",
    instructions="You are a personal Data Analyst Assistant",
    tools=tools_list,
    model="gpt-4-1106-preview",
)

# Step 3: Create a Thread
thread = client.beta.threads.create()

# Step 4: Add a Message to a Thread
message = client.beta.threads.messages.create(
    thread_id=thread.id,
    role="user",
    content="Can you please provide me stock price of Apple, Tesla, and Microsoft?"
)

# Step 5: Run the Assistant
run = client.beta.threads.runs.create(
    thread_id=thread.id,
    assistant_id=assistant.id,
    instructions="Please address the user as Jorge."
)

logger.info(run.model_dump_json(indent=4))

while True:
    # Wait for 5 seconds
    time.sleep(5)

    # Retrieve the run status
    run_status = client.beta.threads.runs.retrieve(
        thread_id=thread.id,
        run_id=run.id
    )
    logger.info(run_status.model_dump_json(indent=4))

    # If run is completed, get messages
    if run_status.status == 'completed':
        messages = client.beta.threads.messages.list(
            thread_id=thread.id
        )

        # Loop through messages and log content based on role
        for msg in messages.data:
            role = msg.role
            content = msg.content[0].text.value
            log_message = f"{role.capitalize()}: {content}"
            logger.info(log_message)
            print(log_message)  # Print the message

        break
    elif run_status.status == 'requires_action':
        logger.info("Function Calling")
        required_actions = run_status.required_action.submit_tool_outputs.model_dump()
        logger.info(required_actions)
        tool_outputs = []

        for action in required_actions["tool_calls"]:
            func_name = action['function']['name']
            arguments = json.loads(action['function']['arguments'])

            if func_name == "get_stock_price":
                output = get_stock_price(symbol=arguments['symbol'])
                tool_outputs.append({
                    "tool_call_id": action['id'],
                    "output": output
                })
                log_output = f"Function Output: {output}"
                logger.info(log_output)
                print(log_output)  # Print the function output

            else:
                raise ValueError(f"Unknown function: {func_name}")

        logger.info("Submitting outputs back to the Assistant...")
        client.beta.threads.runs.submit_tool_outputs(
            thread_id=thread.id,
            run_id=run.id,
            tool_outputs=tool_outputs
        )
    else:
        logger.info("Waiting for the Assistant to process...")
        time.sleep(5)

Thanks. But that’s going to require me to store the logging information, no? All the info I need is already stored in either messages or runs. It’s just not easy to get. I think I can do it by retrieving messages for a thread, retrieving runs for each assistant message, and retrieving function call output for each of those messages when it exists.

Okay, here’s how to do it. Below users Streamlit for rendering.

Be warned: It’s more complicated than you might like.

import streamlit as st

from ai.client import client

# Start with a thread ID
thread_id = "thread_123"

# Get a list of messages for the thread, and order it from first to last
messages_list = client.beta.threads.messages.list(thread_id)
messages_list = messages_list.data[::-1]

# Loop through the messages
for msg in messages_list:
    # If the message is from the user, show it as a user message
    if msg.role == "user":
        with st.chat_message("user"):
            st.markdown(msg.content[0].text.value)
    # If the message is from the assistant...
    elif msg.role == "assistant":
        # Create the assistant chat message box
        with st.chat_message("assistant"):
            # Get the run steps associated with the message
            run_steps = client.beta.threads.runs.steps.list(
                thread_id=thread_id, run_id=msg.run_id
            )
            # Reverse the steps so they're in the right order
            run_steps = run_steps.data[::-1]
            # For each step...
            for step in run_steps:
                # If the step is a tool call...
                if step.step_details.type == "tool_calls":
                    # For each tool call...
                    for tool_call in step.step_details.tool_calls:
                        # If the tool call is a function and has output...
                        if (
                            tool_call.type == "function"
                            and tool_call.function.output
                        ):
                            # Show the output (here in a status box)
                            with st.status(
                                tool_call.function.name, state="complete"
                            ):
                                st.markdown(tool_call.function.output)
            # After all the function outputs, show the assistant's message
            st.markdown(msg.content[0].text.value)