Force communication through tool

Summary: I want completions to communicate through a tool call, in order for it to respond and call tools at the same time

I am a beginner familiarizing myself with agentic applications.

I built a basic (gratitude journaling) chatbot with the completions API (I only learned about the assistant API later).

I have the following challenge: Completions will either respond or call tools. But cannot respond and call tools at the same time.

My attempt at solving it: Make “responding” a tool and add it to the tool list. Idea: My main API call is only the “brain” and no longer the "mouth.

Weirdly, this worked partly at some point but now I cant replicate the success.

Are there better ways to achieve this?

This is the primary call, the “brain”:

def primary_agent(chat_history=None):
    tools = [
            llm_tools.send_message_to_user,
            llm_tools.tag_gratitude_sentiment,
            llm_tools.update_stats,
            llm_tools.send_reflection,
            llm_tools.share_journal_entry,
            llm_tools.send_random_prompt
            ]

    specific_prompt = f'''
                To respond, use nothing but the tool "respond_to_user". Do not communicate any other way. 
                Use the tool tag_gratitude_sentiment to tag the user's gratitude statement for categorization.
                Use the tool update_stats to update the user's gratitude journaling stats.
                Use the tool send_reflection to give the user a brief, concise retrospective on a certain period of gratitude practice.
                Use the tool share_journal_entry to offer the user to share their gratitude statement with the person its about. This tool is to be used if the user expresses gratitude towards a specific person, a friend or family member.
                Use the tool send_random_prompt to generate a creative prompt for gratitude journaling.

    # Initialize the system message that sets the behavior of the assistant
    messages = [
        {
            "role": "system", 
            "content": general_system_prompt + specific_prompt
        }
    ]
    
    # Loop through the chat history and add each message to the messages list
    if chat_history:
        for entry in chat_history:
            # Append each message with the correct role ('user' or 'assistant') and content
            messages.append({
                "role": entry['Role'],  # 'user' or 'assistant'
                "content": entry['Content']
            })    

    # Now call the OpenAI API with the dynamically created messages list
    completion = client.chat.completions.create(
        model="gpt-4o-mini", 
        messages=messages,
        tools=tools
    )

    response = completion.choices[0].message

    return response
            '''

Tools:



send_random_prompt = {
    "type": "function",
    "function": {
        "name": "send_random_prompt",
        "description": "Gives a prompt as inspiration for gratitude journaling"
    }
}

send_reflection = {
    "type": "function",
    "function": {
        "name": "send_reflection",
        "description": "Looks back at the users history of gratitude journal entries and reflects on it",
        "parameters": {
                "type": "object",
                "properties": {
                    "start_date": {
                        "type": "string",
                        "description": "The start date of the period to look back on. Formatted"
                    },
                    "end_date": {
                        "type": "string",
                        "description": "The end date of the period to look back on"
                    }
                },
                "required": ["start_date"]
            }
    }
}

share_journal_entry = {
    "type": "function",
    "function": {
        "name": "share_journal_entry",
        "description": "Offers the user to share their gratitude statement with the person its about. This tool is to be used if the user expresses gratitude towards a specific person, a friend or family member.",
        "parameters": {
                "type": "object",
                "properties": {
                    "message": {
                        "type": "string",
                        "description": '''
                                The text message that will be shared with the person. It starts with an introduction giving the person context about the gratitude statement. 
                                
                                Example #1
                                Original gratitude message: "I am grateful for my mum having supported me all my life. I understand that I can be very lucky to have had this in my life."
                                message: "Hey, I just wanted to share this with you. I wrote this in my gratitude journal today and thought you should know: 'I am grateful for my mum having supported me all my life. I understand that I can be very lucky to have had this in my life.'"
                            '''
                    }
                },
                "required": ["message"]
            }
    }
}

send_message_to_user = {
    "type": "function",
    "function": {
        "name": "send_message_to_user",
        "description": "Sends a message to the user. Use this tool to respond and communicate with the user.",
        "parameters": {
                "type": "object",
                "properties": {
                    "message": {
                        "type": "string",
                        "description": "The message to send to the user"
                    }
                },
                "required": ["message"]
            }
    }
}

tag_gratitude_sentiment = {
    "type": "function",
    "function": {
        "name": "tag_gratitude_sentiment",
        "description": "Tag the user's gratitude statement with a sentiment or category.",
        "parameters": {
                "type": "object",
                "properties": {
                    "tag": {
                        "type": "string",
                        "description": "The sentiment or category to tag the gratitude statement with"
                    }
                },
                "required": ["tag"]
            }
    }
}

update_stats = {
    "type": "function",
    "function": {
        "name": "update_stats",
        "description": "If the user's message is a gratitude statement, update the user's journaling stats."
    }
}

it can actually return both content and tool_calls at the same time but it is unpredictable. if you want to force the api to include some text message together when a tool is invoked, you can add such property (e.g. preprocessingMessage) in your tool definition. so you can now send it to display in the UI before/while you execute the tool action.

Thanks for this nudge. I’ll play around with that and will be back if I have questions

I also discovered the tool_choice="required" parameter to make the “brain” above work – but results are still a bit weird (responses are not as good as when sent directly and sometimes there is not response when there should be one).

Sure it can!


:memo: Here’s a chat session

– your message? –
>>you’ve got a weather tool, right? how do you use it?

Yes, I have a weather tool! Here’s how I use it explained step-by-step:

  1. Purpose: It’s for retrieving current weather information in the
    continental USA or southern Canada.
  2. Parameters:
  • Location: You provide the city, place, or region where you want the
    weather data.
  • Unit of temperature: Either “fahrenheit” (common in USA) or
    “celsius” (common elsewhere).
  1. Process: If you ask me for weather information:
  • I’ll let you know that I need to use the weather tool.
  • I’ll call the tool with the necessary parameters (e.g., location and
    temperature unit).
  • Once the response comes, I’ll share the weather details with you.

Would you like to check the weather for a location? If so, tell me the city
or area and whether you prefer the temperature in Fahrenheit or Celsius!
– your message? –
>>Sure - I want weather in Detroit
Would you like the weather in Detroit displayed in Fahrenheit or Celsius?
– your message? –
>>oh, F plx
Got it! I will now use the tool to get the current weather for Detroit in
Fahrenheit. Let me proceed.
*** Tool call detected ***
get_current_weather with arguments:
{“location”:“Detroit”,“unit”:“fahrenheit”}
Please provide the result for get_current_weather: Detroit, MI conditions: 36F, rainy (2:28am upd)
The current weather in Detroit is 36°F with rainy conditions. Let me
know if you need anything else!
– your message? –



:memo: How does it work?

You see the AI saying “Got It!..” That’s the assistant reply to me, and then it invokes the tool automatically.

Then I see the arguments the AI emitted, and fill in the value to return when a tool function is called, because this is just demonstration code.

This is streaming, both displaying “content” as received, and collecting the chunks of “tool_call” for processing.

The tool description is what does it, and is what the AI described in the chat:

"type": "function",
"function": {
    "name": "get_current_weather",
    "description": ("Returns the current weather in continental US or S Canada. "
        "\nUsage:\n1.AI must always first announce that it has decided to employ this function, and only then\n"
        "2. send to the function in the same response as the description of the plan to call."),
    ...

The model used was gpt-4o-2024-11-20.

1 Like

Just tested this. it works without additional property. Nice!