Structured outputs with Function calling

What i want to do is use Structured outputs to force the model to output on a certain way which is by dask and date info by using Pydantic. Which works correctly if i only do this.

But now I want to add funtions, to be called by the model. For example i have the function get_date_from_today(X) that returns the date X days from today. So it will be useful for the llm to use as a tool to know which day is it. But i just want it to use it like in the thinking process, and later, regarding of if it did used the funciton or no, usea Structured Outputs for desired output format.

So looks like these 2 things cannot go together. I was able to do the Structure Output if i didn’t add the tools. And if i added the toold with self.client.chat.completions.create, it works. But i cannot use Structure Output and i have to use json. Why?

class DateInfo(BaseModel):
    start_date: str
    end_date: str
    time: Optional[str] = None

class TaskExtraction(BaseModel):
    task: str
    date_info: DateInfo

class CalendarTaskExtractor:
    def __init__(self, api_key: str):
        self.client = OpenAI(api_key=api_key)

    def get_day_of_week(self, date_str: str) -> str:
        """Given a date in YYYY-MM-DD format, returns the day of the week."""
        date_obj = datetime.strptime(date_str, "%Y-%m-%d")
        return date_obj.strftime("%A")

    def get_date_from_today(self, days: int) -> str:
        """Returns a date that is today plus a specified number of days."""
        new_date = datetime.now() + timedelta(days=days)
        return new_date.strftime("%Y-%m-%d")

    def extract_task_info(self, text: str) -> TaskExtraction:
        """
        Extract task and temporal information from text using GPT-4 with function calling.
        """
        tools = [
            {
                "type": "function",
                "function": {
                    "name": "get_day_of_week",
                    "description": "Get the day of the week for a given date",
                    "parameters": {
                        "type": "object",
                        "properties": {
                            "date_str": {
                                "type": "string",
                                "description": "Date in YYYY-MM-DD format"
                            }
                        },
                        "required": ["date_str"],
                    }
                }
            },
            {
                "type": "function",
                "function": {
                    "name": "get_date_from_today",
                    "description": "Get a date that is a specified number of days from today",
                    "parameters": {
                        "type": "object",
                        "properties": {
                            "days": {
                                "type": "integer",
                                "description": "Number of days from today (positive or negative)"
                            }
                        },
                        "required": ["days"],
                    }
                }
            }
        ]

        today_date = datetime.today().strftime('%Y-%m-%d')
        day_of_week = datetime.today().strftime('%A')

        messages = [
            {
                "role": "system", 
                "content": f"""

                Extract the task and temporal information from the text. 
                For dates, use YYYY-MM-DD format. For times, use HH:MM format.
                Today is {today_date}, which is a {day_of_week}.
                All the dates should be same or later than {today_date}, because it doesn't make sense to have a past date.
                - `task`: A brief description of the activity or event.
                - `date_info.start_date`: The start date in `YYYY-MM-DD` format.
                - `date_info.end_date`: The end date in `YYYY-MM-DD` format.
                - `date_info.time`: The time of day in `HH:MM` format if applicable.

                Use the available functions to determine and confirm dates if needed:
                - get_day_of_week(date_str) - Gets the day of week for a date
                - get_date_from_today(X) - Gets a date X days from today
                """
            },
            {"role": "user", "content": text}
        ]

        try:
            while True:
                completion = self.client.beta.chat.completions.parse(
                    model="gpt-4o-mini",
                    messages=messages,
                    tools=tools,
                    tool_choice="auto",
                    response_format=TaskExtraction
                )

                message = completion.choices[0].message


.... etc .... (more code)
1 Like

This is not something I have experimented with. Still, it seems that this should be possible – see Function Calling with Structured Output in API documentation here: https://platform.openai.com/docs/guides/function-calling/function-calling-with-structured-outputs

I’ve seen the docs and what it says here is Structured Outputs for function calling. Not Structured Outputs and function calling. Like it uses Structured Outputs to match the function output. But what i want is just be able to use normal function calling or Structured Outputs for function calling, with stucture output at the response_format. And looks like this ain’t possible. It’s like in “client.beta.chat.completions.parse” you cannot use tools

It looks like you are correct then. From what I read, "Function calling is supported in the Chat Completions API, Assistants API, and the Batch API. " That does not include “client.beta.chat.completions.parse’” unfortunately.