Function Calling Not Triggering in ChatGPT API – Need Help

[Gracy]

Hi, I’m working on an AI project using the ChatGPT API. I’ve added a function calling tool, but the function is never invoked in the choices. Below is my code — please help me figure out what I’m doing wrong

try {
      const updatedMessages = [...messages, userMessage];
      const messagesToSend = trimMessages(updatedMessages);

      const res = await axios.post(
        'https://api.openai.com/v1/chat/completions',
        {
          model: 'gpt-4o',
          messages: messagesToSend,
          tools: [
            {
              function:
              {
                name: "store_itinerary",
                description: "Structured itinerary with days, descriptions, and images",
                parameters: {
                  type: "object",
                  properties: {
                    location: { type: "string" },
                    days: {
                      type: "array",
                      items: {
                        type: "object",
                        properties: {
                          day: { type: "string" },
                          title: { type: "string" },
                          description: { type: "string" },
                          image_query: { type: "string" } // you can use this with Bing
                        },
                        required: ["day", "title", "description", "image_query"]
                      }
                    }
                  },
                  required: ["location", "days"]
                }
              }
              ,
              type: "function"
            }
          ],
        },
        {
          headers: {
            Authorization: `Bearer ${apiKey}`,
            'Content-Type': 'application/json',
          },
        }
      );
      const choice = res.data.choices[0];
      console.log("choice", choice)
      if (choice.finish_reason === "function_call") {
        const funcName = choice.message.function_call.name;
        const args = JSON.parse(choice.message.function_call.arguments);

        if (funcName === "store_itinerary") {
          console.log("📦 Structured Itinerary JSON:", args);
        }
      }
      setErrorMessage('');
    } catch (err) {
      console.error('OpenAI Error:', err);
      setErrorMessage('Something went wrong.');
    } finally {
      setLoading(false);
    }

[_j]

Is the AI supposed to use that to store literariness or then get them? And when, does it need to conduct an interview to get all information. Does it need to have some reason to search bing for images before it can write that field? The purpose and where these go is unclear. Will the AI be able to see these later? Do they have real-world consequences?

The function should be self-contained and self-explanatory. The AI decides to use the function when it fits the needs of the user input.

---

Here’s the chat completions JSON with cleaned function, and the messages just seen. You can use messages as multishot to also inspire further tool use, appending your messagesToSend.

{
  "model": "gpt-4o",
  "messages": [
    {
      "role": "system",
      "content": [
        {
          "type": "text",
          "text": "You are a helpful planner."
        }
      ]
    },
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "What itinerary storage ability do you have? "
        }
      ]
    },
    {
      "role": "assistant",
      "content": [
        {
          "type": "text",
          "text": "I can store structured itineraries, which include details about the location, daily plans, descriptions, and image search queries. Each itinerary consists of multiple days, with each day having a specific day number or date, a summarized title, a detailed description, and an image search query suitable for finding relevant images. Let me know if you want me to store an itinerary for a specific location!"
        }
      ]
    },
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "Store a minimal schedule test for today 2025-06-18, where I feed my dog and wash my dishes. Discuss how you would do this with the tool, and then automatically invoke the tool."
        }
      ]
    },
    {
      "role": "assistant",
      "content": [
        {
          "type": "text",
          "text": "To store an itinerary for today with your activities of feeding your dog and washing your dishes, I would create an entry for \"today\" with the following details:\n\n1. **Day:** June 18, 2025\n2. **Title:** Daily Pet Care and House Chores\n3. **Description:** Start your day by feeding your dog a healthy breakfast. Afterwards, take some time to wash the dishes until they're sparkling clean.\n4. **Image Query:** \"feeding dog\" for feeding your pet and \"washing dishes\" for doing dishes.\n\nI will then use the `functions.store_itinerary` tool to save this detailed plan. Here's how I would do it:\n\n1. Prepare the itinerary with the specified location (in this case, your home or similar).\n2. Include the two activities planned for today as the day's description.\n3. Provide a relevant image search query for each activity.\n\nNow let's store it automatically:"
        }
      ],
      "tool_calls": [
        {
          "id": "call_D2aTou7lNzRmDXWyVtjPnnmw",
          "type": "function",
          "function": {
            "name": "store_itinerary",
            "arguments": "{\"location\":\"Home\",\"days\":[{\"day\":\"2025-06-18\",\"title\":\"Daily Pet Care and House Chores\",\"description\":\"Start your day by feeding your dog a healthy breakfast. Afterwards, take some time to wash the dishes until they're sparkling clean.\",\"image_query\":\"feeding dog and washing dishes\"}]}"
          }
        }
      ]
    },
    {
      "role": "tool",
      "content": [
        {
          "type": "text",
          "text": "{\n  \"status\": \"success\",\n  \"message\": \"Itinerary stored successfully.\",\n  \"data\": {\n    \"location\": \"Home\",\n    \"days\": [\n      {\n        \"day\": \"2025-06-18\",\n        \"title\": \"Daily Pet Care and House Chores\",\n        \"description\": \"Start your day by feeding your dog a healthy breakfast. Afterwards, take some time to wash the dishes until they're sparkling clean.\",\n        \"image_query\": \"feeding dog and washing dishes\"\n      }\n    ]\n  }\n}"
        }
      ],
      "tool_call_id": "call_D2aTou7lNzRmDXWyVtjPnnmw"
    },
    {
      "role": "assistant",
      "content": [
        {
          "type": "text",
          "text": "The itinerary for today, June 18, 2025, has been successfully stored with the activities of feeding your dog and washing your dishes. If you have any more plans or need further assistance, feel free to let me know!"
        }
      ]
    }
  ],
  "response_format": {
    "type": "text"
  },
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "store_itinerary",
        "description": "Store a structured itinerary containing location, daily plans, descriptions, and image search queries.",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "The location or destination of the itinerary."
            },
            "days": {
              "type": "array",
              "description": "List of daily itinerary items.",
              "items": {
                "type": "object",
                "properties": {
                  "day": {
                    "type": "string",
                    "description": "The day number or date of the itinerary item."
                  },
                  "title": {
                    "type": "string",
                    "description": "A short title summarizing the day's activities."
                  },
                  "description": {
                    "type": "string",
                    "description": "Detailed description of the day's activities."
                  },
                  "image_query": {
                    "type": "string",
                    "description": "A search query suitable for finding a relevant image using Bing or similar image search services."
                  }
                },
                "required": [
                  "day",
                  "title",
                  "description",
                  "image_query"
                ]
              }
            }
          },
          "required": [
            "location",
            "days"
          ]
        },
        "strict": false
      }
    }
  ],
  "temperature": 1,
  "max_completion_tokens": 2048,
  "top_p": 1
}

[Gracy]

I’m new to AI and still learning. I want to receive a JSON response only when an itinerary is being created — is that possible?

The reason I’m using Bing is because GPT usually returns broken or outdated image URLs

[_j]

A function is for offering a utility to the AI.

It has to be something real and concrete and understood, such as get_available_appointments("date_range...) or delete_memory("index": 1).

Then if that isn’t enough, you describe further, “this function interacts with the ongoing AI self-created memory state, to remove a memory item that the user no longer wants to be referenced. Write the index number seen in the ## your memory state header that matches the desired item for removal”.

If it is not your provided tool that will return information or a status to the AI, then, instead you might use structured outputs as the response format. That would be the case where you want the AI to simply produce its answer in two different ways. They are specified with a JSON schema similar to how functions works, but the AI is not expecting any action or any response - the output to a response format is the final answer.

In a structured output response format, you can have a schema that internally has “anyOf” - a choice of two sub-schemas. The AI might decide to use the “response_to_user” sub-schema, or may instead decide to invoke the different JSON created by a “calendar_display_response”.

The whole “when an itenerary is being created” doesn’t match what you provided so far - that schema for a function implies that there would need to be lots of purposeful data collected by the AI and relayed into a function at once (besides the AI not knowing what it actually does.)

Treat the AI like a dumb intern, explain to it the job it is doing and the form it will sometimes fill out, and why.

[Gracy]

Change of Plan
I’ve decided to generate the JSON data directly using the prompt instead of using function calling. However, I’m running into an issue: the image URLs I get from the responses are often broken or outdated.

To fix this, I’m thinking of integrating a web search API to fetch the latest and working images in real time.

My question is:

• Can I integrate an external web search API (like Bing or SerpAPI) directly into OpenAI’s tools so that it fetches updated images during the response itself?
• If that’s not possible, what’s the best way to resolve this broken image issue and ensure reliable images are shown in the response?

Looking for the most effective approach here.