Advice to make parallel function calling work?

Hi,

I have several tools (functions) setup. It works perfectly if I don’t depend on parallel function calling (if i just need individual function calls) , but if I need two or more functions, it will systematically output the arguments to one of them. I checked and I am on the right gpt version. I used few shot prompting, I explicitely asked to output the tools at the same time, and it still doesnt work, except sometimes where I really explicitely tell it do to both. Normal function calling works fine tho. It’s really annoying!

Thanks, if anyone has advice or has figured out a way to enforce that it’d be great.

can you give sample functions and conversation how you would like to trigger parallel function? it would help us to find solution for your problem.

2 Likes

Thank you for your reply. Of course here are examples:

{
      type: "function",
      function: {
        name: "add_to_cart",
        description:
          "This function adds one or more products to the customer's cart. Each product is defined by its ID, quantity, optional modifiers, and optional special instructions. If the product has required modifiers, these must be included. Do not trigger this function if the customer did not explicitely told you the required modifiers he wants.", // TODO: Updated
        parameters: {
          type: "object",
          properties: {
            products: {
              type: "array",
              items: {
                type: "object",
                properties: {
                  productId: {
                    type: "string",
                    description:
                      "The unique identifier of the product to be added to the cart. This must correspond to a valid product in the store's inventory.",
                  },
                  quantity: {
                    type: "number",
                    description:
                      "The number of units of the product to be added to the cart. This must be a positive integer.",
                  },
                  modifiers: {
                    type: "array",
                    items: {
                      type: "object",
                      properties: {
                        id: {
                          type: "string",
                          description:
                            "The unique identifier of the modifier to be added to the product. This must correspond to a valid modifier for the specified product.",
                        },
                        quantity: {
                          type: "number",
                          description:
                            "The number of units of the modifier to be added. This must be a positive integer.",
                        },
                      },
                      required: ["id", "quantity"],
                    },
                    description:
                      "An optional array of modifiers to be added to the product. Each modifier is defined by its ID and quantity. If the product has required modifiers, these must imperatively be included.",
                  },
                  instructions: {
                    type: "string",
                    description:
                      "Optional special instructions for the product. This could include preparation or delivery instructions.",
                  },
                },
                required: ["productId", "quantity"],
              },
              description:
                "An array of products to be added to the cart. Each product is defined by its ID, quantity, optional or required modifiers, and optional special instructions.",
            },
          },
          required: ["products"],
        },
      },
    },
    {
      type: "function",
      function: {
        name: "remove_from_cart",
        description:
          "This function removes one or more products from the customer's cart. Each product is defined by its cartProduct ID.",
        parameters: {
          type: "object",
          properties: {
            cartProducts: {
              type: "array",
              items: {
                type: "object",
                properties: {
                  cartProductId: {
                    type: "string",
                    description:
                      "The unique identifier of the cartProduct to be removed from the cart. This must correspond to a valid cartProduct in the customer's cart.",
                  },
                },
                required: ["cartProductId"],
              },
              description:
                "An array of cartProducts to be removed from the cart. Each cartProduct is defined by its ID.",
            },
          },
          required: ["cartProducts"],
        },
      },
    },
{
      type: "function",
      function: {
        name: "send_order_summary",
        description:
          "This function sends the order summary to the customer (it shows the cart). This should be triggered at the same time as the add_to_cart, remove_from_cart, and modify_cart tools (functions), so that the user can see the changes operated to the cart. IMPORTANT: ALWAYS CALL THIS FUNCTION WITH THE add_to_cart, remove_from_cart, and update_cart functions. ",
        parameters: {
          type: "object",
          properties: {
            language: {
              type: "string",
              description:
                "The latest language in the user spoke. If you can't identify the language, or in case of doubt, you should default to english. Output only the name of the language.",
            },
          },
          required: ["language"],
        },
      },
    }

1m

So here are some of the functions I have. And for example if a user modifies his cart by adding a new item and removing another, it should know to trigger the add_to_cart, remove_from_cart AND send_order_summary. But it only ever does one of them.

I noticed you asked to call other functions in individual function descriptions. Maybe try moving it to the system prompt if that helps. Also try gpt4 if that helps.

I have also written a blog post on parallel function calling if that helps. Thanks

I tested your functions and modify them a bit then wrote a simple system prompt to tell the AI what to do and it works as you expect it, calling the order summary after add to cart or remove from cart.

Here are the modified functions:

{
    "name": "add_to_cart",
    "description": "Add products to cart.",
    "parameters": {
        "type": "object",
        "properties": {
            "products": {
                "description": "An array of products to be added to the cart. Each product is defined by its ID, quantity, optional or required modifiers, and optional special instructions.",
                "type": "array",
                "items": {
                    "type": "object",
                    "properties": {
                        "productId": {
                            "type": "string",
                            "description": "The unique identifier of the product to be added to the cart. This must correspond to a valid product in the store's inventory."
                        },
                        "quantity": {
                            "type": "number",
                            "description": "The number of units of the product to be added to the cart. This must be a positive integer."
                        }
                    },
                    "required": ["productId", "quantity"]
                }
            }
        },
        "required": ["products"]
    }
},

{
    "name": "remove_from_cart",
    "description": "Remove products from cart given their productId.",
    "parameters": {
      "type": "object",
      "properties": {
        "products": {
            "description": "An array of productId to be removed from the cart.",
            "type": "array",
            "items": {
                "type": "string"
            }
        }
      },
      "required": ["products"]
    }
},

{
    "name": "get_order_summary",
    "description": "Gets the order summary.",
    "parameters": {
      "type": "object",
      "properties": {
        "language": {
          "type": "string",
          "description": "The language used in given text in ISO 639 format."
         }
      },
      "required": ["language"]
    }
}

Here is a simple system prompt:

let system_prompt = `You are a helpful assistant.\n` +
        `# Available Tools\n` +
        `You have the following available tools that you can use depending on user query.\n` +
        `- add_to_cart, when the user wants to add products in the cart.\n` +
        `- remove_from_cart, when the user wants to remove products from the cart.\n` +
        `- get_order_summary, when the user wants to see the cart.\n` +
        `After calling either add_to_cart or remove_from_cart, call get_order_summary to show the user the current orders.`

Here are sample output:

  1. Add to Cart

Current shopping cart:

{
  shopping_cart: [
    { productId: 'nsp3012-05', quantity: 2 },
    { productId: 'wae5001-02', quantity: 1 }
  ]
}

user query:

{ query: 'please add three usb0001-01 in the cart.' }

output:

// loop 0
{
  id: 'call_AgCuHeAGBgzZy1Jx3U8Nr4dV',
  type: 'function',
  function: {
    name: 'add_to_cart',
    arguments: '{"products":[{"productId":"usb0001-01","quantity":3}]}'
  }
}
// mock tool output
[
  {
    tool_call_id: 'call_AgCuHeAGBgzZy1Jx3U8Nr4dV',
    role: 'tool',
    name: 'add_to_cart',
    content: '{\n  "status": "success",\n  "message": "1 items added to cart."\n}'
  }
]
// 2nd Chat Completions API call
{
  role: 'assistant',
  content: null,
  tool_calls: [
    {
      id: 'call_AgCuHeAGBgzZy1Jx3U8Nr4dV',
      type: 'function',
      function: [Object]
    }
  ]
}
//loop 1
{
  id: 'call_AgCuHeAGBgzZy1Jx3U8Nr4dV',
  type: 'function',
  function: { name: 'get_order_summary', arguments: '{"language":"en"}' }
}
// mock tool output
[
  {
    tool_call_id: 'call_AgCuHeAGBgzZy1Jx3U8Nr4dV',
    role: 'tool',
    name: 'get_order_summary',
    content: '{\n' +
      '  "status": "success",\n' +
      '  "message": "This is the current order summary:\\nproductId, quantity\\nnsp3012-05, 2\\nwae5001-02, 1\\nusb0001-01, 3"\n' +
      '}'
  }
]
// 3rd Chat Completions API call
{
  role: 'assistant',
  content: 'Three USB0001-01 have been added to the cart. Here is the current order summary:\n' +
    '- NSP3012-05, 2\n' +
    '- WAE5001-02, 1\n' +
    '- USB0001-01, 3'
}
  1. Remove from cart

Current shopping cart:

{
  shopping_cart: [
    { productId: 'nsp3012-05', quantity: 2 },
    { productId: 'wae5001-02', quantity: 1 }
  ]
}

user query:

{ query: 'please remove wae5001-02 from the cart.' }

Output:

// loop 0
{
  id: 'call_8NuUInOzLUVK2yo1Yzj5whN0',
  type: 'function',
  function: {
    name: 'remove_from_cart',
    arguments: '{"products":["wae5001-02"]}'
  }
}
// mock tool output
[
  {
    tool_call_id: 'call_8NuUInOzLUVK2yo1Yzj5whN0',
    role: 'tool',
    name: 'remove_from_cart',
    content: '{\n  "status": "success",\n  "message": "1 items removed from cart."\n}'
  }
]
// 2nd chat completions api call
{
  role: 'assistant',
  content: null,
  tool_calls: [
    {
      id: 'call_VdGFh9yOhm5GwqR9BDEn13Cn',
      type: 'function',
      function: [Object]
    }
  ]
}
// loop 1
{
  id: 'call_VdGFh9yOhm5GwqR9BDEn13Cn',
  type: 'function',
  function: { name: 'get_order_summary', arguments: '{"language":"en"}' }
}
// mock tool output
[
  {
    tool_call_id: 'call_VdGFh9yOhm5GwqR9BDEn13Cn',
    role: 'tool',
    name: 'get_order_summary',
    content: '{\n' +
      '  "status": "success",\n' +
      '  "message": "This is the current order summary:\\nproductId, quantity\\nnsp3012-05, 2"\n' +
      '}'
  }
]
// 3rd chat completions api call
{
  role: 'assistant',
  content: 'The product "wae5001-02" has been successfully removed from the cart. The current order summary is as follows:\n' +
    '- Product ID: nsp3012-05, Quantity: 2'
}

I think the system prompt is the important part that made it possible to execute the operation as you expected it.

2 Likes

Thank you very much. My system prompt was along the line of yours, including even few shot examples etc… so surely it was not a problem.

The problem was that I did not know I had to loop and call the model AGAIN, I thought it would output both functions directly. Also I didn’t know I had to pass the previous tool calls as an argument … Where did you see that ? The documentation doesnt mention this anywhere, except in some obscure cookbook examples… Thank you again