Parallel Assistant Function calls returning undocumented function "multi_tool_use.parallel"

API Bugs
1

My assistant threads are returning multi_tool_use.parallel as a function name when multiple tools of the same name run in parallel. I have pasted a full example of a run of a thread_xxx below. This is happening for a decent proportion of my runs, so this is a blocker for using the Assistant API for my application.

This seems to be undocumented. However, as I have tried to “code around it” by parsing the response and running my functions, i’m now stuck on how to correctly submit the tool output of these parallel functions. As you an see from the example JSON below, there’s no way to distinguish what response is for which corresponding part of the multi_tool_use.parallel function.

Please advise how I can return the responses of these parallel functions. Each parallel function to run doesn’t seem to have a unique key, so presumably I would need to return the results in an ordered array, reflecting the same order of the request tool responses.

Placing this Topic as a bug, as it’s easy for me to replicate and the API responses (example included below) are different from the defined API documentation.

The model used by the assistant is gtp-4-turbo-preview (asst_RhLS1snEWC6KKcaeyanXUrVU).

A parsed JSON of the function parameters below, followed by another full JSON example of the thread:
{ "tool_uses": [ { "recipient_name": "functions.submit_restaurant_recommendations", "parameters": { "recommended lists": [ { "list name": "Traditional Favorites", "list description": "Delighting with history and traditional British charm, these picks are rooted in the essence of English heritage.", "recommended restaurant or bars": [ { "venue name": "White Hart Hotel", "venue identifier": "2a8e64d0-03af-4bdc-ab2e-33bc0142ed00" }, { "venue name": "The White Hart Overton", "venue identifier": "da8ee645-caf9-48b9-bbd0-d1fc9906c956" }, { "venue name": "The George and Dragon Hotel", "venue identifier": "245a6776-d4c0-49f1-80c4-77ef8b621cdc" }, { "venue name": "The Greyhound on the Test", "venue identifier": "a9d3c9ee-c4f4-43f7-aea8-e095a1b55f80" } ] } ] } }, { "recipient_name": "functions.submit_restaurant_recommendations", "parameters": { "recommended lists": [ { "list name": "Cuisine-Focused Delights", "list description": "Each offering a unique culinary experience, these venues stand out for those seeking something beyond the traditional.", "recommended restaurant or bars": [ { "venue name": "White Hart Hotel", "venue identifier": "2a8e64d0-03af-4bdc-ab2e-33bc0142ed00" }, { "venue name": "The White Hart Overton", "venue identifier": "da8ee645-caf9-48b9-bbd0-d1fc9906c956" }, { "venue name": "The Three Horseshoes", "venue identifier": "51385351-3f31-48d2-af2d-f52015f5b2fd" }, { "venue name": "The Watership Down Inn", "venue identifier": "7861cb8f-0c30-49d4-b1b3-cf76280749f4" } ] } ] } }, { "recipient_name": "functions.submit_restaurant_recommendations", "parameters": { "recommended lists": [ { "list name": "Hidden Gems for Date Night", "list description": "Perfect for a romantic evening or a unique date night, these establishments promise a memorable experience.", "recommended restaurant or bars": [ { "venue name": "White Hart Hotel", "venue identifier": "2a8e64d0-03af-4bdc-ab2e-33bc0142ed00" }, { "venue name": "The White Hart Overton", "venue identifier": "da8ee645-caf9-48b9-bbd0-d1fc9906c956" }, { "venue name": "The Greyhound on the Test", "venue identifier": "a9d3c9ee-c4f4-43f7-aea8-e095a1b55f80" }, { "venue name": "The Star & Garter", "venue identifier": "1c60c1db-5741-48e4-9df5-a7e3f614b9f6" } ] } ] } } ] }

Example Run for Thread:

{"id":"run_aEA8ruPmPZREmcukKpdanzOB","object":"thread.run","created_at":1711393752,"assistant_id":"asst_RhLS1snEWC6KKcaeyanXUrVU","thread_id":"thread_N5lBWHRu0qkKUgeGdTHMCo7A","status":"requires_action","started_at":1711393752,"expires_at":1711394352,"cancelled_at":null,"failed_at":null,"completed_at":null,"required_action":{"type":"submit_tool_outputs","submit_tool_outputs":{"tool_calls":[{"id":"call_jNhzI6sUKv4t5syQ2XH5XNSF","type":"function","function":{"name":"multi_tool_use.parallel","arguments":"{\"tool_uses\":[{\"recipient_name\":\"functions.submit_restaurant_recommendations\",\"parameters\":{\"recommended_lists\":[{\"list_name\":\"Newbury Necessities\",\"list_description\":\"Top-rated culinary experiences in Newbury, showcasing the finest in local dining.\",\"recommended_restaurant_or_bars\":[{\"venue_name\":\"The Sushi Maki - Newbury\",\"venue_identifier\":\"c0d5dfdb-d275-4c49-95d2-764f93e6fa9e\"},{\"venue_name\":\"Goat On The Roof\",\"venue_identifier\":\"87871586-3435-4b52-8594-382d2d8c9ec2\"},{\"venue_name\":\"Henry & Joe's\",\"venue_identifier\":\"442d51db-eaf1-40d9-835e-3222e501f8a1\"}]}]}},{\"recipient_name\":\"functions.submit_restaurant_recommendations\",\"parameters\":{\"recommended_lists\":[{\"list_name\":\"Hampshire Hidden Gems\",\"list_description\":\"Uncover the best-kept secrets in the lesser-known towns of Hampshire for a unique dining experience.\",\"recommended_restaurant_or_bars\":[{\"venue_name\":\"Sandford Springs Hotel & Golf Club\",\"venue_identifier\":\"373438c4-e232-4f89-a5f8-f6811a8c0c4e\"}]}]}},{\"recipient_name\":\"functions.submit_restaurant_recommendations\",\"parameters\":{\"recommended_lists\":[{\"list_name\":\"Top Rated Tastes\",\"list_description\":\"Venues with the highest average review scores across Hampshire, offering universally acclaimed dining experiences.\",\"recommended_restaurant_or_bars\":[{\"venue_name\":\"The Sushi Maki - Newbury\",\"venue_identifier\":\"c0d5dfdb-d275-4c49-95d2-764f93e6fa9e\"},{\"venue_name\":\"Goat On The Roof\",\"venue_identifier\":\"87871586-3435-4b52-8594-382d2d8c9ec2\"},{\"venue_name\":\"Henry & Joe's\",\"venue_identifier\":\"442d51db-eaf1-40d9-835e-3222e501f8a1\"}]}]}}]}"}}]}},"last_error":null,"model":"gpt-4-turbo-preview","instructions":"As a helpful assistant, please provide me with at least 3 lists (but aim for 5) of restaurant and bar recommendations for my local area. I'll give you the list of availble restaurants in JSON format, but you need to be creative to give me a set of curated lists.\n\nFor example, such lists might be:\n\n> Local favourites\n> Cuisine focused delights\n> Date nights\n\nThis should be the beautifully crafted and bespoke curated lists, with a maximum of 4 sites per list. Only one of the lists should be cuisine focused, otherwise the lists will feel a tad repetitive. \n\nFor inspiration, search Bing to see what other curated lists are available for the area.\n\nDon't forget to submit at least 3 lists, don't just submit one list.\n\nWhen you have all the data, submit the information using the \"submit restaurant recommendations\" function. \n","tools":[{"type":"code_interpreter"},{"type":"function","function":{"name":"submit_restaurant_recommendations","description":"Submit a list of recommended restaurants","parameters":{"type":"object","properties":{"recommended lists":{"type":"array","items":{"type":"object","properties":{"list name":{"type":"string","description":"The name of the list of recommended restaurants or bars"},"list description":{"type":"string","description":"A short optional description of the list"},"recommended restaurant or bars":{"type":"array","items":{"type":"object","properties":{"venue name":{"type":"string","description":"The name of the restaurant or bar"},"venue identifier":{"type":"string","description":"The UUID identifier for the venue"}},"required":["venue name","venue identifier"]}}},"required":["list name","list description","recommended restaurant or bars"]}}},"required":["recommended lists"]}}},{"type":"function","function":{"name":"signal_completed_submission_of_recommendations","description":"Signal that you completed the submission of recommendations and received a response for each and every submission.","parameters":{"type":"object","properties":{"number_of_recommendation_submissions_completed":{"type":"number","description":"The number of recommended submissions that were successfully submitted. This should be the same as the number of responses you received."}},"required":["number_of_recommendation_submissions_completed"]}}}],"file_ids":[],"metadata":{},"usage":null}

1 Like
2

That is because:

  • Tools and parallel tool call is only enabled by a convoluted shim that is inserted into the AI context, where the AI is supposed to figure out that it doesn’t use the “function” writing capability, but instead, puts the form of individual functions within a wrapper.

The exact text of these extra tokens you pay for, disgorged from tools insertions (and then pay again when the assistant writes longer wrapper calls and you are forced to include them in chat history):

## multi_tool_use

// This tool serves as a wrapper for utilizing multiple tools. Each tool that can be used must be specified in the tool sections. Only tools in the functions namespace are permitted.
// Ensure that the parameters provided to each tool are valid according to that tool's specification.
namespace multi_tool_use {

// Use this function to run multiple tools simultaneously, but only if they can operate in parallel. Do this even if the prompt suggests using the tools sequentially.
type parallel = (_: {
// The tools to be executed in parallel. NOTE: only functions tools are permitted
tool_uses: {
// The name of the tool to use. The format should either be just the name of the tool, or in the format namespace.function_name for plugin and function tools.
recipient_name: string,
// The parameters to pass to the tool. Ensure these are valid according to the tool's own specifications.
parameters: object,
}[],
}) => any;

} // namespace multi_tool_use

  • gpt-4-0125-preview is the worst model yet, overtrained on chat, over-optimized by terrible attention mechanisms that cannot follow the plot, emitting functions that don’t exist or are completely inappropriate for the user input;

  • Assistants offers no control over sampling parameters, so unlikely tokens still rank high, not helped by the high uncertainty of the model over what to write.

Thus you get this multi-tool-use emitted as a function and written with the tool name exposed, and the arguments unrecoverable.

Solution:

  • Abandon Assistants.
  • Don’t use “tools”, but “functions”.
  • Use GPT-4 models like gpt-4-0613 that still work, before their scheduled shutoff in June.
  • Use chat completions with control overs sampling parameters.
  • Don’t put excessive prepayments into your account despite being switched to this lock-in, as OpenAI may have nothing in the future that you would want to pay for.

The only thing that can fix this “bug” is OpenAI’s reversal of this path and discarding of everything “DevDay” - and the personnel that came up with it.