# New API feature: forcing function calling via \`tool\_choice: "required"\` **URL:** \ **Category:** API\ **Tags:** chat-completion, function-calling, assistants-api, tools, tool-choice\ **Created:** [April 29, 2024, 5:51pm UTC](https://community.openai.com/t/new-api-feature-forcing-function-calling-via-tool-choice-required/731488 "2024-04-29T17:51:25Z")\ **Posts on this page:** 17\ **Page:** 1
**Author:** ![brianz-oai](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/brianz-oai/32/354481_2.png) [@brianz-oai](https://community.openai.com/u/brianz-oai)\ **Post date:** [April 29, 2024, 5:51pm UTC](https://community.openai.com/t/new-api-feature-forcing-function-calling-via-tool-choice-required/731488/1 "2024-04-29T17:51:25Z")
We’ve added the ability to force the model to call one of your provided functions (via tools) in the Chat Completions and Assistants API by setting `tool_choice: "required"`. When you set `tool_choice: "required"`, the model will determine which function(s) are relevant and call them. For more details, reference our [guide](https://platform.openai.com/docs/guides/function-calling/function-calling-behavior) and [API reference](https://platform.openai.com/docs/api-reference/chat/create#chat-create-tool_choice). Give it a try and let us know your feedback! ---
**Author:** ![\_j](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/_j/32/766292_2.png) [@\_j](https://community.openai.com/u/_j)\ **Post date:** [April 29, 2024, 6:04pm UTC](https://community.openai.com/t/new-api-feature-forcing-function-calling-via-tool-choice-required/731488/2 "2024-04-29T18:04:33Z")
Thanks. It seems OpenAI keeps the special tokens and sequences the AI writes out of the scope of logit\_bias, including those initial generated tokens which are signaling tool calls, preventing flexibility and even workarounds of AI model training issues. Even then, to the point of providing an alternate version of logprobs and just shutting logprobs off on function call, as if token numbers and their encoding strings were some secret genius plan. Feedback: Especially on chat completions, `tool_bias` as an abstraction performed on the first tokens could let one control the entry into a tool-generation sequence with more nuanced hinting to the AI softmax. Essentially piggybacking on how certain the AI is of invoking a tool as a response to increase or discourage that certainty. ---
**Author:** ![ramn7](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/ramn7/32/195226_2.png) [@ramn7](https://community.openai.com/u/ramn7)\ **Post date:** [April 30, 2024, 11:53am UTC](https://community.openai.com/t/new-api-feature-forcing-function-calling-via-tool-choice-required/731488/3 "2024-04-30T11:53:20Z")
Thanks for this update! 1. Is there a dedicated way to follow on this kind of minor API updates in the future? (I care about this stuff, and almost missed this post) 2. Will this have an effect on whether or not `message.content` is returned along with `message.tool_calls`? (guessing not but I guess worth asking, as a request for `tool_calls` only is imho a valid use case) ---
**Author:** ![LLM](https://avatars.discourse-cdn.com/v4/letter/l/7ba0ec/32.png) [@LLM](https://community.openai.com/u/LLM)\ **Post date:** [May 1, 2024, 3:31am UTC](https://community.openai.com/t/new-api-feature-forcing-function-calling-via-tool-choice-required/731488/4 "2024-05-01T03:31:13Z")
Thank you @brianz-oai , this is really useful. I’m wondering for option 2: > To force the model to call only one specific function Is there a way to force calling _multiple_ functions? ---
**Author:** ![brianz-oai](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/brianz-oai/32/354481_2.png) [@brianz-oai](https://community.openai.com/u/brianz-oai)\ **Post date:** [May 1, 2024, 5:36am UTC](https://community.openai.com/t/new-api-feature-forcing-function-calling-via-tool-choice-required/731488/5 "2024-05-01T05:36:39Z")
Hi Ram, Thanks for your questions. 1. There is a [changelog page](https://platform.openai.com/docs/changelog) you can follow for API feature releases like this! 2. I assume you are referring to the behavior where under the default `tool_choice: "auto"` setting, the model sometimes outputs a `message.content` along side `message.tool_calls`. When you set `tool_choice: "required"` or to a specific tool, the model won’t return any `message.content` in the current implementation. Brian ---
**Author:** ![brianz-oai](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/brianz-oai/32/354481_2.png) [@brianz-oai](https://community.openai.com/u/brianz-oai)\ **Post date:** [May 1, 2024, 5:44am UTC](https://community.openai.com/t/new-api-feature-forcing-function-calling-via-tool-choice-required/731488/6 "2024-05-01T05:44:20Z")
Hi, Unfortunately we don’t provide such an option in our API at the moment. As a workaround, since all of our newer models supports [parallel function calling](https://platform.openai.com/docs/guides/function-calling/parallel-function-calling), when you provided multiple tools and set `tool_choice: "required"`, the model can generally do a good job picking multiple relevant tools as long as you have a good prompt. Let me know if you have further questions. Brian ---
**Author:** ![ramn7](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/ramn7/32/195226_2.png) [@ramn7](https://community.openai.com/u/ramn7)\ **Post date:** [May 1, 2024, 8:01am UTC](https://community.openai.com/t/new-api-feature-forcing-function-calling-via-tool-choice-required/731488/7 "2024-05-01T08:01:57Z")
Thanks Brian! Regarding 2: yes, that’s what I meant. I have another followup questions if that’s ok: With `required`- does the model return an empty `tool_calls` in case nothing fits, or will it return anything that it thinks fits best? ---
**Author:** ![brianz-oai](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/brianz-oai/32/354481_2.png) [@brianz-oai](https://community.openai.com/u/brianz-oai)\ **Post date:** [May 1, 2024, 10:11pm UTC](https://community.openai.com/t/new-api-feature-forcing-function-calling-via-tool-choice-required/731488/8 "2024-05-01T22:11:06Z")
Under `tool_choice: "required"`, the model will be forced to pick the most relevant one to call, even if none really fits the prompt. Returning an empty `tool_calls` when there isn’t an appropriate tool to call is what the default `tool_choice: "auto"` is for. ---
**Author:** ![\_j](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/_j/32/766292_2.png) [@\_j](https://community.openai.com/u/_j)\ **Post date:** [May 2, 2024, 1:59am UTC](https://community.openai.com/t/new-api-feature-forcing-function-calling-via-tool-choice-required/731488/9 "2024-05-02T01:59:18Z")
Perhaps add to documentation how many steps this is imposed of the AI. An assistant that could not output anything but tools would have no mechanism to respond to the user, and an “out” like a “final\_answer” function could not be used in assistants without abandoning the thread. Likewise, a tool combo requiring iterations like a “search” and then a “load” would not have the same mandate on followups if it was just the initial turn, defeating expectations. ---
**Author:** ![will\_becker](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/will_becker/32/361126_2.png) [@will\_becker](https://community.openai.com/u/will_becker)\ **Post date:** [May 10, 2024, 10:48pm UTC](https://community.openai.com/t/new-api-feature-forcing-function-calling-via-tool-choice-required/731488/10 "2024-05-10T22:48:00Z")
I’ve found this works poorly with the Assistants API when I am trying to get it to answer a prompt in relation to a file and then perform a function call @brianz-oai . If I don’t have `tool_choice={"type": "file_search"}` then it seems to not want to call my function if the file search doesn’t return any results (even though in this case I have prompted it explicitly to call my function in a specific manner) If I call it with `tool_choice={"type": "function", "function": {"name": "my_fn"}},` it neglects to do a file search. If I call it with `tool_choice="required"` its a crap shoot! I guess I have to rely on validating the result and retrying if it doesn’t match, but it would be nice if function tools and internal tools were handled separately here. ---
**Author:** ![WilliamBakst](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/williambakst/32/372537_2.png) [@WilliamBakst](https://community.openai.com/u/WilliamBakst)\ **Post date:** [May 18, 2024, 12:58pm UTC](https://community.openai.com/t/new-api-feature-forcing-function-calling-via-tool-choice-required/731488/11 "2024-05-18T12:58:43Z")
I’m noticing that when setting `"tool_choice": "required"` that `finish_reason` is `stop` and not `tool_calls`. This seems wrong? ---
**Author:** ![brianz-oai](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/brianz-oai/32/354481_2.png) [@brianz-oai](https://community.openai.com/u/brianz-oai)\ **Post date:** [May 20, 2024, 8:52pm UTC](https://community.openai.com/t/new-api-feature-forcing-function-calling-via-tool-choice-required/731488/12 "2024-05-20T20:52:04Z")
I’m not super familiar with Assistants API best practices but it sounded like in your case, one solution would be to first trigger a run for the file\_search tool use (without enabling the function tool), and then a separate run that requires `tool_choice` while enabling function tool? ---
**Author:** ![brianz-oai](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/brianz-oai/32/354481_2.png) [@brianz-oai](https://community.openai.com/u/brianz-oai)\ **Post date:** [May 20, 2024, 9:00pm UTC](https://community.openai.com/t/new-api-feature-forcing-function-calling-via-tool-choice-required/731488/13 "2024-05-20T21:00:29Z")
Yes, that is the expected behavior for `tool_choice: "required"`, though I agree it’s somewhat confusing. To provide a bit more context, before we introduced this new feature, when you set `tool_choice: {"type": "function", "function": {"name": "my_function"}}`, the `finish_reason` would always be `stop` rather than `tool_calls`. Only when you used the default `tool_choice: "auto"` option, and the model chose to use a tool, the `finish_reason` would be `tool_use`. So when we designed this new feature, we thought it made more sense to provide the consistent behavior as `tool_choice: {"type": "function", "function": {"name": "my_function"}}` as the two are more similar (i.e. model is forced to use a tool). Fixing this now could potentially break some users’ integration, but we will almost certainly fix this when we release the next API version. ---
**Author:** ![WilliamBakst](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/williambakst/32/372537_2.png) [@WilliamBakst](https://community.openai.com/u/WilliamBakst)\ **Post date:** [May 20, 2024, 9:50pm UTC](https://community.openai.com/t/new-api-feature-forcing-function-calling-via-tool-choice-required/731488/14 "2024-05-20T21:50:45Z")
Sounds good! Thanks for clarifying. This isn’t an issue and I understand where you’re coming from. I’ve made updates to handle this, and agree that it’s better to maintain existing integrations with your API. Thank you! ---
**Author:** ![bonsaielectric](https://avatars.discourse-cdn.com/v4/letter/b/e19b73/32.png) [@bonsaielectric](https://community.openai.com/u/bonsaielectric)\ **Post date:** [May 28, 2024, 4:13pm UTC](https://community.openai.com/t/new-api-feature-forcing-function-calling-via-tool-choice-required/731488/15 "2024-05-28T16:13:07Z")
@brianz-oai Do you happen to know when will tool\_choice: “required” be available in azure openai? ---
**Author:** ![grandell1234](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/grandell1234/32/286916_2.png) [@grandell1234](https://community.openai.com/u/grandell1234)\ **Post date:** [May 28, 2024, 4:28pm UTC](https://community.openai.com/t/new-api-feature-forcing-function-calling-via-tool-choice-required/731488/16 "2024-05-28T16:28:39Z")
---
**Author:** ![brianz-oai](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/brianz-oai/32/354481_2.png) [@brianz-oai](https://community.openai.com/u/brianz-oai)\ **Post date:** [May 28, 2024, 6:13pm UTC](https://community.openai.com/t/new-api-feature-forcing-function-calling-via-tool-choice-required/731488/17 "2024-05-28T18:13:44Z")
I don’t have visibility into or knowledge on that, but I’d guess it won’t be long.