# What is the exact system prompt that gets inserted when calling a tool? **URL:** \ **Category:** API\ **Tags:** gpt-4, chatgpt, api, assistants-api\ **Created:** [December 30, 2024, 2:31am UTC](https://community.openai.com/t/what-is-the-exact-system-prompt-that-gets-inserted-when-calling-a-tool/1075113 "2024-12-30T02:31:09Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![GoldenJoe](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/goldenjoe/32/647877_2.png) [@GoldenJoe](https://community.openai.com/u/GoldenJoe)\ **Post date:** [December 30, 2024, 2:31am UTC](https://community.openai.com/t/what-is-the-exact-system-prompt-that-gets-inserted-when-calling-a-tool/1075113/1 "2024-12-30T02:31:09Z")
I review the documentation for providers that we use periodically, and I came across this helpful snippet in Anthropic’s [tool use](https://docs.anthropic.com/en/docs/build-with-claude/tool-use#tool-use-system-prompt) documentation: > When you call the Anthropic API with the `tools` parameter, we construct a special system prompt from the tool definitions, tool configuration, and any user-specified system prompt. The constructed prompt is designed to instruct the model to use the specified tool(s) and provide the necessary context for the tool to operate properly: ```auto In this environment you have access to a set of tools you can use to answer the user's question. {{ FORMATTING INSTRUCTIONS }} String and scalar parameters should be specified as is, while lists and objects should use JSON format. Note that spaces for string values are not stripped. The output is not expected to be valid XML and is parsed with regular expressions. Here are the functions available in JSONSchema format: {{ TOOL DEFINITIONS IN JSON SCHEMA }} {{ USER SYSTEM PROMPT }} {{ TOOL CONFIGURATION }} ``` I assume OpenAI does something similar. It would be helpful if you provided the structure of your tool usage prompt in the documentation. I’ve experimented with hundreds of different formats for tool calls, changing parameter order, description formatting, anything you can think of. I can confirm that _formatting_ is a factor in model performance for calling tools. Knowing the format that already exists in the behind-the-scenes prompt would allow us to craft schema that is faster and more accurate for us, and less demanding of OpenAI compute resources. ---
**Author:** ![Diet](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/diet/32/562749_2.png) [@Diet](https://community.openai.com/u/Diet)\ **Post date:** [December 30, 2024, 3:43am UTC](https://community.openai.com/t/what-is-the-exact-system-prompt-that-gets-inserted-when-calling-a-tool/1075113/2 "2024-12-30T03:43:13Z")
This isn’t an answer to your question, but my understanding of the matter: * * * As far as I know OpenAI, what they would want to tell you is this: “don’t worry about it.” In like a NY italian drawl kinda way. OAI likes its black boxes, and trying to figure them out can get your account flagged. (discussion about o1 [Your prompt was flagged as potentially violating our usage policy](https://community.openai.com/t/your-prompt-was-flagged-as-potentially-violating-our-usage-policy/1060163)) My personal suggestion - if you want maximal control - would be to ditch tools/function calls altogether and roll you own schema. > [@GoldenJoe](#): > > and less demanding of OpenAI compute resources. OAI pricing is completely divorced from cost anyways, if they really cared about that then we’d know by now… ---
**Author:** ![GoldenJoe](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/goldenjoe/32/647877_2.png) [@GoldenJoe](https://community.openai.com/u/GoldenJoe)\ **Post date:** [December 30, 2024, 9:34am UTC](https://community.openai.com/t/what-is-the-exact-system-prompt-that-gets-inserted-when-calling-a-tool/1075113/3 "2024-12-30T09:34:05Z")
> [@Diet](#): > > My personal suggestion - if you want maximal control - would be to ditch tools/function calls altogether and roll you own schema. To be clear, are you suggesting using structured outputs instead of tool calls, or something else? I suppose that’s all they are behind the scenes, but it is exhaustive to constantly [reinvent the wheel](https://community.openai.com/t/best-alternative-to-assistants-api/1073526/3). It’s ridiculous. The total lack of support even on these forums has demolished my organization’s confidence in their platform. Once our service has stabilized, one of the first upgrades will be to roll out our own small model and cut all dependencies on OpenAI, Anthropic, and whatever other fallbacks we use in the meanwhile. ---
**Author:** ![Diet](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/diet/32/562749_2.png) [@Diet](https://community.openai.com/u/Diet)\ **Post date:** [December 30, 2024, 12:24pm UTC](https://community.openai.com/t/what-is-the-exact-system-prompt-that-gets-inserted-when-calling-a-tool/1075113/4 "2024-12-30T12:24:12Z")
> [@GoldenJoe](#): > > To be clear, are you suggesting using structured outputs instead of tool calls, or something else? I would ditch structured outputs and potentially json mode too. I use none of that stuff anywhere, and it gives you a clean, vendor agnostic model adapter (with some minor caveats). The models are plenty capable of abiding by a (reasonable) schema. Have been since davinci-002. I always argue that if a model can’t abide by the schema (i.e. parsing error) it’s a good indicator that it wouldn’t really understand the prompt either. But it depends on what you’re doing, ofc. > [@GoldenJoe](#): > > The total lack of support even on these forums has demolished my organization’s confidence in their platform. Ah, I wouldn’t throw all the babies out with the bathwater. OAI has some redeeming qualities. The “base” models are pretty good. I don’t really care for all the vendor lock-in stuff (assistants, tools, structured outputs, their caching scheme, o1, etc.) If you ignore all that, it’s pretty stable. No wheel reinventing required. And everything becomes quite portable too. ---
**Author:** ![GoldenJoe](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/goldenjoe/32/647877_2.png) [@GoldenJoe](https://community.openai.com/u/GoldenJoe)\ **Post date:** [December 31, 2024, 11:37pm UTC](https://community.openai.com/t/what-is-the-exact-system-prompt-that-gets-inserted-when-calling-a-tool/1075113/5 "2024-12-31T23:37:28Z")
> [@Diet](#): > > OAI has some redeeming qualities. 4o-mini offers by far the best cost/capability ratio of anything I’ve used so far. It legitimately opened up new possibilities for AI-supported applications. I don’t know how long it will last - no company likes to have loss leaders in the lineup forever - but it is the reason I have stuck around and put up with the black boxes. ---
**Author:** ![arata](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/arata/32/523049_2.png) [@arata](https://community.openai.com/u/arata)\ **Post date:** [January 1, 2025, 1:38am UTC](https://community.openai.com/t/what-is-the-exact-system-prompt-that-gets-inserted-when-calling-a-tool/1075113/6 "2025-01-01T01:38:43Z")
The only “tool” that you have access to is a `functions` tool, for function-calling. OpenAI reserves their actual higher level tools (and a named API backend recipient for catching triggered responses to them) for themselves. The models are post-trained on function-calling, so the transformation of a function specification into AI language being placed, a heading “# Tools” after your system prompt language with a function tool, is not directly instructed but rather is understood by the AI model in the structure used. The only useful thing to know is when the function fails to be understood, such as capricious use of the ‘required’ signal, or convoluted object or array nesting that a human would have a hard time deciphering. You can make a system prompt giving you developer authority to have the `# Tools` section dumped out completely. ---
**Author:** ![GoldenJoe](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/goldenjoe/32/647877_2.png) [@GoldenJoe](https://community.openai.com/u/GoldenJoe)\ **Post date:** [January 1, 2025, 4:20am UTC](https://community.openai.com/t/what-is-the-exact-system-prompt-that-gets-inserted-when-calling-a-tool/1075113/7 "2025-01-01T04:20:47Z")
Thanks for the thoughtful response. > [@arata](#): > > The models are post-trained on function-calling, so the transformation of a function specification into AI language being placed, a heading “# Tools” after your system prompt language with a function tool, is not directly instructed but rather is understood by the AI model in the structure used. Are we certain of this? Different implementations are certainly possible, but I would be surprised if there was such a fundamental difference between OpenAI and Anthropic. > [@arata](#): > > You can make a system prompt giving you developer authority to have the `# Tools` section dumped out completely. Can you provide an example? I’m having difficulty getting it to return a reasoning message alongside the tool request message. Or do you achieve this by including a “reasoning” parameter in each tool?