Can app metadata and tool descriptions improve how ChatGPT discovers tools from installed Apps?

ChatGPT Apps SDK
1

Hi everyone,

I’m working on a ChatGPT App with several tools that help users with travel-related tasks, such as:

  • comparing travel insurance options
  • finding eSIM/mobile data plans for a destination
  • checking visa or entry requirements
  • finding airport lounges or VIP lounge access

I’m trying to understand how much we can improve tool discovery through metadata and prompt-style descriptions.

For example, if a user says:

“Do I need internet for Japan?”
“Can I wait comfortably at Madrid airport?”
“Do I need a visa for Canada with an Argentine passport?”

I want ChatGPT to correctly infer the relevant installed App/tool, even when the user does not mention the exact tool category directly.

Questions:

  1. How much does ChatGPT rely on the app description, tool names, tool descriptions, and parameter descriptions when deciding whether to call a tool from an installed App?

  2. Is it recommended to write tool descriptions using explicit intent patterns like “Use this when the user wants…”?

  3. Should tool descriptions include indirect examples such as “internet abroad”, “mobile data in Europe”, “airport lounge access”, or “entry requirements”?

  4. Is there a risk of adding too many synonyms or examples and making the tool appear too broad?

  5. What is the best way to test precision and recall for tool discovery?
    For example, should we create a prompt set with:

    • direct prompts
    • indirect prompts
    • negative prompts where the tool should not be called

  6. For purchase-related flows, is it better to separate browsing tools from checkout tools?
    Example:

    • one tool to browse eSIM plans
    • another tool only when the user chooses a specific plan or clearly asks to purchase

  7. Are there any current best practices for improving installed App/tool discovery without making metadata look like keyword stuffing?

Any guidance, examples, or documentation references would be appreciated.

2

Great questions, @ManuelDario.

I’ve not dove in on apps myself, but I’m watching this thread with interest!

3

Hey there – good questions all around, I’m going to give you the vague answer, less because your question isn’t valid and more because the best practices have been extremely in flux, and as I’m sure you’ve heard about our plans to merge ChatGPT and Codex, many of my answers will change. My recommendation in general is to accept that there are very few known best practices here besides:

  1. Making the suggested intent of the tool clear to the model,
  2. Leaning on model-written language for tool descriptions (models are generally fairly good at writing for themselves), and
  3. Writing lots of evals + hill-climbing on them

Our dev docs are similarly vague for this reason, and our apps/plugin-writing teams all find new successful practices on a daily basis. That’s of course not to say you should give up on learning to build on the models, and more to say that the only way to truly learn them is to try things, and try things constantly.