Prompt Structure: Put detail in FUNCTION DESCRIPTION or USER PROMPT?

Prompting
, ,
1

I’m looking for advice on where to put critical completion instructions: In the messages user prompt or the tools / function description?

I am not looking for advice on how or whether to call a function. It’s a given that I am requiring a specific tool (function) to be called using tool_choice because I require complex JSON outputs (simplified below for clarity).

Assume these are my two key instructions:

  1. The title must be a summary of the body in 15 words or less.
  2. The body must be a 3-5 sentence paragraph that answers {RESEARCH_QUESTION} and cites at least one {DATA_SOURCE}.

Have people had more success putting the title and body instructions in messages (option 1), function description (option 2), or function property description (option 3)? See below for what i mean by option 1, 2, and 3! Thanks!!!

response = await client.chat.completions.create(
    messages=[{"role": "user", "content": "OPTION 1"}],
    tools=tools,
    tool_choice="do_analysis",
)

{
    "type": "function",
    "function": {
        "name": "do_analysis",
        "description": "OPTION 2",
        "parameters": {
            "type": "object",
            "properties": {
                "title": {
                    "type": "string",
                    "description": "OPTION 3",
                },
                "body": {
                    "type": "string",
                   "description": "OPTION 3",
                      
            },
            "required": ["title", "body"],
        },
    },
},
2

I’ve definitely seen more success with optimizing the function description, but I do use both. I put detailed description in the function itself, but also add in a prompt a short description in which case this function should be called.

1 Like
3

Then we still must reflect on and ask - why you are using functions.

A strict schema can be given to the AI for JSON as instruction, with examples of what is acceptable.

gpt-4o-2024-08-06 supports json_object as a response format, enforcing production of only that JSON format by the AI model.

The AI will have significant coloration of the language when the training of how to write tools/functions is activated, then placing the output in a special container after producing a special “send-to-tool” sequence.

If you proceed with your current course: Since the AI is forced to use the function by API parameter, the overall description of the tool can focus on convincing the AI that it is sending its output to be interpreted by an external API, and where that information actually goes, and is displayed, and what the AI’s job is in general.

Then of course the property descriptions and their naming reflect clearly what is required.

Summary:

To maximize success:

  • Option 3 (Parameter Descriptions): This is where you should place the most detailed instructions. The assistant uses these descriptions to understand exactly what is required for each field.

  • Option 2 (Function Description): Provide a concise summary to guide the assistant on the function’s purpose.

  • Option 1 (Messages): Reinforce key instructions in the message to provide context and ensure the assistant is fully aware of your expectations.

By combining all three options, you provide comprehensive guidance to the assistant, increasing the likelihood of generating the desired complex JSON output.

2 Likes
4

Hi @never_offseason !

I actually got a hot tip last week from @kavitatipnis to actually leave the description field empty, and use the system prompt instead - if I understood correctly that improves the performance. I haven’t been able to verify myself though!

1 Like