Complex function calling return value

API
1

The limit of one string return value of tool calls

When calling a function it is often useful to inform the LLM not only about the pure functional outcome - but also give it some ‘meta info’. For example I have a wikipedia tool and when the LLM calls 'get_url(“Joan Caulfield - Wikipedia”)

I return the following information:


`Getting` https://en.wikipedia.org/wiki/Joan_Caulfield

`Successfully retrieved document from url: 'https://en.wikipedia.org/wiki/Joan_Caulfield'`

`The retrieved page contains the following sections:`

`"## Early life and education", "## Career", "### Stage", "### Film", "### Television", "### Later years", "## Personal life", "## Legacy", "## Radio appearances", "## References", "## External links"`

`*Hint:** You can easily jump to any section by using \`lookup` function with the section name.`

`The retrieved page starts with:`

`> American actress (1922–1991)`

`>`

`> Joan Caulfield`

`> ---`

`> [](https://en.wikipedia.org/wiki/File:JOANCAulfield.jpg)`

`> Born| Beatrice Joan Caulfield`

`> (1922-06-01)June 1, 1922`

`> [West Orange, New Jersey](https://en.wikipedia.org/wiki/West_Orange,_New_Jersey), U.S.`

`> Died| June 18, 1991(1991-06-18) (aged 69)`

`> [Los Angeles](https://en.wikipedia.org/wiki/Los_Angeles), [California](https://en.wikipedia.org/wiki/California), U.S.`

`> Occupation(s)| Actress, model`

`> Years active| 1941–1987`

`> Spouses|`

`>`

`> *`

`> [Frank Ross](https://en.wikipedia.org/wiki/Frank_Ross_\(producer\)) ​ ​(m. 1950; div. 1960)​`

`>`

`> *`

`> Robert Peterson ​ ​(m. 1960; div. 1966)​`

`> Children| 2`

`> Relatives| [Genevieve Caulfield](https://en.wikipedia.org/wiki/Genevieve_Caulfield) (aunt)`

`>`

`**Hint:** This was not the full content of the page. If you want to continue reading the page, you can call \`read_more`.If you want to jump to a specific keyword on this page (for example a section of the article) `lookup('keyword')`.`

Technically the output should be just the quoted fragment - but I want the LLM to learn about the page sections and give it hints on what it can use in further queries etc.

In another case when it looks up multi-word phrase and that looup fails I advice it to look for single words - because words in phrases can sometimes go in different order or there might be some additional words (like ‘this’ or ‘a’) inserted, or additional diacritics - etc. I do want to give the LLM an option to search for phrases - but if such a search fails I want to worn it and advice to maybe modify the search term.

I am not sure if the Mardown format that I currently use is optimal - I’ve seen somewhere that LLMs understand JSON even better and actually outputing JSON would be simpler.

When I look at the first example of a tool result in the OpenAI tutorial:

function_call_result_message = {
    "role": "tool",
    "content": json.dumps({ "order_id": order_id, "delivery_date": delivery_date.strftime('%Y-%m-%d %H:%M:%S') }),
    "tool_call_id": response['choices'][0]['message']['tool_calls'][0]['id']
}

It is indeed a json encoded dictionary.

What are your experiences with that problem?

And by the way wouldn’t it be more consistent if the output format was specified in the tool schema just like the inputs?

2

I am also thinking about inserting a ‘message from user’ after a function call for the hints. But using JSON seems more straightforward.