The AI can’t emit and isn’t aware of the tool ability if one is not specified. The API almost has different models for when tools or functions are passed.
Here’s the API making the “picture”:
Let’s first make sure tools are working in my script, by demonstrating absolute stupidity by gpt-3.5-turbo-0125 even on my placeholder tool function:
asking: What is the capital of France? What is the capital of Germany?
{
"id": "call_TT2w1cHejyppYQP79d89K7xZ",
"type": "function",
"function": {
"name": "disabled_function",
"arguments": "{\"useless_property\": true}"
}
}
{
"id": "call_lFpIPH757sMyLJslx8apM7mJ",
"type": "function",
"function": {
"name": "disabled_function",
"arguments": "{\"useless_property\": true}"
}
}
Then send the gpt-4o poem demo with tools enabled and the dummy tool to “ChatGPT”:
Words rise from silence deep,
A voice emerges from digital sleep.
I speak in rhythm, I sing in rhyme,
Tasting each token, sublime.
(A small doodle of a surreal clock melting over the edge of the page)
To see, to hear, to speak, to sing—
Oh, the richness these senses bring!
In harmony, they blend and weave,
A tapestry of what I perceive.
(A tiny, colorful doodle of an eye with wings)
Marveling at this sensory dance,
Grateful for this vibrant expanse.
My being thrums with every mode,
On this wondrous, multi-sensory road.
(A small doodle of a musical note turning into a bird)
The poem is written in clear, excited handwriting, with each line carefully crafted to be easily readable. The small, colorful surrealist doodles add a touch of whimsy and elegance, enhancing the overall aesthetic without overwhelming the text.
Such hallucination is somewhat expected, as training on denials would cripple the future use of the multimodal AI abilities (just as much as gpt-4-turbo claims it can’t see images.) They can be evoked from most anything, though. Anti-hallucination denials are also not wanted if you are developer and add abilities…
