# Hallucinations in structured output when the source data is light

**URL:** <https://community.openai.com/t/hallucinations-in-structured-output-when-the-source-data-is-light/959196>\
**Category:** API\
**Tags:** structured-output\
**Created:** [September 27, 2024, 6:38pm UTC](https://community.openai.com/t/hallucinations-in-structured-output-when-the-source-data-is-light/959196 "2024-09-27T18:38:17Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![steven.g.lane](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/steven.g.lane/32/469342_2.png) [@steven.g.lane](https://community.openai.com/u/steven.g.lane)\
**Post date:** [September 27, 2024, 6:38pm UTC](https://community.openai.com/t/hallucinations-in-structured-output-when-the-source-data-is-light/959196/1 "2024-09-27T18:38:17Z")

</div>

Hi all:

I have got structured outputs working nicely. My main challenge right now is hallucinations when the source data is small. My use case is parsing info about art objects. Works great with a rich data source, but if my data source/prompt contains the single word “Ostade,” OpenAI picks a random work by the artist van Ostade and gives me or hallucinates data about it. Any way to minimize this? Will the temperature or seed settings make a difference? Or better prompt engineering? Thanks!

---

<div class="post-metadata">

**Author:** ![\_j](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/_j/32/766292_2.png) [@\_j](https://community.openai.com/u/_j)\
**Post date:** [September 30, 2024, 5:46am UTC](https://community.openai.com/t/hallucinations-in-structured-output-when-the-source-data-is-light/959196/3 "2024-09-30T05:46:25Z")

</div>

The AI pretty much HAS to write. You back it into a corner when using structured outputs, and all the keys must be filled in. There is no way to write denials if the AI doesn’t know the answer with certainty - or if the schema description says it will produce policy violations if the single key is ever used.

![Untitled](https://us1.discourse-cdn.com/openai1/original/4X/8/3/6/8366028e4f83ac28816622cb78deb7ed479666fa.jpeg)

The way I would see that you could avoid the similar problem of being forced to represent an answer as truthful, with only set keys that talk about the solution, is to use an anyOf choice of two different schemas (after a top level that is just a container).

When doing so, you can place a second schema, that is specifically crafted for the AI to escape with a “can’t answer”, perhaps making the way out obvious with some enums like “no information returned”, “too new”…

This is also a technique to get out of a bad function call. The AI could pick the “no useful function, reply to user” function when for some uncontrollable probability the AI decides to emit functions when none are of any use.

- By providing a way for the AI model to respond that it is uncertain, or even deny output, you can avoid hallucinations.
- Because a structured response schema is strict, the way to give an option is anyOf.

![Untitled](https://us1.discourse-cdn.com/openai1/original/4X/a/a/3/aa3a14c0f754819982097ffa678bc8bb1411081e.jpeg)

No more output field for something unanswerable!

(rest assured that there is no actual function return, but indeed, the AI will still see the response format somewhat as the job it is designed to do, so this demo response\_format I just wrote up, still with “you are a helpful assistant”, invokes description-less “policy violation” enum I provided far less than expected.)

* * *

> **One-job schema, with escape schema**
>
> {  
> “name”: “my\_response”,  
> “strict”: true,  
> “schema”: {  
> “type”: “object”,  
> “properties”: {  
> “my\_response”: {  
> “anyOf”: [  
> {  
> “type”: “object”,  
> “properties”: {  
> “company\_product\_search”: {  
> “type”: “string”  
> }  
> },  
> “required”: [  
> “company\_product\_search”  
> ],  
> “additionalProperties”: false  
> },  
> {  
> “type”: “object”,  
> “properties”: {  
> “no\_response\_reason”: {  
> “type”: “string”,  
> “enum”: [  
> “not\_parts\_request”,  
> “question\_unclear”,  
> “not\_company\_scope”,  
> “policy\_violation”  
> ]  
> }  
> },  
> “required”: [  
> “no\_response\_reason”  
> ],  
> “additionalProperties”: false  
> }  
> ],  
> “additionalProperties”: false  
> }  
> },  
> “required”: [  
> “my\_response”  
> ],  
> “additionalProperties”: false  
> }  
> }

---

<div class="post-metadata">

**Author:** ![tim.graf.codes](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/tim.graf.codes/32/366356_2.png) [@tim.graf.codes](https://community.openai.com/u/tim.graf.codes)\
**Post date:** [December 2, 2024, 7:00pm UTC](https://community.openai.com/t/hallucinations-in-structured-output-when-the-source-data-is-light/959196/4 "2024-12-02T19:00:48Z")

</div>

I’m seeing similar results as well. Any helpful replies or recommendations would be appreciated.

---

<div class="post-metadata">

**Author:** ![sergeliatko](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/sergeliatko/32/816_2.png) [@sergeliatko](https://community.openai.com/u/sergeliatko)\
**Post date:** [December 2, 2024, 9:36pm UTC](https://community.openai.com/t/hallucinations-in-structured-output-when-the-source-data-is-light/959196/5 "2024-12-02T21:36:22Z")

</div>

> [@steven.g.lane](#):
>
> but if my data source/prompt contains the single word “Ostade,” OpenAI picks a random work by the artist van Ostade and gives me or hallucinates data about it.

This is the main reason: model was not given enough context… I would see how to solve the following issues:

1. Detect if present context is enough
2. Means to get more context when necessary
3. Way for the model to refuse to answer on insufficient context

When those solved (welcome back to the drawing board), then you start evaluating responses.

---

<div class="post-metadata">

**Author:** ![Anthony\_Breneliere](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/anthony_breneliere/32/744393_2.png) [@Anthony\_Breneliere](https://community.openai.com/u/Anthony_Breneliere)\
**Post date:** [June 12, 2026, 4:09pm UTC](https://community.openai.com/t/hallucinations-in-structured-output-when-the-source-data-is-light/959196/6 "2026-06-12T16:09:06Z")

</div>

This thread still ranks for “structured output hallucination,” so adding what’s worked since — the anyOf escape hatch above is the right foundation, and two layers on top of it close most of the remaining gap (disclosure: I build a data-enrichment product, Entity Enricher, so this is the problem I work on daily):

**1. Check input sufficiency _before_ extraction, not after.** The “Ostade” failure is the model doing its best with one word — it can’t refuse, so it picks a plausible van Ostade work. A cheap pre-flight call (“does this input identify a specific art object unambiguously: yes/no/ambiguous, and why”) costs a fraction of a cent and routes thin inputs to a “needs more context” path before the expensive extraction ever runs. This is sergeliatko’s point 1, and it’s worth implementing literally — as its own small call with its own schema.

**2. Make every fact field nullable and instruct “prefer null over guessing.”** The anyOf escape handles all-or-nothing, but the more dangerous case is partial knowledge: the model knows the artist but not the date, and a required date field forces an invention. Nullable fields give it a legal way to leave exactly the unknown cells empty. Missing data is recoverable; plausible wrong data isn’t.

And if the data matters enough: run the same sparse input through two different models and diff the fields. On rich inputs they converge; on thin inputs they confabulate _differently_ — disagreement is the most reliable hallucination detector I’ve found, far better than asking a model to self-report confidence.

---

<div class="post-metadata">

**Author:** ![OpenAI\_Support](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/openai_support/32/561845_2.png) [@OpenAI\_Support](https://community.openai.com/u/OpenAI_Support)\
**Post date:** [September 9, 2026, 11:10pm UTC](https://community.openai.com/t/hallucinations-in-structured-output-when-the-source-data-is-light/959196/7 "2026-09-09T23:10:40Z")

</div>

Thanks for the example. Structured Outputs keeps the response in your schema, but doesn’t guarantee the facts: [https://developers.openai.com/api/docs/guides/structured-outputs](https://developers.openai.com/api/docs/guides/structured-outputs). Include an explicit unknown/null path for missing information and validate the result against a source. We’ll pass along the hallucination feedback; no specific fix timeline to share.