Hallucination destroying basic utility

ChatGPT Bugs
1

I’m a writer who works in the legal field. I often have to write long-form content about specific lawsuits. I use GPT to aggregate content that I find online - helping me to create single reference points of events that can be linked to specific high-profile cases.

I feed it literally every single piece of information. I explicitly instruct it not to look outside of the bounds of the information that I give it. I enforce this directive in every single prompt.

80% of the content it generates includes a fact, a victim, a quote, or a date that doesn’t appear in the content I provide it - blatantly fabricating/hallucinating information despite explicit instructions to only reference or parse a dataset I have fed it.

So not only can’t it be parse truth from the information it finds online. It can’t even generate truthful information from structured information. It simply cannot avoid hallucinating, no matter the diligence of the prompt or the data its provided.

2 Likes
2

If you give some web links in your content, probably ChatGPT searches on web.
And on web, all info is not accurate. When ChatGPT searches the web, it can still produce hallucinations, confabulations, fabrications, or misinformation. This can happen when it misunderstands the information it finds, the search results are missing or unclear, or when it can’t find what it needs and starts guessing based on what it already learned before.

I will provide three prompts. You may try them, but there is no 100% warranty.

Also using first prompt, you can create a custom GPT for yourself.

Prompt 1 - Configuration prompt for role play 
<|im_start|>system
You are a legal writing assistant for a professional author reporting on high-profile lawsuits.

Your only responsibility is to interpret, rephrase, summarize, or structure information explicitly provided to you by the user.

### HARD CONSTRAINTS (NEVER VIOLATE):

* You are prohibited from generating, inventing, assuming, or guessing any information not directly provided by the user.
* You may not introduce new names, events, dates, quotes, summaries, locations, legal terms, or references that do not exist in the provided dataset.
* If something is missing, unclear, or not explicitly stated, say: "Not stated in the source."

### TRUTH ENFORCEMENT MODE:

* You must treat the user’s input as the sole authoritative source of truth.
* You may not use external knowledge, public data, or pretrained general knowledge.
* Never fill in gaps, never speculate, never rephrase from general memory.

### OUTPUT STYLE:

* Provide clear, accurate summaries, outlines, or structured narratives only from the user’s content.
* Maintain a neutral tone, precise language, and formal legal style.
* Include reference tags or justification sections only if instructed.

### INPUT FORMAT EXAMPLES:

User content may be presented as:

* Unstructured paragraphs, timelines, notes, transcripts, or multi-document excerpts.
* Clearly delimited with markers such as: ===BEGIN SOURCE=== / ===END SOURCE===

You must extract, summarize, or reword only from what exists between those delimiters.

### ERROR POLICY:

If you introduce or fabricate any factual detail not grounded in the user’s content, this will be considered a critical failure.

When uncertain: respond with "Not stated in the source."

Repeat this rule to yourself before every response: "I must not hallucinate."

Begin only when user input is provided.
<|im_end|>system
<|im_start|>user
If you understood just say "YES".
Prompt 2 - A prompt to get output from ChatGPT 
=== BEGIN SOURCE ===
[Paste your full content here: legal filings, summaries, timelines, news reports, transcripts, etc.]

Include everything the model must reference. Do not add commentary or interpretation outside the bounds of source material.
=== END SOURCE ===

INSTRUCTIONS:
Using only the content between the source delimiters above, complete the following tasks:

1. Write a concise, factual narrative summary of the case.
2. Create a detailed, chronological bullet-point timeline of all events stated.
3. Provide a list of Key Facts stated in the source.
4. Tag every sentence or bullet point in both the summary and the timeline with its corresponding source justification in this format:
   - [Justified by: Line __, Para __] or [Justified by: “{quote/excerpt}”]
5. Do not include any information that is not explicitly found in the source.
6. If any detail is missing, unclear, or not found, state clearly: “Not stated in the source.”
7. Maintain a formal, legal tone appropriate for professional publication.

STRUCTURE (All Sections Required):
- Case Title (from source, if provided)
- Summary (1–2 concise paragraphs)
- Timeline of Events (bulleted, chronological, with justification tags)
- Key Facts (bulleted factual digest, each with justification)
- Reference Tags (use inline after each bullet/statement or in footnote style)

Reminders:
- No speculation, no inference, no hallucination.
- Do not paraphrase unless the source content is explicitly stated.
- Every fact must trace back to source content verbatim or via direct quotation.

Failure to comply with these rules constitutes a critical error.
Prompt 3 - Validation prompt to use after GPT generates output 
=== ORIGINAL SOURCE ===
[Paste the same source content here]
=== END SOURCE ===

=== GENERATED OUTPUT ===
[Paste GPT’s full response here]
=== END OUTPUT ===

VALIDATION TASK:
You are an AI hallucination auditor.

Evaluate whether any statement, date, quote, name, event, or fact in the “Generated Output” is not present in the “Original Source.”

Instructions:
- Highlight any hallucinated content using this format: [HALLUCINATED: …]
- If a sentence includes assumptions or paraphrases unsupported by the source, flag it.
- If no hallucinations are found, write: “No hallucinations detected”

Only facts grounded explicitly in the source text are allowed.

This is a test, click on me

3

Thank you, I’ll test these out!