You can line up the times and cut the moments when the VAD is off. In post processing.
For live transcripts you’d need to use a buffer that records only when the VAD is active
Thanks @anon10827405 for your input. At the moment, I am using VAD to esclude audio clips where there is no voice activity at all. Because I am working on a live environment, those clips can be just skipped, so the hallucination issue doesn’t get that bad. What really improved things for me was to use prompts. I am currently using the transcript results of the previous 30 seconds of audio.
Curiously enough, it seems like hallucinations can be triggered with a sort of prompt injection, whenever the same word is repeated in the audio over and over again. For instance, 4 or 5 NOs in the input audio will result in tons of NOs in the output transcript.
from my experience, this is caused by poor quality microphone input usually.
It is still happening on 08/May/2024. Has anyone solved this issue? Should I remove all the silence from the mp3 file?
My apologies, I feel like I was addressing a different issue. I was experiencing hallucinations in silence. I am going to check the ‘no_speech_prob’ attribute, and I believe it will help me. 
Can someone please explain how the ‘no_speech_prob’ attribute is incorporated into code like this:
from openai import OpenAI
client = OpenAI()
audio_file= open("/path/to/file/audio.mp3", "rb")
transcription = client.audio.transcriptions.create(
model="whisper-1",
file=audio_file
)
print(transcription.text)
Thanks…
Late addition for anyone finding this thread through search — this is still
a live problem in 2026.
Re the original two approaches, both are worth separating out:
Asking a model to flag suspicious transcriptions stays unreliable, and the
reason is structural: these hallucinations are grammatically clean and
contextually plausible, so there is nothing for the model to catch. Someone
recently posted measurements from 13k+ dictations where an LLM cleanup pass
removed 0 out of 172 known bad tails.
“Is there a more intelligent way than checking 2, 3, 4 repeated words?”
— yes, n-gram counting. Build every 4-word window in the text, count them, and
flag windows that make up more than ~5% of all windows. One pass, no need to
try each length separately.
One caveat that cost me a debugging session: this silently returns nothing for
Chinese, Japanese and Korean. Without spaces, splitting on whitespace turns an
entire line into a single token, so there are no 4-grams to count. Character
n-grams (6-8 chars) work there instead.
The phrase-tail half is a different problem and easier — those come from a
closed set (“Thank you for watching!”, “Subtitles by the Amara org community”,
“Sous-titres par …”, “Продолжение следует…”). Plain string matching against
a known list beats any model. There is a multilingual phrase list published on
Hugging Face under sachaarbonel / whisper-hallucinations — worth filtering out
the common words in it first (“you”, “the”, “bye”) or ordinary speech gets
flagged.