Whisper hallucination - how to recognize and solve?

[anon10827405]

@vb

I am running whisper locally and never experienced hallucinations aside from the cases when there were microphone issues.

I use the API and never have ran into issues either. Even with Spanish. I have been running it on my phone using Silero VAD and only experience hallucinations when maybe a single word or two is accidentally caught.

@nikola1jankovic

However, here, the text I posted, is much longer. Instead of correct transcription, it just repeats these same words and phrases. It is not that there is only one sentence - there are sentences and sentences replaced by repeated babbling.

Strange. Whisper actively tries to prevent this exact issue using Beam Search and by using a dynamic temperature setting (if you have set it to 0). Whisper has a ~13% error rate with Croation.

So, three questions:

1. Are you using a prompt to prime the transcription process?
2. What is your temperature setting?
3. How are you starting the audio? You said it’s live. How are you capturing the audio?