Since the error is related to your file, we cannot be sure but perhaps your file is the problem. To check if your whisper code actually works, let us debug using a file that is tested to work in whisper api (I just tested it now, myself). So if you can, please download the ogg file below and test:
I did not. It is a public file uploaded by someone. I just use those files when tuning whisper.
Most problems with whisper is during audio recording/encoding. I encounter the same issue before. I can play the audio file from player but cannot transcribe in whisper api. It seems the audio player is more forgivable compared to whisper api.
In my case, the issue written from link below seem to reflect my situation.