Whisper not running on Nvidia GPU

Community
1

Hello!

I have been trying to use whisper. I have previously managed to get it to work on my Linux Laptop, but on this windows system it has not been working consistently.

I have had it spit out some text for a few minutes worth of audio but it gives me the following error. I have tried rebooting the PC, but not a lot else for fear of breaking anything.

I installed using a single command install from a youtube video that i am not allowed to link… If you search “One-Click Whisper Install” from youtube user “troublechute”

Graphics Card Nvida GTX 970
Processor	Intel(R) Core(TM) i7-5820K CPU @ 3.30GHz   3.30 GHz
Installed RAM	16.0 GB (15.9 GB usable)
Windows 10 (SP unknown)

Traceback (most recent call last):
  File "C:\Python310\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Python310\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\Python310\Scripts\whisper.exe\_main_.py", line 7, in <module>
  File "C:\Python310\lib\site-packages\whisper\transcribe.py", line 437, in cli
    result = transcribe(model, audio_path, temperature=temperature, **args)
  File "C:\Python310\lib\site-packages\whisper\transcribe.py", line 229, in transcribe
    result: DecodingResult = decode_with_fallback(mel_segment)
  File "C:\Python310\lib\site-packages\whisper\transcribe.py", line 164, in decode_with_fallback
    decode_result = model.decode(segment, options)
  File "C:\Python310\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Python310\lib\site-packages\whisper\decoding.py", line 811, in decode
    result = DecodingTask(model, options).run(mel)
  File "C:\Python310\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Python310\lib\site-packages\whisper\decoding.py", line 724, in run
    tokens, sum_logprobs, no_speech_probs = self._main_loop(audio_features, tokens)
  File "C:\Python310\lib\site-packages\whisper\decoding.py", line 686, in _main_loop
    logit_filter.apply(logits, tokens)
  File "C:\Python310\lib\site-packages\whisper\decoding.py", line 423, in apply
    logits[:, self.tokenizer.encode(" ") + [self.tokenizer.eot]] = -np.inf
RuntimeError: CUDA error: the launch timed out and was terminated
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

Sorry if this does not provide as much info as you will need, I am doing this for a friend who’s PC I do not currently have access to and also I am quite new to all of this! If you would like to know any system info I can get them to punch a command into powershell/cmd and pass on the output for me to post up.

Screenshot as well showing that SOME transcription was successful. Detials blurred out as its confidential.

p whisper issues
p whisper issues1600×900 174 KB

3

The key error message here is:

RuntimeError: CUDA error: the launch timed out and was terminated

That indicates a timeout error when trying to run a CUDA operation on the GPU. You can try updating the nvidia drivers, or maybe try setting set CUDA_LAUNCH_BLOCKING=1 in your environment.

1 Like
4

Hello,

Thank you for the rapid reply!

I gathered something like that with searching what I could…

What you mean by my environment? I’m running this by opening powershell in the folder where the audio file is. The drivers are up do date insofar as I can tell.

5

yes, so in your powershell environment, or even in the windows enviroment settings, do the set cuda… command prior to executing your code.