Whisper API cannot read files correctly

API
42

Hi Daniel, I am facing the same issue. It’s working on simulator but not on iPhone where audio format is m4a. Please share if you fixed it.

43

Hi, I am also facing the same issue by using record library for flutter in iOS.

Here is my code for recording

      await record.start(
        path: path,
        encoder: AudioEncoder.aacLc, // by default
        bitRate: 128000, // by default
        samplingRate: 44100, // by default
      );

After that, I upload the file to Google drive to trigger a pipeline which invoke WhisperApi

final DriveApi driveApi = await _getDriveApi();

    File fileToUpload = File();
    fileToUpload.parents = [_googleDriveFolderId];
    fileToUpload.name = '${file.getFileName()}.m4a';
    fileToUpload.mimeType = 'audio/mp4';

    try {
      await driveApi.files.create(
        fileToUpload,
        uploadMedia: Media(file.openRead(), file.lengthSync()),
      );
    } catch (_) {
      print('Fail to upload to google drive');
    }

Then I get the same invalid file format issue. Can anyone please give some advice? Thank you

44

Dear All,
To overcome the iOS audio issue, I utilized the “mic-recorder-to-mp3” npm package. This package not only allows for seamless audio recording but also ensures compatibility across various platforms, including iOS.

As this record your microphone audio input and get an audio/mp3 file in the end so there will be no issue related to any file format.

Once you get the blog url, convert that in base64 by using below code:

useEffect(() => {
		if (blobURL != null) {
			fetch(blobURL)
				.then(response => response.arrayBuffer())
				.then(arrayBuffer => {
					const base64Data = btoa(new Uint8Array(arrayBuffer).reduce((data, byte) => data + String.fromCharCode(byte), ''));
					const rawData = `base64,${base64Data}`;
					fetchData(rawData);
				})
				.catch(error => console.error(error));
		}
	}, [blobURL]);

And then use your transcribe API to return then text from using above base64.

And there will no invalid file format issue in iOS :slight_smile:

1 Like
46

Thanks, the polyfill suggestion fixed my issue with using MediaRecorder audio in Vite.

47

Hi all, stumbled upon this in my googling and believe I was having the same Safari problem. I would get one word or empty responses from the api.

I can’t explain it, but when I pass in a time slice parameter to mediaRecorder.start(), then it Safari seems to produce files that work. This is with mimeType: ‘audio/mp4;’

mediaRecorder.start(1000);

I just figured this out, have not tested extensively. Definitely would be curious if anyone can explain lol.

3 Likes
48

I just wanted to point out that this is still an issue. I see it is marked with a solution but it is still a problem.

49

I am using this code, and it only fails to work on Safari on iPhones. I always get Invalid file format. Supported formats: ['m4a', 'mp3', 'webm', 'mp4', 'mpga', 'wav', 'mpeg']

        const audioBlob = new Blob(this.recordedChunks, { type: "audio/mp3" });
        const formData = new FormData();
        const file = new File([audioBlob], "audio.mp3", { type: "audio/mp3" });
        formData.append("file", file, "audio.mp3");
        formData.append("model", "whisper-1");
        axios
          .post("https://api.openai.com/v1/audio/transcriptions", formData, {
            headers: {
              Authorization: `Bearer ${import.meta.env.VITE_OPENAI_API_KEY}`,
              "Content-Type": "multipart/form-data"
            }
          })
          .then((response) => {
            const userResponse = response?.data?.text;

It also doesn’t work if I change these lines:

   const file = new File([audioBlob], "audio.mp3", { type: "audio/mp3" });
   formData.append("file", file, "audio.mp3");

to

   formData.append("file", audioBlob, "audio.mp3");
50

check my post above. curious if it works for you too.

1 Like
51

Hey, super old post, but I’m getting desperate while struggling with this exact same problem right now.

I can’t figure out how to get the [audio-recorder-polyfill] code included in my sveltekit vite project.

I’ve tried the ‘quick hack’ suggesting of including

import AudioRecorder from 'https://cdn.jsdelivr.net/npm/audio-recorder-polyfill/index.js'
window.MediaRecorder = AudioRecorder

I’ve also tried downloading that file and putting it and the associated code in my /lib folder, but no go.

I’m beginning to think it’s something very specific to my Sveltekit preprocessor, but I don’t have enough expertise to figure out what’s causing the issue. The error message is extremely unhelpful.

My code in my front end looks like

onMount(() =>{
		import MediaRecorder from '../../lib/audio-recorder-polyfill//index.js'
		window.MediaRecorder = MediaRecorder
	})

And the error is popping up as

[plugin:vite-plugin-svelte] Error while preprocessing /home/jared/nomnomnom/nomnomnom/src/routes/home/+page.svelte - Transform failed with 1 error:
/home/jared/nomnomnom/nomnomnom/src/routes/home/+page.svelte:12:9: ERROR: Unexpected "MediaRecorder"
/home/jared/nomnomnom/nomnomnom/src/routes/home/+page.svelte
Unexpected "MediaRecorder"
 10 |  
 11 |  	onMount(() =>{
 12 |  		import MediaRecorder from '../../lib/audio-recorder-polyfill//index.js'
    |           ^
 13 |  		window.MediaRecorder = MediaRecorder
 14 |  	})
Click outside

I have a project that was working perfectly in desktop browser and it’s now falling apart on mobile because of the mp4 → whisper problem.

52

Replying to my own post in case this helps anyone. After hours of poking at this I finally got it. In SvelteKit you either need to disable SSR for your page, or you can load this module in an onMount block

	onMount(async () =>{
		const {default:AudioRecorder}  = await import('audio-recorder-polyfill')
		 window.MediaRecorder = AudioRecorder
	})

I’m finally able to submit microphone recordings for transcription from IOS!

1 Like
53

mediaRecorder.start(1000);

This fixed it for me, after weeks of trying other fixes! Thanks so much.