Text to speech audio format support in Firefox

API
1

TLDR: Is it possible to stream text-to-speech in Firefox using Media Source Extension API?

Here’s my conclusion so far: Openai supports 5 types of response_format: “mp3”, “opus”, “aac”, “flac”, and “pcm”. From my research it seems that Media source extension API in firefox supports “opus” codec both in webm and mp4 containers, but it does not support it in “ogg” container. Other formats that openai supports are not supported in FF.

I’m not sure what’s the “opus” response format in openai, but I tried using it in both mp4 and webm containers in firefox, it didn’t work.

I’ve found similar topic here: community openai com /t/read-aloud-feature-not-working-on-chatgpt-via-firefox/664947/12 , but looks like there’s no solution for audio streaming, and I tried updating firefox, also didn’t help.

Is there any solution for this, or a workaround? Any help will be much appreciated.
I’ll gladly provide more details if needed.
Thanks

2

You can remux the opus audio into mp4 by using the “copy stream” method of ffmpeg.

ffmpeg -i tts-1_onyx_20240425_015848.opus -c:a copy test.mp4

performed on a tts.response_format = “opus” I just obtained from the API:

[ogg @ 000001edf72447c0] 693 bytes of comment header remain
Input #0, ogg, from ‘opustest_tts-1_onyx_20240425_015848.opus’:
Duration: 00:00:21.93, start: 0.000000, bitrate: 40 kb/s
Stream #0:0: Audio: opus, 48000 Hz, mono, fltp
Metadata:
ENCODER : opusenc from opus-tools 0.1.10
File ‘test.mp4’ already exists. Overwrite? [y/N] y
Stream mapping:
Stream #0:0 → #0:0 (copy)
[mp4 @ 000001edf7241c00] track 0: codec frame size is not set
Output #0, mp4, to ‘test.mp4’:
Metadata:
encoder : Lavf61.1.100
Stream #0:0: Audio: opus (Opus / 0x7375704F), 48000 Hz, mono, fltp
Metadata:
ENCODER : opusenc from opus-tools 0.1.10
Press [q] to stop, [?] for help
[out#0/mp4 @ 000001edf728f6c0] video:0KiB audio:105KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: 4.811867%
size= 110KiB time=00:00:21.92 bitrate= 41.2kbits/s speed=5.81e+03x

For the particular browser install, check the about:support of Firefox. See if Opus is supported by the libraries and the build. If this is a possible issue (for example, some enterprise hat Linux) , you might want fallback re-encoders.

3

Thanks, I’ll try it in the next couple of days.