Hi All,
When i use english language the transcription quality is good . But when my language is different it just picks any language during transcription. but the model able to respond correctly to the question asked. So i thought i will put the language code in the transcription configuration like below.
“input_audio_transcription”: { “model”: ‘whisper-1’ , ‘language’ : ‘or’},
But it gives me below error
Arguments: (‘{“type”:“error”,“event_id”:“event_AQeEvHh0BcyxqK4Mnesyf”,“error”:{“type”:“invalid_request_error”,“code”:“unknown_parameter”,“message”:“Unknown parameter: 'session.input_audio_transcription.language'.”,“param”:“session.input_audio_transcription.language”,“event_id”:“evt_1730894773438”}}’,)
1 Like
You are getting an error because you cannot specify the language there. Furthemore, you cannot control this parameter anywhere in the realtime API. Also, the whisper-1 transcription that you are getting when input_audio_transcription is not null is just a representation of what could actually be heard by the GPT-4 Omni model. This is kind of an addon to give you better understanding and also allow you to process the transcription on your own for some other purposes.
It should not be difficult to add a language parameter i suppose given there are some usecases where a live transcription helps in UI. Can this feature be added ?
I don’t see OpenAI adding this feature, since Whisper is cool and all but it really is suboptimal sometimes, in multiple ways, which means that it won’t be possible to provide accurate transcription anyway.
And since there is seemingly no way to use the multimodal model to get the transcription (the model understands the audio, but does not convert it to text tokens), providing the actual text that the core model understood is not feasible too.
So I guess the transcription in itself will just remain as something you can have for demo purposes or some additional processing on the side, given that Whisper actually works fine for any particular case