Transcription errors in realtime API

API
1

I’m using the realtime api where I have set the input_audio_transcription field as I require the transcriptions from user audio as well. I’m facing issue where the transcriptions returned by the api are incorrect sometimes however, the model seems to get the correct context and generate correct responses. The transcriptions frequently switch languages even though I’m using English only. I have tried specifying other models from transcriptions as well - whisper, gpt-4-mini-transcribe but still same issue.

My question is how do I improve the transcriptio9n accuracy? I should atleast have the option to set the desired language for transcribing to english to enforce the api to generate english transcriptions only.

2

AFAIK the transcription should be treated as guidance of input audio content rather than precisely what the model heard. You have additional instructions for the transcription mode, that might help.

"transcription": {
	"model": "gpt-4o-transcribe",
	"language": "en",
	"prompt": "Transcribe speech accurately as spoken. Do not translate or summarize."
},
2 Likes
3

Where do I specify these in the realtime API? I can only see input_audio_transcription which lets me sepcify only the model name.

4

I believe input_audio_transcription is deprecated and/or beta API. For latest updates see: https://platform.openai.com/docs/api-reference/realtime-sessions/create-realtime-client-secret#realtime_sessions_create_realtime_client_secret-session-realtime_session_configuration-audio-input-transcription