Hi everyone,
We are building a professional real-time interpretation platform for conferences and live events.
Our requirements are:
- Very low latency (simultaneous interpretation)
- Natural and fluent speech output
- One-way real-time speech translation for conference audiences
- Support for a custom glossary (customer-specific terminology)
- Ability to update the glossary per event/session
We initially tested gpt-realtime-translate, and the translation quality and latency are excellent. However, since it currently doesn’t support system instructions or glossary enforcement, it doesn’t meet our terminology requirements.
We then started testing gpt-realtime-2.1 with reasoning and a system prompt containing our glossary. While the glossary works much better, we’re experiencing several issues:
- The translation is less fluent than
gpt-realtime-translate.
- The model often waits before speaking instead of translating incrementally.
- We experience pauses and occasional interruptions during long speech.
- Overall, it behaves more like a conversational assistant than a simultaneous interpreter.
Our goal is to build something that behaves like a human interpreter, continuously translating while the speaker is talking.
So we’d like to ask:
- Is
gpt-realtime-2.1 the recommended model for this type of application?
- Are there any best practices for configuring sessions (VAD, buffering, prompting, reasoning, etc.) for simultaneous interpretation?
- Is there a recommended way to enforce a glossary while maintaining low latency?
- Is glossary support planned for
gpt-realtime-translate, or is using gpt-realtime-2.1 currently the intended approach for this use case?
Any guidance on the recommended architecture or implementation patterns would be greatly appreciated.
Thank you!
Hey @victor, welcome to the forum!
`gpt-realtime-translate` is purpose-built for streaming speech translation and can return translated speech and transcript deltas while the source audio is still arriving.
With general Realtime sessions, turn detection typically generates a response after the end of a speech turn. Semantic VAD may wait longer to determine whether the speaker has finished, which can be useful for natural conversations but may add unwanted latency for simultaneous interpretation. Higher reasoning effort can also increase latency and may contribute to the delayed, assistant-like behavior you observed.
For lower latency with `gpt-realtime-2.1`, a few settings are worth testing:
- Use `server_vad` with a shorter `silence_duration_ms`
- Use Semantic VAD with `eagerness` set to `high`
- Reduce reasoning effort where possible
Keep in mind that a shorter silence duration may cause the model to respond during natural pauses. The ideal settings will depend on your audio environment and language pair, so I would treat these as experiments rather than an officially recommended architecture.
I also wouldn’t describe `gpt-realtime-2.1` as the official replacement when glossary support is required. I don’t have confirmed information about planned glossary support for `gpt-realtime-translate` or an officially recommended glossary architecture, so those parts would require further product guidance.
Good luck building!
Avinash
An interesting thing I’ve noticed is that gpt-realtime-translate does accept gpt-live-transcribe as the input transcription model and that improves a bit the results.
However, it doesn’t accept the other parameters like keywords.
Perhaps this would be a potential improvement to gpt-realtime-translate ? It appears we just need a small tweak in the API to get there.
That's a useful observation, aprendendo.next.
I agree that event- or session-level terminology guidance would be valuable for professional interpretation workflows, and I've captured this as product feedback.
Avinash