Current voice interaction already allows us to speak naturally with ChatGPT. But there is a layer of human communication that is largely lost when speech is reduced to text: how something is said.
Tone, rhythm, pauses, hesitation, emphasis, laughter, changes in vocal intensity and other prosodic signals can carry information that the words themselves do not contain.
I would love to see a future version of ChatGPT capable of using these signals directly during a voice conversation, rather than relying primarily on the transcribed words.
The goal would not be to claim that an AI can “read emotions” perfectly. That would introduce another class of errors. Instead, the system could treat vocal characteristics as additional contextual information and combine them with the linguistic content and conversation history.
For example, the sentence: “Va bene.” can mean agreement, resignation, irritation, amusement or sarcasm depending on how it is spoken.
The transcript may be identical. The interaction is not.
For users, this could make conversations considerably more natural and better calibrated. It could also have applications well beyond casual conversation: education, accessibility, tutoring, training, rehabilitation and human-AI interaction research.
I have been experimenting with voice interaction extensively, and this is one of the clearest limitations I have noticed: the difference between understanding what a person says and understanding how they communicate it.
I don’t know whether systems capable of doing this are already being tested internally, and I don’t want to speculate about undisclosed research. But I would strongly encourage OpenAI to explore this direction.
Not simply speech-to-text → language understanding → response, but a richer interaction model in which voice itself becomes part of the context. That could be a very significant next step for conversational AI.
What do you think? Would you find this useful in your own interactions with ChatGPT?
chat