I want to write a serious follow-up feature request for OpenAI about native perceptual audio understanding in ChatGPT.
I have already submitted a previous request asking for ChatGPT to understand uploaded audio perceptually—not merely accept audio files, transcribe speech, or perform technical signal analysis.
This new request should NOT simply repeat the previous proposal. I want it to focus specifically on the missing link in a real creative multimodal workflow.
The central problem is this:
ChatGPT can be an excellent creative collaborator while I am developing a project. It can understand images, discuss visual composition, lighting, camera movement, character design, storytelling, dialogue, and emotional intent.
But when the project reaches its audio layer, that collaboration suddenly breaks.
I can upload an audio file. ChatGPT can know that the file exists. Depending on available tools, audio may potentially be transcribed or technically analyzed.
But if I simply ask:
“What are you hearing?”
the assistant cannot truthfully listen to the file and discuss the actual perceptual experience of it with me.
For a creative professional, transcription is not enough.
I need the assistant to perceive and discuss things such as:
-
The personality and character of a voice
-
Whether a performance sounds confident, shy, playful, serious, warm, restrained, nervous, or emotionally convincing
-
Vocal pacing, pauses, emphasis, rhythm, and delivery
-
Whether two generated voices actually sound like the same character
-
Musical mood and emotional progression
-
Instrumentation and arrangement
-
Changes in energy and dynamics
-
Builds, drops, transitions, and dramatic timing
-
Whether music supports or competes with dialogue
-
Whether a soundtrack fits the visual language of a scene
-
Sound design and its relationship to storytelling
-
The relationship between voice, music, visuals, and narrative context
A practical example:
I may develop a character with ChatGPT, write her dialogue with ChatGPT, generate several possible voices for her, and then upload those performances for evaluation.
At that point I do not only need:
“This file contains speech.”
And I do not only need a transcript of what she says.
I need to be able to ask:
“Does this actually sound like her?”
That question requires perceptual listening.
The same applies to music.
If we develop a scene together and I upload two possible soundtracks, I want to discuss which one emotionally belongs to that scene, where the music changes, whether its movement matches the editing, and what I am actually hearing—not merely metadata or an analysis of frequencies.
This creates an important asymmetry in multimodal creative collaboration:
ChatGPT can see the frame, understand the story, help direct the scene, and evaluate the visual performance.
Why should that collaboration become effectively deaf the moment I press Play?
Native perceptual audio would complete a workflow that already exists:
Character Design → Dialogue → Voice Generation → Performance Evaluation → Scene Creation → Music Selection → Sound Design → Final Creative Review
The assistant already participates meaningfully in most of this chain. Audio perception is the missing connection.
Please make a clear distinction in the feature request between:
-
Audio file handling
-
Speech transcription
-
Technical audio/signal analysis
-
Native perceptual listening and multimodal reasoning
The fourth capability is what I am requesting.
The goal is not simply for ChatGPT to process audio.
The goal is for a user and their AI creative collaborator to be able to listen to the same creative material and meaningfully discuss what they are hearing.
Please write this as a thoughtful, technically grounded, professional feature request suitable for the OpenAI Developer Community.
Avoid exaggerated claims about what current ChatGPT audio capabilities can or cannot do universally; frame the limitation as the experience/workflow I am encountering with uploaded audio in ChatGPT.
Avoid demanding language. Make it constructive and specific, explain why this would materially improve professional creative workflows, and emphasize that this is about genuine multimodal understanding rather than another transcription feature.
Include a concise title and a strong closing paragraph.
The central message I want the request to preserve is:
“I don’t need ChatGPT to tell me that my audio file contains frequencies. I need my creative partner to hear what I hear.”
Please, it’s a mandatory feature needed for my personal AI assistant.
Thanks, with all my respects.