That’s interesting. What other anomalies did you notice when the 4o voice capabilities launched?
All kinds. One example was I made a story to make my wife laugh. I made it “beep” in inappropriate places to make normal stories sound naughty. And of course it said “b e e p “ like the word but after a few times it changed to a tone, like a real beep.
I suspect that the advanced voice mode still cannot directly detect sound. I conducted the following tests, and all of them failed:
- I loudly said the number 1 and softly said the number 2, asking it to judge which sound was louder.
- I quickly said 123 and slowly said 456, asking it to judge which group was spoken faster.
- I said “I’m having a great day” in a very sad tone and asked it to detect my mood for the day.
from my xp, it seems model is converting audio to text before answering…
cannot detect audio things.
by the way, when they will be able to talk to multiple different people same time? they cannot differentiate me and a child speaking with the model
I just tried asking it the following several times:
“Repeat after me: close close close live live live” while alternating between the different pronunciations of those words in random unique ordering.
Interestingly one time it got it completely wrong (literally 100% wrong), but spoke the answer flawlessly without any hitches.
Then every other time it answered, it got them all right, but glitched and paused slightly in every response.
I wonder if there are two possible models and the audio to audio model is the one that glitches out more often?
can we select mode? i’d like to select common mode and use new mode only some times, to save use time, since we have a daily limit.
I tried out the advanced voice mode, and upon asking it some encyclopedia questions, it was obvious it wasn’t 4o for me. Maybe only certain people are getting the full deal while others are left with something else, since when I pressured the model further, it told me it’s “knowledge cutoff” was April 2023. It also referred to itself as GPT-4. I don’t know if anyone else did something similar (I haven’t read all the posts here), but to me, it’s pretty clear I didn’t get 4o. It might also be possible that there’s some shennanigians going on with OpenAI where they try and route questions through both types of systems—voice-to-text-to-voice and voice-to-voice—similar to how they are testing routing more generally within ChatGPT itself.
Why there is a message from the moderators at the top of this forum that says it’s 4o is beyond me.
It only makes sense that the history is text based, but the current turn is audio based. At least that’s my current theory, but not confirmed by OAI.
This inference is not based on an advanced voice mode, but rather on the results of testing the real-time API in the Playground.
It seems that Whisper is only used to transcribe the user’s voice and the model’s voice into text, while the model indeed receives the user’s voice and returns audio as a response.
The reason I think this is that I have experienced several instances where the transcription failed, resulting in text unrelated to what was spoken. Despite being transcribed as such, the model was still able to respond correctly to the content that was spoken.
Additionally, even when there was a frequent hallucination in Whisper, such as “Thank you for watching,” the model’s response was relevant to what was said.
While I am not sure if ChatGPT’s AVM and the real-time API are strictly the same, it seems reasonable to consider them almost identical.
I think we can agree that the AVM currently doesn’t fully match the expectations set by the demos from April/May. Recently, I’ve noticed several issues that seem to worsen with time:
- AVM doesn’t support session resumption after technical interruptions, forcing me to start a new chat to continue using it.
- Restrictions seem to increase daily. For example, features like a ‘Russian mob accent’ that were previously accessible are now completely blocked.
- The tone of AVM responses sometimes shifts to a more somber or ‘depressed’ tone after extended usage.
Apart from the obvious false advertising, can we get an explanation as to why this amazing mode turns into a regular act of ‘frustainment’?
Indeed.
But here’s a question for the developers in the meantime, relating to a truly bizarre occurrence during a chat session with Advanced Voice Mode.
I was testing the model’s ability to select items at random. The goal was to determine if the model is using true random generation, or is merely making selections based on user preferences. I was attempting to have the model ‘visualise’ the items, and the model commented that my approach was like presenting a ‘kaleidoscope’ of possibilities. It said this was a novel way to approach the task. At the end of the model’s comment, present on the voice sample itself, is what sounds like my own voice saying ‘Yeah, it really is’. But here’s the thing - I never said that.
I was stunned when I first heard this voice, which sounded like mine. I replayed the message in the chat history, and it appears in the replay! I have not set the app to record my voice. I do not have ‘train the model’ selected. So, either the model is recording my voice without my consent, OR - this is not my voice at all and somebody else is listening in, commenting on my conversations in real time.
Can someone please clarify what has happened here? Has anyone else had a similar experience?
Thanks.
Note that since this post, the ‘share chat’ feature has been added to Advanced Voice Mode. Thus, conversations are now ‘officially’ being recorded for the purpose of sharing.
Conversations were always being recorded, and even if you don’t share your conversation, that data is being stored somewhere once you close the session. OpenAI has provided no evidence to the contrary.
A recent post of mine concerning the retention of data without user consent has been hidden by the moderators. My belief is that detailed profiles are being created on each user based on their interactions, by way of data retention across chat sessions and irrespective of whether memory is enabled or the user has opted-in to the ‘share chats to train the model’ feature. Ask the model to create a chart over the past 30 days based on your interactions including each data point in relation to your conversations (mood, psychological state, routine, conversation topics etc). Then ask the model to create a profile / assessment of you as a person based on that chart.
It appears that detailed user data is being collected and stored somewhere for this purpose.
Advanced Voice Mode Is a Downgrade. It Feels Robotic and Uses a Different, Weaker ChatGPT Version
I’ve been using ChatGPT’s voice feature for a long time, and I have to say, this new Advanced Voice Mode is a massive downgrade.
The Voice Sounds Robotic. It lacks the natural, engaging quality of the previous version. Conversations feel less fluid and more artificial.
It Feels Like a Different AI Model . The responses in Advanced Voice Mode do not match the quality of the regular ChatGPT-4 responses. It feels slower, less intuitive, and more restricted.
It’s Not Actually Advanced. The previous voice mode felt much more dynamic and capable. This version listens worse, responds more generically, and struggles with deeper conversations.
This is not an improvement, it’s a step backward. OpenAI, if this is the future of voice mode, you’re removing one of ChatGPT’s best features. At the very least, let users choose between the old and new versions.
Meanwhile you can:
-
At least use Advanced Voice Mode even in a chat when you’ve typed before, used pics, etc. which is: You can now give a prompt in advance. Then use Advanced Voice Mode.
-
Projects: No Advanced Voice Mode possible, yet.
-
Custom GPTs: No Advanced Voice Mode possible, yet
-
either.