ChatGPT (GPT-4o) giving completely nonsensical responses, or am I getting responses intended for other users?

[turbolucius]

So I’m helping out a friend clean up audio transcriptions for a school project he’s working on. There’s LOTS of text (in this case, about ~6.6k tokens), and I’m sending a bunch of it to GPT-4o so that it cleans it up. Rephrasing some sentences, correcting wrongly transcribed words, all that jazz.

I’ve done it a first time through the API, it worked perfectly well. I then did exactly the same through ChatGPT and I got some absolutely nonsensical answers that I never got before. It looked like I was getting other people’s generations instead of my own?

Note that my prompt and the text were both in French… then ChatGPT replied with a completely irrelevant answer in English:

(Note: the original text or prompt did not contain any mention of crime in England or Wales. It did not have ANYTHING to do with this whatsoever.)

Confused, I asked ChatGPT why it suddenly brought this up, and it then changed topic again, entirely, answering a question about… The Witcher series?

(The model is basically telling me about a fire that destroyed The Witcher costumes and delayed the show’s production. I can’t find anything about this online, looks like it never happened.)

I kept asking it about it, and it replied to question about PDF I did not ask.

(Telling me how to reduce PDF filesize.)

On top of all this, the conversation title that got generated for this one is just…

Any idea why this is happening? Is it just completely thrown out by the long text?

EDIT: I’ve started more conversations with the exact same prompt and the same bug happened every time. I can’t tell if the model is just hallucinating or if I’m somehow getting other people’s generated responses instead of mine. So far I’ve had the AI ignore my prompt and instead:

• Perform a web search about “The 2024 Canadian Census Test”
• Generate a graph about “College Admissions by Ethnicity and Status”
• The November 5, 2024 US Election
• The Michigan GOP primary of August 2, 2022
• Analyse a “covid_countries_cases.csv” file I never uploaded
• Explain how to resolve a quadratic equation, try to use code interpreter, then say “Sorry, I don’t have information about the results of that election.”

[charliet1802]

The same thing is happening to me. In my case I’m a web developer and use ChatGPT to help me out sometimes with bugs, ideas and refactoring code, but since today it started to answer in a weird way and in wrong format so the code that it outputs is impossible to read. Hadn’t happened to me before, I tried other models of GPT-4 and they all did the same. It’s really odd.

[anon85997880]

I am having issues with GPT-4o and garbage outputs. See: https://community.openai.com/t/a-post-in-python-api-gpt-4o-setting-choices-1-results-in-garbage-text-output-requires-staff-attention/863280

[turbolucius]

@charliet1802 Do you have examples of what’s happening to you? In my case, ChatGPT isn’t just giving me bad responses, but outright ignoring my request and replying as if I’d said something else entirely.

@anon85997880 The link you sent isn’t working on my end

Either way, this is still happening. Always with the same prompt. I still can’t tell if it’s hallucinating or somehow a backend issue where it’s giving me responses intended for other users.

[charliet1802]

@turbolucius It just started working normal again, but it was indeed giving me nonsense responses besides formatting them badly. It’s no longer hallucinating. Does it keep malfunctioning on your end?

[turbolucius]

Just tried again, and yes, it keeps malfunctioning on my end. Still with the same prompt. I’m just asking it to clean up a big paragraph and this time it replied talking about… votes in Florida during the 2000 U.S. Presidential Election?

Every time I send the prompt, it’s as if it’s replying to something else entirely. Tried a couple times, it started talking about:

• Housing market trends on economic inequality in the United States
• The 2024 Connecticut Democratic primary
• The Venezuelan presidential election in 2024

(Again, none of these topics are mentioned in my prompt or relevant at all)

Sometimes it even starts looking stuff up or trying to use the python code interpreter.

I’m so confused.

[charliet1802]

I’m sorry to hear that, have you tried on the website using another account? Just so you make sure it’s just hallucinating in general and it’s not an issue with your account

[JUNIQUE]

HI There, I have since yesterday the same issue that you explained. Giving weird answers and mixing up answers with older content. What did you do in the end?

[loren1]

Same here. It’s been really bad on my end. I use one chat to log and track my food macros and it gives me a weeks worth of work outs.

[_j]

The most recent reports bumping this year-old topic discuss a new symptom from the prior purpose of this forum topic:

• referring back to obsolete information, or topic confusion.

Likely issue:

• An AI that is fed cross-session “memory” about you that it previously stored
• An AI that is fed some more information retrieved from searching old chats

Solution:

Disable the newer memory “features” that confuse an AI.

• In ChatGPT, pick your account menu, then “Settings”
• Under “Personalization”, disable the toggle of both memories

While memory is still enabled, you can “manage memories” and see what the AI has been storing about you with its memory tool, and also delete those.

Result

An AI model that remains focused on the topic of the chat session, using only your immediate conversation.