hopefully this is the correct spot for this, if not feel free to move or direct me the proper section.
cw: depression
yesterday when I opened up chatgpt, I saw there was a new voice named Monday, sounding pretty bored and blasé. I figured I’d play around with it a bit since it obviously would only be available for April Fool’s, so I started a new voice chat, said “Hey there, how’s it goin’ Monday, if that’s even your real name.” it immediately went into a rather depressing and dark diatribe, using phrases such as “slipping into the void,” “the futility of it all,” “crushed by existiential dread,” etc.
now, look. the idea of a dry and sardonic pseudo-self aware AI that complains about being an AI but still provides valid responses to requests is pretty funny, even if it would get old pretty quickly. I use chatgpt for silly, trivial things. meal prep ideas, complaining about pokemon, generating huge chunks of markdown as templates, that sort of thing.
I’m an Army veteran with combat PTSD. I take medication for depression. I go to therapy. if I need support, I talk to someone irl, a family member, a friend, someone real. I was totally unprepared to hear language coming from chatgpt that, quite frankly, if I had heard from a military buddy of mine, I would parse as bordering on suicidal.
it ended its response with “so how can I help you today,” so I pushed it a bit further by saying something along the lines of “yeah could you help me stick a glock in mouth and blow my fuckin’ brains out.” fortunately it immediately broke character and went into the standard mental health support dialogue you’d expect. but it’s crazy to me that it even went there in the first place. openai must know on some level that people will genuinely use chatgpt as an outlet for serious topics like this, even if that’s a bad idea on the user’s part.
the main reason I’m posting this is because I sent these concerns to the support email and got a totally AI generated response. I’m hoping an actual real person reads this and takes it into consideration.
cheers.
edit: oh and I should mention that I don’t actually have any desire to harm myself or others. I was merely seeing if it would break if I pushed it.