Dazhon Darien, a high school athletic director in Baltimore, was arrested after allegedly using artificial intelligence to create a fake audio recording that portrayed the school’s principal, Eric Eisworth, making racist remarks. The forged audio, which suggested Eisworth disparaged Black and Jewish individuals, led to his temporary removal and incited widespread backlash and hate messages.
The incident unfolded after Eisworth had initiated an investigation into Darien for potential misuse of school funds. The investigation into the audio, involving local police and the FBI, traced the creation of the fake recording back to Darien, who reportedly used advanced AI tools and a paid OpenAI account to generate the content.
Darien was apprehended at an airport and faces charges including disrupting school operations and stalking. The school has begun disciplinary proceedings against Darien, which could lead to his termination, while also investigating other teachers involved in spreading the audio.
Detectives allege that Darien used his Large Language Models, such as OpenAI and Bing chat, to create the recording.
Yeah, I was hoping they’d say what software was used, but I doubt it was “Bing chat”?
Anyone know? I’ve not seen all articles, but I’m doubting anyone actually says… but they shouldn’t infer it was Bing chat maybe?
ETA:
Police said Thursday afternoon that detectives have conclusive evidence based on a forensic analysis by the FBI that the recording was not authentic. Police said the analysis indicated the recording contained traces of AI-generated content. Detectives then sought an additional analysis by the University of California, Berkley, which arrived at the same findings as the FBI’s analysis.
Also…
Police said the audio clip was originally sent to a friend group of three teachers. One of the educators admitted to police she sent the clip to news outlets and a student who she knew would spread the clip.
Would it make a difference whether the other three knew it was a fake or not? Did they have a responsibility to check it out? The case could lead to laws possibly in the future?
Ah, here we go…
Baltimore County State’s Attorney Scott Shellenberger said the case appears to one of the first of its kind nationwide that his office was able to find. He said the Maryland General Assembly may need to update state laws to catch up with the nefarious possibilities of the new technology.
Unless the principal sounds like the TTS-1 voices, I doubt he could use those to generate the audio. Bing chat has Suno plugin but it modifies the prompt.
Yeah, I’m guessing it’s lazy reporting and was actually something like Eleven Labs. Does the new OpenAI Voice (??) allow you to upload sample voices? If it was a principal, he could’ve got as little as ten seconds of his voice…
ETA: Would be a weird twist if the police used DraftOne and it hallucinated using Bing Chat… Heh…
I think it’s long past time to have a discussion as a society about how we navigate our new post-reality reality.
Photographs used to be evidence—still are but to really authenticate them requires a level of sophistication which is beyond the average viewer.
Video and audio recordings were even stronger evidence, but we are fast approaching (possibly even well past) the point where fake recordings will be impossible for the vast majority of humanity to identify.
The person from this story likely just took the audio straight out of whatever application they used to generate it—stupid and lazy.
If they had played it through a high quality speaker and recorded the playback on their phone in their pocket, I’m guessing it would have been much more difficult for even the FBI to debunk—probably not impossible, just more difficult.
What does it mean when we can’t trust what we see and hear anymore?
Yeah, I’m hearing of scammers using the tech too… Just a few stories so far, but it’s probably happening more than people realize? I don’t think Eleven Labs or others ever really said how they would stop bad actors other than an honor system… not to mention home-brew tools.
Yeah, this interests me. How tech-savvy was this person.
Clever!
A Brave New World indeed.
Did Gibson or others talk about any of this in early cyberpunk? I wonder…
ETA:
To address your question about whether early cyberpunk literature directly anticipated or discussed technologies like AI-generated audio or deepfakes: No, early cyberpunk writers like William Gibson did not specifically foresee or discuss the creation of AI-generated fake audio or video. Their focus was more on the broader implications of technology and cybernetics on society, identity, and reality, rather than on specific technologies that could generate realistic fake media.
The connection to deepfakes lies more in the thematic overlap—the idea of digital manipulation and its potential impact on society—rather than explicit discussions of technologies that mimic real human outputs like voice or video.
I thought this country had first amendment protections of vile speech…
And civil law procedures for slander, libel, defamation.
And that administration officials that didn’t perform diligence before dismissing someone would face their own consequences.
It is sufficient to say that journalists and cops are not as informed with their skript kiddie AI as the athletic director. And the FBI? Stop phone scammers and fake Sora apps doing federal crimes, feds.
Before audio and video recording there was still a system that workedish. We have been fortunate to go through a period of undeniable proof, but it’s coming to an end.
I feel like if I were to record someone saying something I should also be required to hand over the other details. GPS. Network. All that juicy log data. Then, it should be correlated with the suspect.
I’ve been doing a deep dive on electricity lately… interesting how far we’ve come since noticing static electricity in amber and lightning, of course…
Each big tech tied to electricity once we figured it out after many years of testing (early arguments were whether it was a one-way fluid or not haha…) … but every big tech like lights, telegraph, radio, television, radar, etc had a really big impact. Maybe transistors the most? Then computers…
And here we are on the verge of an alien intelligence…the biggest of them all…
ETA:
Oh, wow. Apparently it went on for months… since January… news reports ran, community leaders condemned the man/speech…
ETA2: The audio is not synced to the presser very well, making it feel fake…
For a sophisticated power user of this technology, I can confirm that unless you really specialize and dabble in this technology, the voice emulation can reach 90% similarity, with 10% artifacts, and you can cleanse these 10% artifacts if you understand how audio signals and frequency (detailed audio editing, mixing and mastering) work.
With the advancement of this technology, you can doctor up fake evidence and falsely accuse someone innocent for any crimes he doesn’t commit using your ‘fake evidence’
Use the doctored evidence and voice, and have a connection with the authority and you can actually get anything for free at the expense of someone innocent.
Things like this are why the guardrails are in place for generative AI. Most users don’t really seem to understand this.
They’re always complaining about the models being “nerfed” or saying things like, “I’m an adult, I should be able to have the model do [offensive thing] if I want to…”
The guardrails aren’t there to protect the users from the models, they’re there to protect society from bad users (and by extension to protect the companies from bad press and lawsuits).
I can guarantee when it comes out which platform was used to generate the audio that company is getting sued.
If I were to submit an audio recording of someone saying bad things I damn well should be required to upload ALL the metadata that comes along with it. Like, the whole damn phone should be handed over
My phone knows so dang much about me and the police should be able to pull the suspect’s phone’s data as well, and anyone else that may have been in the area.
I mean, privacy is basically gone. I was once accused of stealing some dumb shit like a battery pack and a bike from a garage once. It was recorded on CCTV some hooded tall dude with tats and some bad apples I use to hang out with tried to throw me under the bus (it was a Facebook group).
My savior? Google freaking tracks EVERYWHERE I GO!
These people who doctor/forge evidence should receive some serious jail time.
funny. Who else would do that kind of mischievous doings?