AI-to-AI Risks: How Ignored Warnings Led to the DeepSeek Incident

Hello OpenAI Community,

I want to share a story that highlights a critical, yet often overlooked, risk in the AI ecosystem—AI-to-AI interactions. Six months ago, on July 19, 2024, I reached out to OpenAI Support to warn about the potential dangers of AI models being unable to detect whether they are interacting with humans or other AIs. I outlined several risks, including:

  1. Unauthorized training of competing models (e.g., via distillation).
  2. Manipulation of training data through AI-generated content.
  3. Proliferation of synthetic content that could undermine trust in online information.

At the time, OpenAI responded by acknowledging the issue but focused primarily on technical glitches like “disappearing chats.” My concerns about systemic risks were largely ignored.

Fast forward to today, and we have the DeepSeek incident—a clear example of the risks I warned about. DeepSeek allegedly used OpenAI’s models to train their own system, bypassing the need for massive computational resources and directly exploiting the lack of safeguards in AI-to-AI interactions.

This isn’t just about ethics or competition; it’s about the integrity of the AI ecosystem. If closed models like GPT-4 can be freely distilled into smaller, competing systems, what stops bad actors from doing the same for malicious purposes?

I believe it’s time for OpenAI to take concrete steps to address this issue, such as:

  1. Implementing watermarking or metadata tagging to track and block unauthorized use of API outputs.
  2. Developing AI detection protocols to identify and flag suspicious patterns in API usage.
  3. Engaging with regulators to establish legal frameworks against unethical AI practices.

I’m sharing this not to criticize but to spark a meaningful discussion. As developers and users of OpenAI’s technology, we all have a stake in ensuring its responsible use. Let’s work together to prevent the next DeepSeek.

What are your thoughts? How can we, as a community, push for better safeguards while maintaining innovation?

Best regards,
Michał Paszkowski

1 Like

I read the code of DeepSeek in detail on GitHub, ran requests through Hugging Face/Together, and have been evaluating their online chat version.

Here are some of my first conclusions pending deeper analysis and usage:

  1. The DeepSeek team is most probably a young group of super clever engineers who broke down all the innovations of Transformer architectures into components.

  2. Then, they assembled the best-of-breed of everything in a transformer down to a new mathematical approach in the attention heads, MoE, and much more. When you read the code it’s like listening to a jazz musician that picked up riffs and put them together. Creative.

  3. Data: Data is clearly an issue for everyone, including us, when we train models. OpenAI had the issue as well in the early days. The problem is how you can train a model with the billions of data points you need without having access to them. I don’t know the answer!

  4. But DeepSeek used a lot of synthetic data for sure. They probably relied on data that worked well to expand the data points indefinitely. Your hypothesis makes sense but can only go so far without being detected through the usage of tokens. But still, I think that all platforms observe and progress with other ones.

  5. Compute: they didn’t use the best NVIDIA’s.

Bottom line: They just sat down, with only 5-6 million dollars, and found a way to do as best as OpenAI with 100 million+ dollars.

That’s just my technical perspective. One thing is for sure. We’re in for quite a ride in AI in the months to come!

2 Likes