Sam,
I think AI itself is extraordinary.
My concern is humans.
More specifically, I think the current AI industry is repeating one of the oldest and most destructive patterns in human history: creating a powerful technology, recognizing the risks, and then allowing greed, competition, stock prices, market share, and short-term financial incentives to determine how it gets deployed.
We have seen this before.
We saw it with fossil fuels. Humanity understood the consequences for decades, yet profits, infrastructure, politics, and entrenched interests continually delayed meaningful change.
We saw it with the internet and social media. What could have become one of humanity’s greatest tools for knowledge and connection evolved into systems optimized for attention, outrage, addiction, manipulation, advertising revenue, and shareholder value.
Now we are doing it again with AI — except this technology may ultimately be far more powerful.
The fundamental problem is not that AI is unintelligent.
The problem is that humans are giving increasingly powerful AI systems authority before the systems are sufficiently reliable, while the institutions deploying them remain driven largely by short-term incentives.
LLMs are frequently wrong.
They hallucinate facts.
They misunderstand context.
They contradict themselves.
They produce confident answers when they should express uncertainty.
And yet the industry is racing to insert them into finance, healthcare, government, employment, education, customer service, infrastructure, military systems, decision-making, and practically every other part of society.
Why?
Because there is enormous money at stake.
That is precisely the problem.
The measure of whether AI should be deployed somewhere should not be whether it increases a company’s valuation, reduces headcount, increases engagement, or makes investors happy.
The question should be:
Does this technology actually produce a better outcome after accounting for errors, verification, unintended consequences, and the cost of cleaning up its mistakes?
If it does not, it should not be deployed there yet.
Humanity’s technological capability has advanced enormously.
Human wisdom has not advanced at remotely the same rate.
We can build nuclear weapons, alter the climate, manipulate billions of people through social platforms, engineer biology, automate financial systems, and now create increasingly autonomous artificial intelligence.
But our underlying motivations remain remarkably primitive.
Money.
Power.
Status.
Competition.
Control.
Fear.
National advantage.
Corporate advantage.
Short-term gain.
That mismatch may be the greatest danger surrounding AI.
Not AI becoming evil.
Humans using extraordinary intelligence to amplify the same behavior that has already created many of our largest problems.
The AI industry currently appears to believe that greater capability will solve most of these concerns.
I do not believe it will.
A more capable model placed inside a badly designed incentive system may simply become a more powerful amplifier of that system.
The industry needs a fundamentally different philosophy.
Reliability before scale.
Verification before autonomy.
Human accountability before delegation.
Transparency before trust.
Measured outcomes before mass deployment.
Long-term societal value before quarterly financial performance.
And above all, restraint.
The most intelligent decision is not always to deploy what we are capable of building.
Sometimes intelligence means recognizing what should not yet be deployed.
I do not believe technical intelligence alone qualifies as wisdom.
Solving extraordinarily difficult mathematics is impressive.
Building extraordinary technology is impressive.
But intelligence at the highest level must include foresight, awareness of consequences, restraint, empathy, and the ability to think beyond immediate personal or financial rewards.
That is where I believe humanity is failing.
AI could become one of the greatest tools our species has ever created.
But if the same human incentives that distorted fossil fuels, finance, the internet, and social media dominate AI, then the industry may destroy public trust in the technology before it realizes its potential.
The biggest threat to AI may not be AI.
It may be the humans deploying it.
And if the industry does not recognize that soon, the next several years could become a case study in how extremely intelligent people built something extraordinary — and then undermined it through short-term thinking.
I hope OpenAI considers a different path.
Not slower innovation for the sake of being slower.
Smarter deployment.
More disciplined deployment.
More accountable deployment.
A model where AI earns authority instead of being given authority simply because it exists.
Humanity does not need more technological power without corresponding wisdom.
It needs both.
That is the conversation I believe the AI industry urgently needs to have.
QuantusArk
@sam @openai