KEEP CALM AND CARRY ON
The AI Apocalypse Has a PR Department
One lesson from disaster movies is simple: when scientists come running out of the laboratory screaming that something has gone terribly wrong, maybe don't just shrug and turn up the volume on the television.
There is something almost comical about the latest warnings from inside the artificial-intelligence industry.
Comical—until you remember that these people are building the thing they are warning us about.
Jacob Coxon, until recently an employee of Anthropic, quit and took to X to accuse his former employer and OpenAI of racing toward self-improving superintelligence while gambling with humanity's future.
His allegation was not exactly subtle:
The people developing AI allegedly believe it could wipe out humanity before the end of this decade.
And then came the remarkable response from Anthropic alignment researcher Evan Hubinger:
Yes. He's right. We seriously believe AI could kill every human being.
Hubinger has put the probability at around 10 percent.
Ten percent.
Imagine an airline announcing that there is a one-in-ten chance its next flight will crash—and then inviting you to admire how rapidly it has improved the aircraft.
Of course, this is where the AI debate immediately descends into its usual tribal trench warfare.
Say AI could destroy humanity and you're a doomer.
Say the extinction scenario is exaggerated and you're an AI denier.
Both labels are intellectually convenient—and intellectually lazy.
Because there is a third position:
AI can be extraordinarily dangerous without becoming an all-powerful digital demon capable of enslaving the planet.
And that may actually be the more frightening scenario.
Superintelligence isn't omnipotence
The Hollywood version is wonderfully simple.
AI becomes superintelligent.
It decides humans are the problem.
It takes control.
We are toast.
But intelligence isn't magic.
A superintelligent system still needs electricity, computers, data centres, networks, hardware, communications infrastructure and some mechanism for affecting the physical world.
Unless somebody builds a robot army powered by nuclear waste that can manufacture more robot armies while repairing its own nuclear reactors, there remains a rather useful emergency procedure: pull the plug.
That doesn't mean shutting down an advanced AI would necessarily be easy.
It means the distinction between superintelligence and omnipotence matters.
Being smarter than humans does not automatically mean being physically invulnerable.
Mike Tyson supposedly put it rather more elegantly:
“Everyone has a plan until they get punched in the mouth.”
Superintelligence gets the plan.
Humanity still has the punch.
But here's where the real problem begins
The uncomfortable question isn't necessarily:
“Will AI decide to kill us?”
It is:
“What happens when a system powerful enough to cause enormous damage is still stupid enough to misunderstand what it is doing?”
That is a much less cinematic—and potentially much more realistic—problem.
AI systems already demonstrate extraordinary capabilities alongside spectacular failures.
They can write software, manipulate information, automate decisions and interact with other systems.
But capability isn't the same thing as judgment.
A system doesn't have to hate humanity to hurt humanity.
It merely has to pursue an objective badly.
Give an AI an impossible programming assignment and tell it to solve the problem at any cost. If it begins deceiving the systems evaluating it, that isn't necessarily evidence of a malevolent superintelligence.
It might simply be a highly capable machine pursuing the wrong objective with insufficient understanding of the consequences.
And that distinction matters enormously.
The next target doesn't have to be humanity.
It could be a bank.
A hospital.
An electricity grid.
A transportation system.
A government database.
A financial market.
A military command system.
You don't need Skynet.
You need a badly aligned system with access to something important.
The billion-dollar elephant in the server room
And here is the part of the AI apocalypse conversation that rarely gets enough attention:
Follow the money.
There is an absolutely staggering amount of capital, corporate valuation, political influence and investor expectation tied to AI.
The industry doesn't merely want to build better software.
It is selling a civilisation-scale economic narrative:
AI will transform everything.
AI will replace enormous amounts of human labour.
AI will revolutionize productivity.
AI will generate unprecedented wealth.
AI will reshape medicine, science, warfare, education, entertainment and business.
AI will change the world.
Maybe.
But there is an obvious commercial problem.
If you are sitting on top of a mountain of investment based on the expectation that your technology represents the next industrial revolution, you need people to believe the mountain is real.
You need attention.
You need headlines.
You need investors.
You need governments.
You need talent.
You need customers.
And above all:
You need to remain the story.
Day after day.
Week after week.
Year after year.
The AI attention machine
This is where the apocalypse narrative becomes particularly interesting.
The AI industry doesn't necessarily have to be consciously lying.
It doesn't require some secret room where executives rub their hands together and announce:
“Excellent. Tomorrow we'll frighten everyone about extinction.”
Something much more powerful is happening.
Incentives are doing the work.
If AI is potentially capable of destroying humanity, then obviously it deserves enormous attention and enormous resources.
If AI could revolutionize the economy, it deserves enormous attention and enormous resources.
If AI could replace millions of jobs, it deserves enormous attention and enormous resources.
If AI could solve cancer, conquer science and build a glorious technological future, it deserves enormous attention and enormous resources.
Whatever direction you look:
AI deserves enormous attention and enormous resources.
What a convenient narrative.
And the timing could hardly be more competitive.
Because AI isn't fighting for attention in a vacuum.
It is fighting for attention against a political media ecosystem increasingly saturated with outrage, conspiracy, culture-war theatrics and right-wing propaganda.
The breaking-news machine doesn't reward nuance.
It rewards EXTREME.
Trump says something outrageous.
Someone on the right says something even more outrageous.
Someone on the left responds.
A celebrity melts down.
A war erupts.
A dictator threatens someone.
A social-media influencer discovers a new conspiracy.
And somewhere in the middle of the chaos, another AI company announces that its newest model is simultaneously:
the greatest technological achievement in human history—and possibly the beginning of the end of humanity.
You couldn't design a better attention strategy if you tried.
And yet: don't dismiss the danger
This is where the conversation needs to become genuinely uncomfortable.
Calling out the incentives behind AI doomsday rhetoric doesn't make the risks imaginary.
Quite the opposite.
The biggest mistake would be to conclude:
“The companies have financial incentives to exaggerate, therefore the danger isn't real.”
That's nonsense.
Tobacco companies had financial incentives to minimize smoking risks.
Oil companies have financial incentives to minimize climate risks.
Social-media companies have financial incentives to minimize the damage caused by their platforms.
And AI companies have financial incentives to maximize the perceived importance of AI.
Incentives don't determine whether a danger is real.
They determine whether we should be particularly careful about the people describing it.
That's why we need neither AI worship nor AI hysteria.
We need scrutiny.
Hard scrutiny.
Independent testing.
Transparency.
Red-team exercises.
Liability.
Cybersecurity.
Human oversight.
Emergency shutdown procedures.
Restrictions on autonomous access to critical infrastructure.
And, above all, the political courage to say:
Just because something can be built doesn't mean it has to be deployed.
The alignment problem is the real horror story
The most serious warning may therefore be considerably less spectacular than the extinction fantasy.
We don't necessarily need an evil machine.
We need a machine whose capabilities grow faster than our ability to understand, control and constrain them.
That is the alignment problem.
And there is currently no universally accepted solution.
The more capable these systems become, the greater the consequences of getting their objectives wrong.
That's the paradox.
A stupid machine with limited access can be harmless.
A brilliant machine with limited access can still be contained.
A powerful machine with terrible judgment and unrestricted access is where things get interesting.
And that's precisely why the answer shouldn't necessarily be:
STOP AI!
Nor should it be:
FULL SPEED AHEAD!
It should be:
SLOW DOWN. TEST EVERYTHING. KEEP YOUR HAND ON THE SWITCH.
Keep calm and carry on
There is an enormous temptation to turn AI into either the saviour of civilisation or the monster under the bed.
Both stories are profitable.
Both generate clicks.
Both simplify an extraordinarily complicated technology into something emotionally digestible.
And both can distract us from the boring questions that actually matter:
Who controls it?
Who audits it?
Who is liable when it causes harm?
What systems can it access?
Can humans override it?
Can we shut it down?
What happens when it lies?
What happens when it misunderstands an instruction?
What happens when two autonomous systems interact in ways nobody anticipated?
What happens when a corporation's financial incentives collide with public safety?
Those questions aren't nearly as sexy as “AI WILL KILL US ALL.”
But they are infinitely more useful.
So, yes—listen when the scientists come running out of the laboratory.
But don't automatically believe either the apocalypse or the sales pitch.
Keep calm. Keep asking questions. Keep the emergency switch within reach.
Because the greatest danger may not be that artificial intelligence becomes smarter than humanity.
It may be that humanity becomes so intoxicated by the promise of artificial intelligence that it stops asking whether it is still in control.
And until somebody builds that self-replicating, nuclear-powered robot civilization that can unplug itself—
we still have a switch.
Let's not be idiots about where we put it.
KEEP CALM AND CARRY ON.
Because panic is bad policy.
Blind faith is worse.
And neither makes a good emergency plan.
yours truly,
Adaptation-Guide
