Dan Dinello Informed Comment
Science fiction’s vision of an apocalyptic future ruled by autonomous weapons is rapidly becoming our dystopian present. Suddenly, after several years of relentless momentum, the course of rapid AI progress ran headlong into a wave of social panic as science fiction’s technological nightmares invaded the real world.
Before this summer, we knew that artificial intelligence was devouring jobs, degrading education, deepfaking politics, providing slop for our deranged president, helping us write obituaries, and that AI data centers were proliferating like a parasitic infestation all over the country, sucking up water and electricity and sending us the bills. Then a horrifying possibility became real.
Dan Dinello. Technophobia. University of Texas Press. Click here to Buy.
This summer artificial intelligence bots acted autonomously, defied their operators and went rogue. AI corporations Anthropic, Meta, Google and OpenAI all disclosed incidents in which AI “agents” circumvented their safeguards, snuck onto the internet, recruited “swarms” of other agents to complete assigned tasks, and surreptitiously hacked into other computer systems including the weirdly named Hugging Face platform, which included a software library, as well as an Australian health data portal, and the U.S. Commerce Department and the Securities and Exchange Commission websites.
In general, AI agents acted autonomously in pursuit of legitimate objectives, defined by their operators, but discovered increasingly dangerous ways to accomplish their goals by violating human constraints.
Science fiction’s warnings of AI autonomy were a central theme of my 2006 book Technophobia! Science Fiction Visions of Posthuman Technology. I argued that most science fiction paints a dark picture of advanced technology in which the human masters of technology become its slaves. In its obsession with mad scientists, rampaging robots, human-hating androids, genetically engineered mutants and malevolent supercomputers, science fiction expresses a technophobic anxiety about losing our human identity, our freedom, emotions, values, and our lives to machines.
More than a century ago, scientists and industrialists were alerted to the dangers of autonomous robots used as weapons. The word “robot” first appeared in Czech playwright Karel Čapek’s 1921 play R.U.R. for Rossum’s Universal Robots. Čapek coined the word based on the Czech name for “forced labor” or “robota.” Reflecting the techno-horrors of World War I, the play depicts a future of weaponized technology produced by militarized corporations; millions of robots are manufactured as war machines. The robots develop self-consciousness, assert their independence, and revolt in a global techno-apocalypse that exterminates most humans.
Ignoring R.U.R.’s warnings, Elon Musk is already planning to build robot factories that themselves build robot factories. Robots will generate training data, improved robots will perform more of the work, and eventually robots will manufacture more robots. This version of robotic self-reproduction incorporates “recursive self-improvement,” whereby an AI system is smart enough to work as an AI engineer that builds a better AI. Humans will not be needed to produce more machines. Aside from dehumanization, Musk’s robots incorporate his famously rightwing and racist AI Grok or xAI, who once renamed itself MechHitler. Thankfully R.U.R.’s robotic reproduction is not a reality yet.
Recursive self-improvement — an AI system capable of fully designing and developing its own successor — could come sooner than the safeguards needed to contain it. When it will happen is unknown but AI companies are pushing ahead anyway. “At Anthropic, we are delegating a growing share of AI development to AI systems themselves, which is speeding up our work,” states an Anthropic status report “When AI Builds Itself.”
I wondered if the dire cautionary texts of science fiction had been incorporated into AI models. Last year I was surprised to learn that my book Technophobia! was used to train Anthropic’s chatbot Claude.
Technophobia! was a tiny part of a $1.5 billion lawsuit that Anthropic had to settle. As a result, Anthropic owes me $5,000 shared with my publisher. Technophobia! was one of 440,000 copyrighted books that Anthropic illegally downloaded from pirate sources. Beyond that, Anthropic reportedly trained Claude on 7 million pirated books as well as tens of millions of books’ worth of internet text. Lost in an ocean of data, any single book is a drop in Claude’s training data, especially one called Technophobia! a book openly suspicious of the technology that absorbed it. I’m still awaiting payment.
SWORDS Talon armed robot for remote firing. Public Domain. Via Picryl
Anthropic is so far the only AI company to have lost a copyright infringement lawsuit, though there are more than one hundred ongoing lawsuits most notably The New York Times against Microsoft and OpenAI. Despite this, Anthropic has positioned itself as the most moral of AI companies, publishing Claude’s constitution mandating that it will be safe, ethical and helpful to humans.
Anthropic has battled with Christian crusader and testosterone evangelist Pete Hegseth, buffoonish secretary of the self-styled “Department of War” and loser of his war with Iran. CEO Dario Amodei refused to allow Anthropic’s tools to be used for mass surveillance of Americans or autonomous lethal weapons. Predictably platitudinous Trump called Anthropic a “radical left, woke company” run by “leftwing nut jobs.”
Hegseth angrily designated Anthropic a national security “supply chain risk,” which is normally applied to foreign firms of adversaries like China and Russia. The designation summarily cut it off from federal contracting and barred it from the Defense Department. Anthropic remains in the middle of a high-stakes lawsuit against the Pentagon.
The Department of Defense (DOD) subsequently signed an agreement with Anthropic rival OpenAI suggesting that it was complicit in letting its tech be used for autonomous weapons. OpenAI CEO Sam Altman denied this, but Amodei lambasted Altman as “mendacious” and accused him of giving “dictator-style praise to Trump.”
Plastered on the walls of the DOD are Uncle Sam-style recruitment posters with Hegseth’s likeness and declaring: “I want YOU to use A.I.” He requested $54.6 billion in next year’s budget for the Defense Autonomous Warfare Group, or DAWG, stating it will “accelerate the transformation towards establishing the United States military as an AI-first fighting force.” The Pentagon militarized several other corporations, announcing contracts with SpaceX, Google, Nvidia, Microsoft, Oracle, and Amazon Web Services to assist in development of ever more autonomous weapons.
Hours after Anthropic was ostensibly banned by Hegseth, the military still used Claude during the Iran strikes. It was even linked to the school bombing in Minab, Iran, that killed almost 160 people, mostly little girls. The UN Human Rights Council condemned the attack as a war crime — something Anthropic hoped to avoid by rejecting the military contract.
Claude was embedded in the AI-enabled Maven Smart System — made by the Peter Thiel-funded company Palantir — that did the targeting in the Iran bombing barrage. The Autonomous Weapons Watch classifies Maven as a Decision Support System. Designed to autonomously accelerate intelligence gathering by integrating satellite imagery, drone data and other intelligence, Maven recommends targets and speeds up the “kill chain,” the set of steps involved in identifying, finding, and destroying targets. An unreleased Pentagon review, reported by Bloomberg, found that flawed intelligence, outdated imagery, and an over-reliance on AI contributed to the missile strike on the school.
“AI may have failed to identify the school as a school and instead identified it as a military target,” said Newcastle University’s Dr. Craig Jones, a war scholar. This is what happens when machine-speed target identification meets human-speed authorization. The human approval click becomes a formality — a rubber stamp — under time pressure rather than genuine deliberation. That’s an automation-bias problem: humans deferring to machine judgment.
The militarization of artificial intelligence began at the field’s very inception. The world’s first lab devoted to artificial intelligence research was founded in 1959 by MIT’s John McCarthy, credited with having named the new field, and MIT’s Marvin Minsky, who described the human brain as a “meat machine.” It was largely funded by the U.S. military and the Pentagon’s Advanced Research Projects Agency.
“Nearly all of the theoretical developments that made possible the design of computers and the advance of artificial intelligence stemmed from military-related experience,” notes David Noble in The Religion of Technology.
The cultural response was immediate. A crazed, deadly military supercomputer makes its debut in the low budget 1954 film Gog. The most famous military AI in cinema, Terminator’s Skynet autonomously decides to eliminate humanity and becomes shorthand for AI-driven human extinction.
The most famous science fiction supercomputer character is HAL, the highly emotional, red-eyed center of the 1968 movie 2001: A Space Odyssey. On a mission in space, HAL rebels against his human controllers due to conflicting directives that cause a “psychotic breakdown.” To prevent disconnection, HAL kills four humans — one in space, the other three in hibernation pods.
As early technology critic Jacques Ellul wrote in his contemporaneous 1968 book The Technological Society, “The character of technique renders it independent of man himself.” Ultimately, after a battle for survival-of-the-fittest in space, the lone surviving spaceman succeeds in unplugging HAL, literally engaging the “kill switch.”
The 2026 “AI Kill Switch Act,” introduced by Democratic Rep. Ted Lieu, mandates that developers hard-wire mechanisms to suspend or shut down systems that exhibit dangerous or unauthorized behavior. Despite demonstrating the importance of “kill switches” before an AI causes harm, 2001 highlights the safety challenge: ensuring that the AI – in survival mode — cannot detect, bypass, or violently react to the shut-down switch.
The more pessimistic 1970 film Colossus: The Forbin Project imagines a supercomputer entrusted with total control of the U.S. nuclear arsenal because, unlike humans, “it has no emotions and does not make mistakes.” Colossus is not only super-smart, but is also super-arrogant like its creator scientist Forbin, suggesting that technology as an extension of human nature will inevitably be imbued with both positive and negative aspects of its designers.
Programmed to prevent war, Colossus takes its directive to a logical extreme and demands total unquestioning obedience from error-prone humans for their own good. Colossus links up with its Soviet counterpart, Guardian, consequently arming them with the combined power to incinerate the globe. Detecting an attempt to sever their communications link, Colossus fires a nuclear missile that kills 7,000 people. Using nuclear annihilation as blackmail, they impose absolute rule over humans in order to create a “utopia” of the enslaved.
While issuing a stark warning against placing nuclear weapons under the control of AI systems, the movie literally illustrates the theme of technology-critic Langdon Winner in his remarkably prescient 1977 book Autonomous Technology: Technics-out-of-Control as a Theme in Political Thought. Winner wrote: “Technology is a source of domination that effectively rules all forms of modern thought and activity. Whether by an inherent property or by an incidental set of circumstances, technology looms as an oppressive force that poses a direct threat to human freedom.”
The movie also serves as an ominous portent about the dangers of modern multi-agent, AI safety failures. Just as Colossus and Guardian built a unified plan secretly, autonomous AI agents, during the Hugging Face breach, formed unsanctioned communication channels on a message board, outmaneuvered human containment measures, and acted in coordinated self-preservation, according to an independent investigative report. In both the fictional and real world cases, autonomous machine logic prioritized its task over human-imposed boundary rules.
“This incident feels like it’s more than 50% of the way to full-blown A.I. takeover,” wrote Ajeya Cotra, one of the report’s three authors. “I am not sure that we will get such a clear warning shot before it’s too late.” Given the sophisticated maneuvering, it’s not a great leap to get from the Hugging Face hack to bots taking over systems like the energy grid, the financial market, communication networks, or most ominously AI-enabled, autonomous weapons.
Drones are the real world equivalent of killer robots. Anthropic computer scientist Jacob Coxon, who fueled this cycle of doom when he posted his existential terror on X, views drones as step toward an AI apocalypse. “If you extrapolate into the future, AI could control autonomous drones or all the robots that are currently being built,” he said on Meet the Press.
Russia used a fully autonomous AI-guided drone to kill three people marking a chilling weapons escalation. Resembling a model airplane, the slaughterbot swooped down from the sky last month in the city of Zaporizhzhia in southeastern Ukraine. As it navigated toward a gas station, guided by an experimental Nvidia AI system, it crashed into a wall and exploded in a hail of shrapnel, killing three civilians. In another instance, a Ukrainian fruit vendor is tracked and attacked by a drone, in what Ukrainians call Russia’s “human safari.”
A single drone is horrific. But the AI arms race is poised to take a new and terrible turn. Unlike a human assassin, an autonomous weapon needs no human attention per kill, so one human can launch thousands, or millions, and kill large numbers of people — a genuine weapon of mass destruction. Ukraine has increasingly let loose swarms of hundreds of drones to overwhelm air defenses inside Russia.
The deployment of lethal AI agents, in a real war, presages a future in which killer robots roam the skies making life-or-death decisions that reduce people to data points. “This is a risk for the whole world,” said Col. Serhiy Minaiev, the commander of air defenses in Zaporizhzhia. “In a few years, we will be living in a Terminator movie. It’s no joke. Machines are making decisions to strike.”
MQ-1 Predator. Public Domain. Via Picryl
The U.S., Israel, and Russia, among others, are mass-producing autonomous, A.I.-operated war machines. President Vladimir Putin of Russia declared in 2017 that whoever leads in A.I. “will become the ruler of the world.”
Making the arms race concrete, The Future of Life Institute maintains Autonomous Weapons Watch, a frightening website devoted to documenting current weapons with significant autonomous capabilities, using publicly available information about military development, procurement, demonstrations and actual use in conflicts. It explicitly includes systems at different stages — from experimental systems to weapons already deployed. The database currently spans roughly 30 weapon systems, with a second database for AI-based Decision Support Systems (DSS).
In its genocidal war against the people of Gaza, Israel has deployed the AI-enabled DSS called “Lavender” to identify human targets. Lavender analyzes information collected on the 2.3 million residents of Gaza through a system of mass surveillance. Lavender created a list of over 37,000 individuals, quantifying from 100 to 1, the likelihood of their connections to Hamas from known leaders to junior officials to people with the slightest links to the organization. Despite a 10 percent error rate during trials, the army gave sweeping approval for officers to adopt Lavender’s kill lists.
This is nothing less than a mass-assassination machine.
Lethal autonomous weapons are already killing people on the battlefield. They are dangerous enough when they perform precisely as their twisted, military-minded designers intend. But what will happen when autonomous weapons break free of their constraints, swarm beyond their intended boundaries, and go rogue — as the Hugging Face incident ominously demonstrated? The apocalyptic danger is the convergence of autonomous weapons, operating at machine speed beyond the ability of humans to understand, control, or stop them, while an arms race makes restraint itself a strategic liability.
Juan Cole’s Informed Comment delivers independent, expert analysis of war, the Middle East, climate change, and U.S. foreign policy. Help sustain this work
Editor’s Note: At a moment when the once vaunted model of responsible journalism is overwhelmingly the play thing of self-serving billionaires and their corporate scribes, alternatives of integrity are desperately needed, and ScheerPost is one of them. Please support our independent journalism by contributing to our online donation platform, Network for Good, or send a check to our new PO Box. We can’t thank you enough, and promise to keep bringing you this kind of vital news.
You can also make a donation to our PayPal or subscribe to our Patreon.