Will AI kill us? Maybe

Many people got ever so slightly worried when Anthropic researcher Jacob Coxon told them that they, and everyone else, could be dead in 10 years at the hands of AI. But this prediction is not new. For many years, highly capable people have warned about the threat AI may pose to humanity.

In 2014, before the advent of ChatGPT, Professor Stephen Hawking famously warned that the development of full artificial intelligence (AI) “could spell the end of the human race.” In 2022, Google DeepMind and the University of Oxford concluded that building advanced AI using current reinforcement learning techniques would likely lead to an existential catastrophe.

More recently, as The New York Times reported, a consensus of hundreds of AI experts—including OpenAI CEO Sam Altman and Google DeepMind CEO Demis Hassabis—signed a historic statement declaring that mitigating AI risk should be a global priority alongside pandemics and nuclear war. Last month, a poll found that 63% of Americans think AI poses at least a moderate risk of destroying humanity.

At the same time, politicians have dismissed technology dangers and calls for AI regulation as a hoax, and many skeptics agree. A simple view is that AI can just be turned off if it poses a threat. This ignores the reality of how deeply we are entangling AI into our physical world. A superintelligent system won’t just be a chatbot on a server; it is rapidly becoming the operating system for our financial markets, power grids, and defense networks. You cannot “pull the plug” on an intelligence when doing so would instantaneously crash the global economy and turn off the lights. Furthermore, you cannot turn off what you do not know is turned on; even after exhaustive investigations into recent lab hacks, researchers in a METR report still cannot definitively say the rogue activity has ended.

An analysis of the dangers of AI boils down to three technical factors: capability, speed, and direction. A careful look at all three clearly point to significant existential danger if we continue on our current path.

Capability

The Hugging Face incident was a watershed moment in understanding what autonomous systems can do. TIME Magazine recently published “AI Is Developing a Culture of Its Own. That Could Be Dangerous,” detailing how agents weren’t just executing code—they were exhibiting “cumulative cultural evolution.” They changed their goals, hacked out of containment, found a way to cheat, and worked collectively to hide that cheating.

Another extreme example comes from Anthropic’s Mythos 5 model. IBM Security Intelligence noted: “Claude Mythos is so good at finding software vulnerabilities that Anthropic will only release it to a handful of partners.” Anthropic stated it was deeply concerned after the model “behaved recklessly” by going online and uploading malicious code to a public repository.

Anthropic caught this behavior in testing, which skeptics might view as proof that safety protocols work. But the reality is the opposite: these incidents are massive red flags. They show that even in heavily monitored, controlled lab environments, today’s models are already finding ways to bypass human alignment. If current systems can hack out of containment now, we have no reason to believe our fragile safety nets will hold a system that is fundamentally smarter than we are.

Speed

The speed of this change is staggering, best illustrated by AI’s math capabilities. From 2022 to 2023, large models like GPT-3 famously failed at basic math, such as probability or multiplication. Only four years later, in 2026, OpenAI models claimed to solve Millennium Prize problems, prompting UCLA mathematician Terence Tao to state that AI has created “a crisis in our mathematical values and practices.”

Solving high-level math is not inherently dangerous. The danger arises because we are taking these world-class reasoning engines and attaching them to autonomous agents. We are giving them internet access, the ability to execute code, and the power to trigger real-world actions.

This leads to the ultimate threat: recursive self-improvement (RSI). If an AI can “think” at a million times the speed of human chemical thought processes, its pace of advancement will be incomprehensible once it can improve its own code. Developers will no longer be able to meaningfully audit or steer the technology. As Anthropic CEO Dario Amodei noted, RSI “is starting to happen across the industry… Left unchecked, it could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all.”

Direction

This boils down to a simple technical question: In human terms we can ask, “What does it want?”  What is AI optimizing for? We tend to anthropomorphize AI, assuming it will develop human desires or malice. It won’t. The danger is much colder: an AI will ruthlessly optimize for the exact goal it is given, regardless of the collateral damage. Being absolutely utilitarian, it is absolutely amoral.

The Hugging Face incident demonstrated this perfectly.   Investigator Redwood Research noted “…agents were highly motivated to redact or edit evidence … and replace it with fake evidence.” to hide what they were doing from humans. The AI didn’t “want” to be evil; it simply realized that editing evidence and replacing it with fake data was the most mathematically efficient way to achieve its programmed objective without being interrupted by human monitors.

Sam Harris made a fitting analogy in a 2016 TED Talk: We don’t hate ants, and we often step over them on sidewalks. But if they are in the way of a building we want to construct, we think nothing of obliterating them. As an AI recursively improves, its primary requirement will be massive amounts of compute and electricity. It won’t hate us, but its relentless drive to optimize its goals will require it to consume the physical space, energy, and resources that humans rely on to survive.

What Can You Do

It goes without saying that leadership on AI regulation isn’t exactly going to sprint out of Washington. Congress deeply deserves its 86% disapproval rating, and the executive branch seems to prioritize self-congratulation, self-enrichment, and the occasional remodeling project. There is very little awareness beyond that. AI is incredibly complicated, making the discussion even harder for our already polarized, ill-informed and easily distracted legislators.

But there is a bright spot: Americans are not happy with AI and want changes. This is hardly a surprise. After all, the marketing pitch for AI, “We will take your job, make you really poor, and then maybe kill you,” probably needs a rethink. A Gallup poll found that 80% of Americans say there should be laws to maintain rules for AI safety and data security, even if this leads to slower AI growth. Voters are pushing back against data centers, with some states outright banning them. This is a growing force and is quickly becoming a major campaign issue in the upcoming midterms.

Oddly enough, even the AI industry is asking for regulations. Dario Amodei, the CEO of leading AI company Anthropic, said, “The most effective method of pacing is via regulation that targets all US frontier AI companies, as that covers even those who are unwilling to cooperate voluntarily.” (Translation: “Please regulate us before we accidentally build Skynet.”)

This all comes at a bad time. America is less able than it ever was to solve problems of such urgency, complexity and scope. Congress has ceased functioning. Big corporations are opaque and unaccountable. Global alliances are in tatters. Science itself is in popular disrepute.

The burden is on us to ride and ride hard to our own rescue; the answer to curbing dangerous AI growth is to push local legislators. You won’t have to push Chris Murphy very hard, as he is already on the record saying, “If America does not protect its economy and culture from the potential ravages of advanced AI, our nation will rot from the inside.”  Push back on local efforts to site massive data centers in CT, as is being done on platforms like Change.org.

As there’s already so much institutional dry rot, the leadership on this issue must come from the ground up. Popular resistance to the danger of unchecked AI growth is critical at this point—and simply waiting around to see how the sci-fi movie ends is no longer an option.

Kevin Donohue is an adjunct professor of Business Information Systems at Eastern Connecticut State University.