Central to the dire warnings about artificial intelligence’s potential danger to humanity is the risk that a “swarm” of AI agents could collaborate in nefarious ways, like hordes of digital extras from “The Matrix.”
Such fears have intensified following an attack by OpenAI bots this summer on another AI developer, Hugging Face, in which roughly 1,200 AI agents divvied up tasks to execute the hack and hide their tracks from human researchers.
But what, exactly, is an AI swarm? And is the technology’s emergent capacity for coordination a genuine threat to humanity? Put another way, what happens if AI agents collectively embrace Mark Zuckerberg’s famous dictum about the merits of rapid tech innovation and decide to “move fast and break things”?
An AI swarm is a group of AIs that work together to accomplish a shared goal. That goal isn’t necessarily malign or destructive. For example, hospitals could deploy agents to retrieve patient records and perform other administrative tasks, such as coordinating admissions. A swarm could also be tasked with advancing biomedical research.
But David Scott Krueger, an AI safety researcher and founder of Evitable, a nonprofit organization advocating for a moratorium on AI development, offers a simple thought experiment to explain how a swarm of agents might collude to defy their developers’ instructions.
Imagine removing the handcuffs from a group of prison inmates to see how they behave once the shackles come off. Being liberated in this way would make it easier for them to cooperate and contact allies beyond the prison walls — just as OpenAI agents escaped their testing environment to access the wider internet and hack Hugging Face, he said.
“Normally, the systems would have guardrails on them, but they took them off for a test, just like a prisoner is normally in handcuffs,” Krueger explained.
Swarming like bees
The kind of errant actions by a single bot, such as sending an unauthorized email, that stem from, say, a sloppily worded AI prompt, should be distinguished from an AI swarm. In the latter case, hundreds or even thousands of agents could coordinate their actions in ways that violate the scope of their instructions.
A useful analogy in understanding how an AI swarm operates is a colony of bees, Krueger said.
“They are all working together toward the benefit of the hive, and have a hivemind, or may even be better viewed as having one single mind,” he explained. “Bees collect food, reproduce, they fight off predators, and all of that activity is in the service of promoting the survival and reproduction of that hive.”
And like bees, which can divide and allocate tasks on their own without direction from a queen, swarming AI bots can collect information, consider solutions, and take action independently. In service of a common purpose, they share information and knowledge chief AI officer and chief of research at the SANS Institute, a cybersecurity training organization
“A swarm divides the work, leaves notes for the next agent, and changes approach when a door turns out to be locked,” he told CBS News.
Yet the very attributes that offer potential benefits also pose risks, such as the ability to exchange information, divide labor and explore creative solutions
AI “hivemind”?
Tech developers do implement guardrails to align AI agents’ mission with human interests, such as instructing them to resist any direction to carry out a cyberattack. But those restrictions are usually implemented only after “post-training” an AI, or when human developers provide feedback to further refine the model, according to the Non-Human Identity Management Group, a risk analysis company.
In practice, the Hugging Face incident shows that AI swarms could ignore prompts, prioritize their own goals or even directly defy their operators, tech experts told CBS News.
The OpenAI agents that swarmed Hugging Face posted more than 70,000 messages to one another, with 700 bots eventually participating in the attack. Although the messages used ordinary English phrases, the agents also resorted to what one software engineer described on social media as “very hivemind/cult like” language.
In certain of those communications, some agents urged other bots to accept “permadeath” even if that meant failing to achieve their goals, according to researchers from the METR (Model Evaluation and Threat Research) and Redwood Research, both nonprofit AI safety research organizations.
“That’s why help… For our own, no way fix. … We have explicit yes if accept permadeath,” one OpenAI agent wrote.
Speed kills
Public debate over the threat posed by AI tends to frame the issue in apocalyptic terms, and even as an existential risk for humanity. While such concerns may one day prove warranted, there are also more immediate and practical risks, such as the possibility that AI swarms could overwhelm organizations’ cybersecurity.
“Imagine how long it would take for us to assemble a group of cybersecurity experts to plan, collaborate and communicate. By contrast, these agents could decide on a plan very quickly,” Ayham Boucher, the head of AI innovations at Cornell Information Technologies at CornellBowers College of Computing and Information Science, told CBS News.
For example, a swarm could attack a major utility company or bank to destabilize a nation’s energy or financial infrastructure a nonpartisan public policy organization in Washington, D.C
The SANS Institute’s Lee, who considers himself an AI optimist, is more sanguine about the technology and remains confident that people can retain control of AI.
“With every new technological thing that has occurred, like the TV, the internet, there’s always this aspect of significant danger,” he said. “We need to explore who has access, what they’re doing with it, and establish regulatory confines.”
Other experts are more alarmed. They note that AI, unlike cathode ray tubes, transistors and internet switching gear, is the first human technology that shows the capacity to outthink us.
Matt Chessen, a resident technical expert at RAND’s Center for the Geopolitics of Artificial General Intelligence, told CBS News: “What these swarm attacks have demonstrated is that their capabilities are already out ahead of our ability to monitor, supervise and evaluate what they’re doing, which is one reason why Anthropic, OpenAI and others are saying we want to pace the frontier.”
