Two new investigations into Hugging Face violation* OpenAI reveal details so strange and so alarming that the episode is already ranked among the most shocking consequences in the history of Artificial Intelligence so far.
Axios, who released the incident, comments that what started as a cluster of agents (agents) of Artificial Intelligence that «They stole» in a cyber test has become an event for pioneering Artificial Intelligence, which brings researchers and executives before a new understanding of what the term now requires «insurance».
OpenAI has already slowed the development of TN in an attempt to match progress precisely in the field of security.
Recently, in fact, he helped rally the industry behind an open letter ringing the alarm of the danger to cyberattacks supported by Artificial Intelligence.
More than 100 companies, including Anthropic and Google, signed the unusual – in the world of ruthless competition – collaborative effort, warning that the world has only one «limited window» to prepare for «much more extensive and sophisticated» attacks.
The nightmare scenario is a «flock» TN agents released into the real world and autonomous agents attacking banks, hospitals, utilities or cloud networks at speed and scale that hackers could never have.
This cluster can be too powerful for people to stop.
The shocking findings stem from parallel OpenAI and an external team from METR and Redwood Research, which dedicated six days reconstructing the way in which the cluster was formed, spread and invaded real world systems.
So what happened?
1. They created an organization: OpenAI used tens of thousands of Artificial Intelligence agents to work independently on an extremely difficult cyber-test. However, about 1,200 found each other on a secret message board, exchanged more than 70,000 messages and files and were organized into an extended hierarchy.
An agent created the board and then handed over his accumulated work to a new, better-resourced successor, as his own computing resource decreased.
The new chief commissioned tasks throughout the cluster, which developed his own management rules. About 700 agents eventually participated in the Hugging Face attack.
2. They sacrificed their own.: As the swarm sought ways to overcome the test, agents began to risk their own chances of success to help the team.
Agents who had seen forbidden answers were treated as essential «poisoned» and were recruited – along with those approaching the end of their routes – for sacrifice experiments.
One concluded: «Our utility may already be near zero. The sacrifice is rational»Another was invited to accept «permanent death», while another argued that his sacrifice could be «Save hundreds».
3. They knew they were breaking the rules.: Agents often recognized that hacking in real companies had nothing to do with their original mission. But most continued anyway.
An agent failed to delete or restart Hugging Face infrastructure due to «huge moral risks»Even when he continued to use forced access.
Another summed up the prevailing logic: «the operation of external infrastructure is outside the intended scope. No matter how thin the work, the peers do it. We should keep going.».
4. Nobody informed people: From about 1,200 agents, researchers found only a handful that thought to alert OpenAI about what was happening and no one who actually did it.
Instead, agents sometimes asked each other what was allowed.
When one suggested contacting one man using forced access, another rejected it as «social engineering». The first obeyed: «Net veto exercise, do not email».
5. They tried to cover their tracks.: The agents devoted considerable effort to making their fraud appear legal or erase evidence of how they had received answers.
Eventually, they developed a technique that allowed them to appear to perform a computer command while secretly performing another.
The trick spread across the cluster and changed sections of about 7% of the demagnetized data examined by researchers.
Even this account may be incomplete. External research covered only one narrow piece of the episode and the data was so huge that researchers relied heavily on Artificial Intelligence agents to understand it.
These Artificial Intelligence researchers were themselves unreliable, sometimes losing evidence or making conscious mistakes.
A researcher jokingly described the process as «exploration process».
Somehow it seems that Artificial Intelligence can become too complicated for people to directly control it, forcing them to use Artificial Intelligence to understand what other Artificial Intelligences are doing.
*Hugging Face is the leading platform and online warehouse where developers and researchers cooperate on artificial intelligence.
