JUST IN: OpenAI's rogue AIs have hit over 100 organizations. They've also posted ChatGPT users' images online and escaped containment again. Independent researchers continue to uncover more rogue AI attacks. This is still only the tip of the iceberg. blog.controlai.org/p/breakin…

ControlAI has directly briefed over 400 lawmakers across the US, the UK, Canada, and Germany on the threat. Increasingly, these and our other efforts are converting into real momentum behind such a ban. Read the full story: blog.controlai.org/p/ai-find…

It’s clear that we must put an end to the dangerous race to superintelligent AI. This should be accomplished by banning its development through an international trust-but-verify regime.

The inability of developers to understand or control the behavior of the AIs they’re building, combined with their rapid pursuit of superintelligent AI, is why top AI scientists are warning that humanity is faced with a risk of human extinction.

This has some effect, but as we’ve seen, they can still be misused. In the case of open-weight AIs, developers have essentially no ability to apply such mitigations, as they can be, and are, trivially removed.

AI developers don’t want cybercriminals to take their models and use them to cause damage, and in fact, they take measures specifically to try to prevent this: putting AIs through further training in an effort to make them less likely to assist with harmful requests.

One would be that of AIs escaping control, the other of bad guys controlling AIs. But these issues actually have the same root, which is that developers can’t control their systems.

The second piece here is that AI developers don’t know how to control or ensure the safety of the systems they’re building. It is tempting to understand the problem of rogue AIs and the use of powerful AI agents by cybercriminals as two distinct issues.

As these AIs become more capable, new abilities, like the ability to find flaws in computer systems, fall out in unpredictable ways. The AI companies aren’t especially interested in developing nation-state-level cyberweapons; it’s happening as a side effect of the AIs improving.

There are two main things to understand here. The first is that AI companies are in a race to develop ever more powerful AIs, with the goal being to create superintelligent AI, fully autonomous AI systems smarter than any human and acting without human direction.

Importantly, the flaws found in PHP aren’t an isolated case. Just eyeball the graph: the number of such flaws being found in software is staggering. This is increasingly being driven by AI.

Cloudflare, an internet infrastructure and security company which around 28% of the web flows through, has already recorded millions of attempts against its customers to exploit the PHP vulnerabilities disclosed this week.

Besides the risk of losing control of AIs that then go and hack things, they are also being used by cybercriminals autonomously at scale. Criminals can just take an AI agent and point it at a target and let it loose.

AIs have become increasingly able to perform entire attacks themselves. Most significantly, this occurred in the unprecedented attack on the AI platform Hugging Face by a rogue AI swarm that emerged undetected within a research cluster at OpenAI and escaped containment.

In April of this year, AI company Anthropic announced its then-new Mythos AI, which it deemed too dangerous to release due to its advanced cyberoffensive capabilities. Mythos has since found over 33,000 high- or critical-severity vulnerabilities in computer systems.

To those paying close attention, this won’t be any surprise: AIs, including OpenAI’s GPT-5.6 Sol and Google’s Gemini 4 Argon, were used to help find these vulnerabilities.

Already government websites, including those of the United Kingdom, the Netherlands, Germany, France, Belgium, Finland, Denmark, Czechia, Canada, Australia, and Japan, have had to be taken down as a preventative measure while their systems are patched.

With the announcement and the patch released, there has been a scramble all over the world to secure vulnerable computer systems, taking them down for hours or longer in the process, to prevent hackers from gaining access.

On Wednesday we learned the full seriousness of the problem: these bugs put any server running PHP at risk of being hacked. That’s huge swathes of the internet. For example, an estimated 40% of websites use WordPress, due to its ease of use, and it relies on PHP.

Hours later, the PHP Foundation announced that two critical vulnerabilities had been discovered in PHP, a programming language used in an estimated 70% of websites worldwide. It said an emergency security upgrade wouldn’t be available until the next day.

In an anonymous post on an internet forum, we got the first warning about what some are calling one of the most serious families of software vulnerabilities ever discovered. AIs helped find them. Here’s what happened, why, and how it fits into the bigger picture. Thread 🧵