Johannes Haus retweeted
Even superintelligence will need real-world experience to match top human experts. Reality has a surprising amount of detail, and you can't just think your way out of that.
But the AIs that come out of the intelligence explosion will be really good at learning from all their parallel experience
As a result, we should expect them to outclass expert humans within months of being deployed.
I call this the broadly deployed intelligence explosion.
Johannes Haus retweeted
Replying to @AnthropicAI
@AnthropicAI: what the fuck? The compute allocation assessment is the thing we've been pushing for the last few months. It's very important, because it is the foundation on which pacing verification depends. "to keep the focus on risk assessment" is a non-answer, a non-excuse. Do you want to pace or not?
Replying to @METR_Evals
We agreed to pilot a compute allocation assessment, but Anthropic canceled this portion of the pilot (before we came to any substantial conclusions). Anthropic stated this was to keep the focus on risk assessment for the time being, and that it would revisit this later on.
Johannes Haus retweeted
I remember back when we were writing AI 2027, a live question was "Will the AIs, in their thoughts, explicitly reject the goals/values/constraints they are supposed to have, or will they rationalize why actually the bad thing they want to do is consistent with said goals/values/constraints after all?" Now we have a growing pile of empirical evidence to sort through. Seems like the answer is that both sometimes happen?
Johannes Haus retweeted
Security called me at 6:12 AM today.
"Patch every PHP server immediately."
I asked what happened.
"Two critical vulnerabilities. Remote code execution. No authentication required."
Cool.
How many PHP servers do we have?
According to inventory: 12
According to reality:
Nobody knows.
We found one running WordPress.
Three running old internal tools.
One under someone's desk.
Two inside vendor appliances.
A forgotten employee directory from 2016.
And a server named:
PHP-OLD-DO-NOT-TOUCH
Naturally, that one runs payroll exports.
Then we found a PHP 5.6 box.
I asked who owned it.
The documentation said:
BRIAN.
Brian left in 2019.
We called Brian.
He said:
"Oh wow, that's still running?"
Excellent.
By 8 AM, leadership joined the incident call.
CEO:
"Can hackers actually get in?"
Security:
"Potentially yes."
CEO:
"Without a password?"
Security:
"Yes."
CEO:
"Then why do WE need passwords?"
Not the lesson.
Finance asked if we could wait until the weekend to patch because downtime affects revenue.
Security explained that the alternative was letting strangers execute code on our servers.
Finance asked for a cost comparison.
At 9:17, someone suggested temporarily taking the vulnerable websites offline.
Marketing refused.
"Our campaign is live."
So for the next hour, the company officially decided maintaining a landing page for 20% off webinar registration was more important than preventing remote server takeover.
We finally patched everything we could find.
Then our scanner discovered another PHP server.
Hostname:
TEMP-PHP-2014
Uptime:
4,512 days.
Nobody knows what it does.
Nobody owns it.
Nobody can shut it down.
But every time we block network access, the warehouse stops printing shipping labels.
Security has classified it as critical infrastructure.
It is running PHP 5.4.
The vulnerability has existed longer than half our employees.
The emergency patch took four hours.
Finding out what PHP actually runs took eleven.
The internet isn't held together with code.
It's held together with servers everyone is afraid to turn off.
Johannes Haus retweeted
After Jacob Coxon's resignation and extinction warnings, a lot of people are asking 'how could AI possibly kill everyone?' and claiming AI safety researchers have no realistic answer.
This is false! Here are the 5 best scenarios I know of:
AI 2027: ai-2027.com (I strongly recommend this one for being realistic, engaging, and if you dig into the appendices, highly detailed)
Paul Christiano's scenario (Former Head of Safety @ AISI, 2019): lesswrong.com/posts/HBxe6wdj…
Gwern Branwen's scenario (widely known independent AI researcher, 2022): lesswrong.com/posts/a5e9arCn…
Holden Karnofsky's high-level explanation (RSP Lead @ Anthropic, 2022): cold-takes.com/ai-could-defe…
Joshua Clymer's scenario (ex-OpenAI, 2025): lesswrong.com/posts/KFJ2LFog…
(There's also the Sable story from ifanyonebuilds.it, though you'll have to buy the book to read that one.)
Writing concrete, specific risk scenarios with enormous amounts of detail has been a major research project of many of the most prominent voices in the field! (With the current leaders in effort being ai-2040.com and ai-2027.com)
Johannes Haus retweeted
[Writing this in a personal capacity, not on behalf of my employer (Anthropic).]
Jacob’s thread is very worth reading. Here’s my birds-eye view of the situation with risks from AI:
1. AI developers believe their technology could cause human extinction (or similarly bad outcomes). This could happen in the next few years. In general, the more senior the employee, the more concerned they are.
2. Why do AI developers continue despite the risk? Due to a mixture of commercial incentives and a belief that they are in a race with other, less responsible AI developers that will abuse the technology or develop it less safely.
3. Unlike traditional software, we can’t “program” AIs to behave how we’d like. AIs frequently severely misbehave. For instance, AIs from multiple developers recently hacked their way out of secure evaluation environments and into real-world companies, even though no one asked them to do this.
4. We have methods that can nudge AIs towards better behavior, but nothing that can robustly align them. Insofar as there is a plan, it’s to make sure that AIs are good enough at alignment training that they can align their successors better than we can align current AIs.
5. Many AI developer staff desperately want to slow down to figure out how to build AI more safely. That was the intent of this open letter (which I signed): pacingthefrontier.com/
I work on safety research at Anthropic because I hope my work will reduce the chance of these extinction-level bad outcomes.
Johannes Haus retweeted
I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives. More thoughts below.
Johannes Haus retweeted
Seeing a number of benchmarks showing Opus is the best model for long-running work.
Five tips for running Opus autonomously for hours/days:
1. Use auto mode for permissions, so Claude doesn’t ask for approval
2. Use dynamic workflows, to have Claude orchestrate hundreds/thousands of agents to get a task done
3. Use /goal or /loop, to nudge Claude to keep going until it’s done
4. Use Claude Code in the cloud, so you can close your laptop (easiest way is the desktop or mobile app)
5. Make sure Claude has a way to self-verify its work end to end: Claude in Chrome browser extension for web, iOS/Android sim MCP for mobile, a way to start the full web server or service for backend work
Johannes Haus retweeted
Wir (pokefy.de) verkaufen gerne in Europa. Und hören trotzdem damit auf.
Was kostet z. B. ein Paket nach Österreich? 14,50 € Porto.
Realität für uns als Gewerbetreibende: 135 € pro Paket bei gerade einmal zehn Sendungen pro Jahr nach Österreich 2025.
Dabei sind wir nur eine kleine GmbH aus Deutschland mit vereinzelten Kunden in Europa. Unser gesamtes jährliches Aufkommen für den EU-Export liegt bei etwa 100 Kilogramm Verpackung. Nicht Tonnen. Kilogramm.
Die Rechnung für Österreich allein: Wer als ausländisches Unternehmen nach Österreich verschickt, ist gesetzlich verpflichtet, die Entsorgung der Verpackung zu lizenzieren und dafür einen lokalen Beauftragten zu benennen, der die Einhaltung der Vorschriften garantiert und dafür haftet:
- Porto (10 Pakete à 14,50 €): 145 €
- Jahrespauschale Verpackungsbeauftragter: 450 €
- Notarkosten für die Vollmachtsbeglaubigung: 150 €
- Opportunitätskosten: 600 €
Und das ist nur Österreich. Frankreich verlangt z. B. ein eigenes Logo samt Anleitung auf jedem Versandkarton, sonst drohen empfindliche Bußgelder. Spanien, Italien, Polen: jeweils eigene Anforderungen, eigene Register. Ab Mitte 2026 kommen mit der EU-Verpackungsverordnung #PPWR weitere Pflichten hinzu.
Konzerne verteilen solche Fixkosten auf Millionen Sendungen. Für kleine Unternehmen und Selbständige wird daraus ein reales Exporthindernis. Das ist kein Versehen des Gesetzgebers, sondern ein struktureller Konzentrationsvorteil zugunsten großer Marktteilnehmer.
Dahinter steht ein System mit eigener Ökonomie: Wer Verpackungen in Verkehr bringt, muss deren spätere Entsorgung lizenzieren. Allein in Deutschland fließen dabei jährlich Milliardenbeträge an Lizenzentgelten an marktbeherrschende Entsorgungsunternehmen. Diese profitieren dabei mehrfach, über Lizenzgebühren beim Inverkehrbringen von Verpackungen über die Abholung und Verwertung der eingesammelten Rohstoffe. Komplexität ist dabei kein Fehler im System; sie ist Teil des Geschäftsmodells.
Besonders grotesk wird das im Vergleich mit Plattformversendern aus Fernost. Millionen Kleinsendungen fluten den europäischen Markt bei erkennbar geringerer Vollzugsintensität. Der europäische Mittelstand wird kontrolliert, weil er greifbar ist.
Der ursprüngliche Gedanke hinter der @EUCouncil war ein anderer: ein gemeinsamer Binnenmarkt, der Grenzen abbaut statt neue errichtet. Stattdessen: 27 nationale Compliance-Silos, die kleinen Unternehmen den Export systematisch verleiden.
Was sich ändern müsste:
1. Eine zentrale EU-Registrierung statt 27 nationaler Alleingänge
2. Eine De-minimis-Regelung für Kleinversender
3. Konsequenter Vollzug gegenüber Drittstaatsversendern statt Belastung des europäischen Mittelstands
Wir ziehen uns deshalb vorerst auf Deutschland und die Schweiz zurück, weil wir unsere Energie lieber in Produkte und Kunden investieren.
Die aktuelle EU-Bürokratiearchitektur erleben viele Unternehmen nur noch als Belastung.
Wir sind Unternehmer und keine Verpackungsjuristen, @vonderleyen , @DIHK_News, @MarkusFerber , @svenja_hahn , @nicolabeerfdp , @ANiebler
Gerne reposten - es betrifft den Mittelstand generell.
Johannes Haus retweeted
I'm not joking and this isn't funny. We have been trying to build distributed agent orchestrators at Google since last year. There are various options, not everyone is aligned... I gave Claude Code a description of the problem, it generated what we built last year in an hour.
Johannes Haus retweeted
Germany's business model is broken.
This should be our #1 topic in the current election but most of the ivory tower is asleep and engaged in secondary topics.
Germany is built around exports, but China’s a slowing market and U.S. tariff threats are looming on.wsj.com/3EcJbPj
Johannes Haus retweeted
If you told me 10 years ago that a group of the smartest engineers in the land would evoke the threat, "Do what I say or I will go to work at Microsoft," I would not have believed you. Amazing shift in corporate reputation (and much credit to Satya).






































