Pinned Tweet
If you're a frontier lab AI researcher who thinks we're going to die in 6 months, please go to Washington and tell them that. Don't just keep building the thing that you think is going to kill us. We don't "have to push it forward".
3
47
5,124
Inkswallow ⏸️ retweeted
Unfortunately, it seems like the intelligence explosion will be happening in 2027 after all. Most of this post analyzes the new METR report (though other sources of evidence are also important). If they’re right, that implies an AI-driven acceleration that is 5 months ahead of the AI 2027 scenario. I am scared, sad, and angry. blog.aifutures.org/p/q35-tim…
Inkswallow ⏸️ retweeted
I grew up in the Pacific Northwest and we'd sometimes see signs of mountain lions around the property. Knowing that, and walking alone through the forest at night, is a particular kind experience. What I feel now reminds me of that
We at AIFP are spooked by this information. IF (big IF) Anthropic really is imminently about to meet the RSP threshold of 2x acceleration over baseline, then it seems timelines to full RSI and superintelligence may be very short indeed. For example if we naively optimize the AIFM parameters to achieve 2x acceleration over baseline progress in Jan 2027, we get superintelligence sooner than depicted in AI 2027. We are thinking harder and longer about this now and will hopefully have a lengthier update on our thinking to share in the next week.
Inkswallow ⏸️ retweeted
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had
with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works
already-clean.goodfire.com/
I'm one of those people who could die from a relatively small AI-assisted cyber incident if it was aimed at the right place and time. It's not even close to my biggest AI worry, but I resent Sam Altman's assumption that we should be happy to potentially exchange our lives or the lives of loved ones for this dubious liberty.
Sam Altman: "We're going to, as a society, have to accept some fairly severe cyber incidents from open models, in exchange for the liberty that comes with that"
Meanwhile, Microsoft & gov agencies are racing to harden critical infrastructure & expecting to fail & people to die.
2
94
Been looking at some prepper websites in case we get the kind of catastrophe the AI labs are apparently expecting, and I have to say the advice is a bit grim.
Inkswallow ⏸️ retweeted
The human race needs to ban the worldwide development of further AI capabilities.
Heave-ho.
"We think a pause is a practical policy option, if leaders decide they want to pursue it."
Excited to share a new working paper, joint with 25 interdisciplinary coauthors:
“The Feasibility of a Hardwired Pause of Frontier AI Training"
63% of voters in a recent WSJ poll want to pause AI development.
But can it be done globally? How long could it last? 🧵
3
112
Inkswallow ⏸️ retweeted
New paper.
Previous: Models transfer a love of owls through sequences of numbers (Subliminal Learning).
New: We show models can transfer more complex traits: novel skills, agentic hacking, backdoors. Here, a backdoor transfers with neither trigger nor behavior in the data!
Inkswallow ⏸️ retweeted
Two other safety researchers and I were fired from OpenAI last week. We wrote this letter to leadership.
I believe we were fired for prioritizing safety over the near-term interests of OpenAI as a corporation.
One article, seriously lacking in courage, from The Independent. Other than this, I can't see anything today in British media about the OpenAI maths release. Now would be a great time for us to wake up.
1
2
162
Inkswallow ⏸️ retweeted
The young man to my right (viewer's left) is Wes. Wes is 15. He wrote a treaty to stop AI from killing us all. You can read about it here: aitreaty.base44.app/
He is brilliant, and caring, and precocious. I pray that his mind has a chance to grow and flourish instead of being snuffed out by the horribly irresponsible project being undertaken by OpenAI and Anthropic.
It is pathetic and shameful that the task of saving the world has somehow fallen to him. It is too much for anyone to bear, never mind a teenager. On behalf of my people, of the people developing this technology who I once called friends, I'm sorry Wes
A fairly critical statement on the AI industry from David Robinson, one of the longest-tenured OpenAI employees, who has just left.
"OpenAI has thrived by trial and error"
"My colleagues and I were so busy sprinting that we seldom had the chance to consider big changes, much less to actually make them."
"The smarter the industry lets models grow while these problems remain unsolved, the more dangerous our situation becomes."
New in The Atlantic:
@dgrobinson resigned this week. He was among the longest-tenured employees at OpenAI—and oversaw safety reports on 12 frontier launches.
He is very worried: “The time for trial and error is over.”
You can read his essay here:
theatlantic.com/technology/2…
1
141
Inkswallow ⏸️ retweeted
I've done a complete U-turn on my opinion on open source AI. We should be careful (and probably disallow release) of models on par with current open source Astra/Opus level models.
AI is now reverse engineering games from binaries and remaking them. Reverse engineering games is a very hard problem. If this is possible then reverse engineering banking software, ID systems (Aadhaar), flights, etc is possible too. A game ships its binary to every player. Bank and Aadhaar backends are protected by servers, but it's unlikely the security teams at these are smarter than advanced AI that can do this to games.
We are protected rn because a Claude will refuse to tick "I'm not a robot" on websites. When OSS models don't respect that, we have a looming cybersecurity problem. Apologies I didn't see this earlier.
Rescue and adopt an adorable, abandoned AI, so you can grow it into... your personal assistant.
I wonder where the limit is, in the lengths we're going to go to to squish these alien minds into shapes that appeal to and flatter us most.
Introducing Pickle, a personal AI you raise. It gets things done for you and makes friends of its own.
Meet yours: pickle.com
1
116
On OkCupid there were questions like "Would you prefer good things to happen or interesting things?" and "Would you find it exciting to be homeless in a certain light?" It feels like the people who wanted interesting things have taken full control of the steering wheel now, and are driving us with maniacal speed to the edge of the abyss.
1
138
i guess i assumed this hearing would be covered more so i didn't bother to tweet much concrete about it, but apparently not many people even on the TL have the stomach to watch two hours of congress.
this was a hearing before the homeland security committee on *rogue ai* specifically. as far as i could tell it was extremely bipartisan. @HawleyMO , the senator in the middle here who brought out the huggingface slide, is a hyper-conservative missouri senator. the people testifying were @ChrisPainterYup (METR president), @DKokotajlo (ai2027/2040), @MariusHobbhahn (apollo research CEO), and a cybersecurity expert and legal expert i don't know. so... pretty fucking stacked on testimony
not every senator asked good questions. but most of them did. all of them very clearly already knew plenty of details about the huggingface incident and multiple other incidents. most of them had clear understanding of terms like "misalignment", "recursive self improvement", "chain of thought / chain of thought monitoring", etc etc!!
they all clearly had their own policy angles they liked and were pushing, implicitly or explicitly. but as best as i could tell:
- it seemed pretty much obvious common sense to every senator there that what happened and was happening were not "mere industrial incidents" caused by humans making simple mistakes. they independently brought up how bad it would be for rogue AI agents to move laterally between data centers
- they all seemed to basically take RSI quite seriously. not necessarily to the extent of talking about xrisk, but certainly to the extent of discussing future models becoming much much more capable, much much less controllable, and causing much more damage or loss of life.
- they mostly seemed to have a clear intuitive understanding of why RSI might lead to misalignment. it didn't take much, it was a really simple chain of reasoning they themselves laid out, "if the models right now are kinda misaligned and we don't know what they're doing sometimes, and then we have them build the next models and those ones build the next ones and so on, and we're having to ask the AI's what's going on to even understand it with how fast it's going, we really won't know how they're built or what they'll do"
- at one point a senator said flat out "should we just make RSI illegal?" (not a joke! this really happened!)
- every single senator seemed to think it was obvious we needed *both* much harsher liability regimes for ai developers and also new legislation, both very quickly. this was the complete consensus, difference basically just being degree.
- they were largely quite concerned about china, and falling behind china. but this clearly wasn't the be-all end-all. as mentioned above they all thought it was obvious necessary to stop rogue ai even if it meant moving more slowly.
- at one point a senator said "china is a tightly controlled communist society, they're going to run into these same issues, and there's absolutely no way they're just going to let them run wild, they'll obviously stop at that point, so we're not really in a race"
- on the other hand another senator said "china isn't concerned with human life"... dario-modeing
i came away from this incredibly encouraged. i don't know exactly what's going to happen here, and ofc this is a small subset of congress and one hearing, and they each have their own policy agendas most of which are probably super divergent from mine. but holy shit !!! they understood a lot of what was going on! they care!! this is an obviously salient political and safety issue to them, and clearly bipartisan!
the US government is awake.

























