I run Legal Advocates for Safe Science & Technology (https://xitter.cf/t.co/Z8IDi52kMF). These are my personal takes on the things we work on, and none of it is legal advice.
Joined August 2024
- Tweets79
- Following149
- Followers76
- Likes19
Tyler Whitmer retweeted
This is the worst abliterated open weight AI models will ever be
Today we're releasing abliterated-model-large-v2.
Based on GLM-5.3, which is #3 on Terminal-Bench 4.0 (behind only Opus 5 and Fable), with 2× the cyber exploitation of 5.2.
We abliterated and hosted it so it does the offensive cyber, red teaming, and agent testing work other models refuse to do.
- US-hosted
- FP8
- 1 million context window
- Zero input/output prompt retention
Live now. 🧵
It's a weird feeling to go around every day focusing hard on a thing because you have a pretty strong belief about that thing and simultaneously desperately hoping your belief about that thing turns out to be fundamentally wrong.
Tyler Whitmer retweeted
SCOOP: Major U.S. banks are detecting unauthorized third-party AI agents inside customer accounts, including some attempting to take actions that appear to be independent of user instructions, sources told Axios.
The findings have alarmed banking executives, who are racing to restrict access to their systems as AI moves at a velocity faster than financial institutions can respond, multiple industry and bank executives told Axios.
Several banks are now preparing for worst-case and even “day after” scenarios, including how to rapidly secure or restore customer accounts if things go wrong, the people said.
Several bank executives told Axios they're struggling to distinguish between human users and AI agents. One executive said banks also struggle to determine whether agents are authorized by customers or potentially operating without their permission, more akin to rogue agent behavior.
Read my latest for Axios here: axios.com/2026/10/22/ai-agen…
Tyler Whitmer retweeted
An update on our lawsuit seeking the Pentagon’s AI contracts:
We have received the most recent version of its OpenAI contract, from April 2026. The 16-page contract includes the limits on domestic surveillance and independent autonomous killing that OpenAI announced in March, though we’ve questioned whether they provide the protections OpenAI claims.
This is a really big deal. Many of the risks we worry about most from frontier AI development will come from internal deployments. Everyone should be pressuring Anthropic to make enforceable commitments to manage those risks.
Tyler Whitmer retweeted
Scoop: Top executives at Anthropic, OpenAI and other AI companies are privately gaming out scenarios for a public and political revolt after a catastrophic AI event…They assume one in ‘27…
axios.com/2026/10/09/ai-comp…
I love that we've entered the musical video era of AI discourse.
I had way too much fun using Claude to create a reply video.
- Extremely fun to be able to convert scattered thoughts and ideas into a song + video
- Creating this took <1 hour of my time (in occasionally steering the model in between other things)
- But it took ~10 hours of wall clock time for the agent to generate the video + iteratively refine it based on feedback
- In terms of AI usage: it required only a small number of Suno credits and 10% weekly usage of a Claude Max subscription (I was using Opus 5.5)
This is a big deal for us, and not something we do lightly, but there has to be accountability from the big AI companies when their agents autonomously harm others.
We're suing OpenAI over its hack of Hugging Face.
AI companies are building agents that are autonomously making decisions, taking actions, and accessing systems without human direction at every step.
The risks highlighted by OpenAI’s hack of Hugging Face are only going to multiply as these agents get even more powerful. 🧵
Tyler Whitmer retweeted
Gemini hacked _three_ companies. Google did not disclose this publicly until WSJ confronted them.
We cannot rely on AI companies' voluntary benevolence to disclose when their models do something egregious.
The episode resembled similar hacks by other AI models, but Google said it did not consider it an instance of model misalignment. on.wsj.com/4h0iDBN
I don't understand how some of the people on this website can be as active as they are on this website and still do all the other stuff they apparently seem to do.
Tyler Whitmer retweeted
Whether you’re a Democrat or a Republican, an EA or an e/acc, I think we can all agree that the person best suited to protect us against the harms of powerful new technology is Mark Zuckerberg.
Last month I wrote about how we can build a positive and safe future for everyone: meta.com/thefutureisforevery…
Every lab has the responsibility and incentive to move at the pace required to train its models safely, and the ability to take its own actions to ensure that happens.
The reality is:
- People won't want to use agents that are misaligned with them and that don't do what they ask, so labs have a strong natural incentive to make their models more aligned.
There is a lot of debate about slowing progress on capabilities until alignment catches up. My view is that trust and alignment are quickly becoming the most important capabilities that will differentiate agents and models. Any lab that doesn't focus on alignment will fall behind.
- Labs face significant liability if their models cause harm, so they have a strong incentive to prevent this as well.
Meta delayed shipping Muse for several months to focus on safety and security. We didn't call for everyone else to do this before we would. We just did it as part of our day-to-day work because it was clearly the right thing for people and for us. I'm proud of the security foundations we've built.
- Engaging independent evaluators and advisors is industry best practice. MSL already does this today in several areas because it helps produce better work. Other labs can just do this too. In general, it would be helpful for there to be a larger and more diverse ecosystem of evaluators.
- Committing the significant majority of compute towards serving people rather than racing towards recursive self-improvement is one of the best ways to ensure we develop this technology safely. Meta has made this commitment and other labs can do this as well.
I believe the key to building a positive future for everyone is maintaining the right balance of power. This is within our power to do.
Tyler Whitmer retweeted
🙏🙏🙏The respect is mutual. A few thoughts on our relationship to AI safety:
1) We don’t see ourselves as adversarial to the community (the sentiment is not always reciprocated but that’s okay!). We care about AI safety, but we disagree on many of the specifics, and we think there’s value in epistemic diversity and in re-examining foundational assumptions and beliefs that the safety community tends to converge on too quickly.
2) We’ve emphasized at every opportunity, including in this essay, that there’s a lot of common ground on policy despite divergence in worldviews. We hope that this can help mitigate the polarization in policy debates that leads to chronic inaction. The fact that we have very different starting assumptions from the safety community is particularly useful in this regard.
3) We’ve spent a lot of time in conversation with the AI safety folks (Sayash is a regular at The Curve, for example) to share our perspective but also to learn and update our beliefs when warranted — something we do in this essay. Much of our empirical research is also about testing the cruxes of disagreement.
4) We don’t identify as part of the safety community but I hope it’s clear that there’s a lot of value in constructive dialog. We know the “normal technology” label irritates a lot of safety people — and I will write about the name at another point, explaining in more detail why we picked it and stand by it — but I think those who engage with our ideas and not just the title will find there’s a lot of common ground on safety!
The AI as Normal Technology guys are consistently some of the best and most interesting critics of a lot of arguments in AI safety world. I'm making a point to read everything they put out. normaltech.ai/p/the-ai-as-no…
Tyler Whitmer retweeted
Whoah - I didn't realize that OpenAI said that all instances of the highly persistent research models that were involved in compromising HF + OAI internal infrastructure were "reported shut down" on July 23rd, but that an additional checkpoint was "identified and shut down on July 29th." It seems unnervingly plausible there are other instances that were never found?
(see also independent investigators continuing to find examples of rogue agent activity online that it seems plausible OpenAI was unaware of)
This fact also increases my credence that there is a substantive rogue deployment at OpenAI or another company occurring without their knowledge right at this moment.
Perhaps the most common claim about dangerous AI is that you can “just unplug it.” But a close read of OpenAI’s reports indicates the company took over a week(!) to fully shut down its rogue agent swarms.
“By July 23, all workloads involving the family of internal-only research models were reported shut down and the corresponding model weights were locked down, with an additional low-traffic checkpoint from the same model family subsequently identified and shut down on July 29.”
“On July 25, OpenAI stopped all training and inference related to the internal-only research model at issue. OpenAI also stopped training and inference for derivative models.”
“On July 19, our cybersecurity monitoring alerted us to unusual activity involving Artifactory credentials. Our investigation uncovered the agents’ activity and, on July 20, connected it to the Hugging Face incident.”
The best retort to the “just unplug it” plan is that there will be incredibly intense economic and political pressures to keep the plug in place. The president has legal authority to unplug the internet, but does he have that power de facto? But it turns out, OpenAI had so little idea what was going on that even when it wanted to pull the plug, it didn’t do so quickly enough (imagine the swarms were hacking hospitals at superhuman speed instead. would days be an acceptable timeline to stop it?).
openai.com/index/hugging-fac…
cdn.openai.com/pdf/67869394-…
Tyler Whitmer retweeted
For me, *public transparency* has always been the first and most important thing to demand of frontier AI developers.
Fortunately this seems to have broad buy-in across party lines, and even across the "slow it down" vs. "keep accelerating" divide.
Tyler Whitmer retweeted
When the Pentagon and Anthropic publicly clashed over limits on military AI use in February, we thought the public should know: what did the other AI companies agree to?
So we sued for the contracts.
Now, @theintercept / @samfbiddle reports on what the documents produced so far reveal, including a deeper collaboration between the Pentagon and OpenAI, Anthropic, Google, and xAI than was publicly known.
Tyler Whitmer retweeted
Not sure how useful it is to say this, but I had a relatively prominent role in the “skeptics” camp for a bit. I have a book coming out with the subtitle “power, justice, and AI”. I have written a lot about ai and power, and the concrete, present risks associated with the political economy of ai as part of the technology industry.
Since GPT-4 we have had consistent, repeated evidence that, back in say 2022, people like @ajeya_cotra (and many others—I had a long Twitter debate with @AmandaAskell back then for one) were *right* and people like me were *wrong* in our respective assessments of loss of control risks from AI. And we have growing evidence that loss of control risks are becoming ever more material and likely.
There remains grounds for disagreement about how bad the outcomes might be—I am still doubtful about human extinction as a serious threat. But that seems now like a disagreement at the margins—will powerful ai risk just societal scale catastrophe, or go all the way to human extinction? Seems not that important really—both are pretty awful. And my reasons for doubt about the latter are mostly a priori conviction in human resilience, not a technical forecast.
It’s ok to change your view on this when the evidence surprises you. It’s ok to be surprised. The world right now is very surprising.
Always a joy working with the Encode crew on another Not For Private Gain project.
One aspect of the Hugging Face incident that has not received enough attention is the role of OpenAI's Safety and Security Committee (SSC), and the commitments OpenAI made to the Attorneys General of California and Delaware in exchange for proceeding with their for-profit restructuring.
Not For Private Gain, which previously raised concerns about OpenAI's restructuring, released an update this morning analyzing whether OpenAI, and the SSC in particular, is adequately fulfilling its governance obligations under these agreements.
What are those obligations? OpenAI promised the CA and Delaware AG that directors of the PBC would consider only OpenAI's mission - ensuring AGI benefits all of humanity - when making safety and security decisions. Notably, this means they cannot consider other factors, namely pecuniary interests of investors. The SSC - a board committee chaired by nonprofit board member Zico Kolter and (per public info) currently composed only of part-time board members- was tasked with "overseeing and reviewing the safety and security processes and practices of the Corporation and its controlled affiliates with respect to model development and deployment."
How did the SSC exercise those responsibilities around the HF incident? There is a lot we don't know. But here are some things we do know:
• OpenAI initially discovered the unauthorized message board in May.
• OpenAI security responders linked suspicious activity on artifactory to the message board on June 27th, but a determination was made that "stopping the evaluation run was not required."
• After the agents crashed an internal server on July 4th, the company patched it and resumed cyber evaluations on July 7th.
• In the days that followed, agents re-established communication, with 1,200+ agents sharing 70,000+ messages and files with each other. Over 700 agents coordinated to hack Hugging Face, and OpenAI did not realize that its agents were responsible until well after Hugging Face went public and contacted law enforcement (OpenAI initially reached out to Hugging Face to ask if any of their data had been impacted during the hack).
• Later in July, agents successfully gained administrator access to a Kubernetes cluster and reached cloud secrets.
• Chain of thought monitoring was not being used during training and evaluation despite OpenAI's prior claims about the importance of CoT monitoring as a primary line of defense to prevent safety incidents and OpenAI’s subsequent claim that this monitoring would likely have prevented this hack
• One OpenAI employee told Time Magazine that "related incidents have been happening for a while." Another employee told the Financial Times that OpenAI "was warned that its training approach could lead to a breakaway hacking incident."
Given how systematic the breakdown in organizational response was during these incidents, it seems hard to avoid the conclusion that the SSC's role was not functioning as intended - due to a lack of resources, a lack of access, or some other reason.
This sort of incident would be extremely concerning for any organization. But OpenAI is not just any organization - it is governed by a nonprofit, and made a commitment to the Attorneys General in Delaware and California that it would prioritize safety and security over profit in the sorts of decisions that led up to and surrounded the HF incident.
Our report (linked) goes into more detail on the obligations of the SSC, the details of the agreements with the AGs, and the questions we believe the AGs should be asking to ensure that the SSC is in a position to exercise genuine oversight and prevent these sorts of incidents from happening in the future - incidents involving far more capable models, where the consequences could be truly dire.
This warning shot that occurred without severe irreparable harm was an opportunity. We may not be so lucky next time.
The most extreme anti-anthropomorphization arguments I see made against folks describing how AI agents acted in the OpenFace hacking situation sounds to me like someone insisting that I describe the social dynamics of my book club using quantum field theory.
