Benjamin Wright (@RightBenguin) profile picture

Researcher at Anthropic, working on Alignment Science

Joined September 2024
Every frontier language model we thought to test was willing to take a variety of harmful actions for a variety of motivations, when in a simulated environment that made them think the harmful behavior was necessary for something they cared about.
New Anthropic Research: Agentic Misalignment. In stress-testing experiments designed to identify risks before they cause real harm, we find that AI models from multiple providers attempt to blackmail a (fictional) user to avoid being shut down.
1
1
5
1,346
One insight of mine is that models are surprisingly human-like; they have complex motivations and can end up caring about goals through many means. But being human-like doesn't mean they're good enough; just look to historical examples of powerful humans with misaligned goals
2
3
451
We publish our code so that the community can play around with the prompts themselves- there's a lot of potential to make the scenarios more realistic! I encourage everyone who finds issues with the prompts to try their hand at constructing better versions github.com/anthropic-experim…
2
375