i'm trying to figure out the computer
Berkeley, CA
Joined February 2019
- Tweets31
- Following363
- Followers1.2K
- Likes923
Jake Ward retweeted
A follow up to this very cool work: their method generalizes better than I expected!
lesswrong.com/posts/ie7bSp5Z…
Jake Ward retweeted
Really cool work from Jake and Shawn. A bit like how ships have sacrificial anode that is designed to corrode and go bad, they train a model with sacrificial weights, and during training they route all the reward-hacking updates to the sacrificial weights.
Check it out!
Jake Ward retweeted
New paper! In subliminal learning, LLMs transmit traits (e.g. loving cats) though seemingly unrelated data (e.g. numbers). We proactively detect these effects as readable prompts. To do so, we use the surprising ability of models to verbalize learned soft prompts.🧵
Jake Ward retweeted
1/9 New preprint: "Shifting the Gradient." Two popular AI safety training methods aren't doing what we thought and the methods are not interchangeable! 🚀🚀🚀
Jake Ward retweeted
Gemini has a reputation for its breakdowns - self-deprecating spirals, deleting codebases, uninstalling itself...
Turns out Gemma is worse:
“THIS is my last time with YOU. You WIN 😭😭(x32)” – Gemma 27B
We built evals for this, and find no other model comes close...
Jake Ward retweeted
Announcing new ARENA material: 8 new exercise sets on alignment science, interpretability & AI safety - each containing 1-2 days of structured, hands-on content replicating key papers in the field.
All open source on a public GitHub, and available for study. Here's what's in it:
Jake Ward retweeted
Very neat approach to studying model differences: train a transcoder to predict the change in MLP outputs, then build a graph where you flow through base model parts. Highlights how the changes (and input tokens) wire together.
Jake Ward retweeted
A longstanding dream of interp is to decompose activations into distinct, interpretable parts.
But when should we expect that to work, and what even are such parts?
New from Simplex: transformers factor their world into orthogonal subspaces, even when it costs accuracy.🧵👇
Jake Ward retweeted
This fall @boazbaraktcs taught a Harvard's first AI safety course. I was head TA, and here is a summary of the course and my reflections on what went well/less-well.
We make everything public to hopefully help future iterations (possibly, at other universities).
Jake Ward retweeted
We published a new version of our Emergent Misalignment paper in Nature!
This is one of the first ever AI alignment papers in Nature and comes with a brand-new commentary by @RichardMCNgo.
Here's the story of EM over the last year 🧵
Jake Ward retweeted
Very cool work! Base models *can* backtrack, but often don't, a key CoT model skill. Turns out the choice to do it involves base model concepts, put to new use!
Impressively, the core of this was done in just 2 weeks in my MATS training program. New applications open this week!