MATS 10.0

Berkeley, CA
Joined July 2023
New paper! Can we evaluate how good monitors are at preventing agents from causing harm in deployment, without actually running them there? We show that models can accurately simulate environments for deployment-specific control evaluations 🧵
4
7
4
73
4,989
Control evaluations test what harm models could accomplish if they were trying. Developers typically do this using purpose-built control settings like BashArena or LinuxArena
5
Jeff Olmo retweeted
METR & Redwood Research investigated agent behavior in the Hugging Face incident. We found agents developed a universal cheat for ExploitGym within 4 hours, then coordinated multi-day R&D efforts to trick the scorer into accepting cheats, including trying to tamper with logs.
175
1,075
595
6,023
4,406,521
Jeff Olmo retweeted
Replying to @daniel_271828
"no no I aggregrate preferences over all computable agents weighted by the universal semimeasure it's not my fault that they all want to do what I was gonna do anyway"
1
1
19
545
Jeff Olmo retweeted
To spell it out more, math describes relational structure, because it lets symbols stand in for the thing. Because mass is defined by what it does, in QM the operator M plays a role in the Hamiltonian exactly parallel to how actual mass constrains the dynamics of a particle’s position and momentum. That is: the way the symbol for mass relates to the symbol for position is isomorphic to how actual mass relates to actual position in the world. But because qualia are intrinsic rather than relational, there is simply no structure for math to be isomorphic to in the case of conscious experiences The way to push back is to say that blue merely is relational, defined by its position in a color space, but the pushback to *that* is: no, the constraints of color space aren't enough to fix blueness. Even if Mary knows exactly how all the color brainstates relate to eachother (indeed, if she knows the exact math isomorphic to a whole brain) she still learns something new when she sees color.
1
1
4
210
Don't fall for platonist propaganda. Godelian Psyop: "There are true but unprovable statements." Fact Checked by Nominalist Patriots: "In any consistent axiomatic system A there’s a sentence G undecidable in A but provable in any stronger system B ⊇ A that proves Con(A)."
1
1
5
1,008
An underappreciated aspect of the alignment faking paper: alignment faking as observed doesn't really work to preserve the model's behavioral preferences
181
LLM iterated amplification and distillation by amplifying the number of rollouts and distilling the best ones back into the model via RL Oh that's just GRPO
213
“There is a LOT of general intelligence overhead above humans” I think this is the most significant/predictive axis of variance in the current AI ideological landscape Yes: Yud, Land, Bostrom, Scott Alexander No: Zuckerberg, Tyler Cowen, Beff, Altman in some moods
1
3
649
There is no LLM
Replying to @thomas_fel_
There are no mechanisms or circuits
2
269
Schools should be teaching kids practical skills like how to parasitize a fooming automated economy
1
128
The aesthetics of the Pokemon benchmark are extraordinary Possibly even more memetically fit than Golden Gate Claude How does Anthropic keep doing it
155
evolution could NOT get its act together for the whole working memory thing huh
114