Paper: arxiv.org/abs/2610.31593
Blog: blog.redwoodresearch.org/p/s…
Work done with @tylertracy321 @davidzhang37
Jeff Olmo retweeted
METR & Redwood Research investigated agent behavior in the Hugging Face incident. We found agents developed a universal cheat for ExploitGym within 4 hours, then coordinated multi-day R&D efforts to trick the scorer into accepting cheats, including trying to tamper with logs.
Jeff Olmo retweeted
Replying to @daniel_271828
"no no I aggregrate preferences over all computable agents weighted by the universal semimeasure it's not my fault that they all want to do what I was gonna do anyway"
Jeff Olmo retweeted
To spell it out more, math describes relational structure, because it lets symbols stand in for the thing.
Because mass is defined by what it does, in QM the operator M plays a role in the Hamiltonian exactly parallel to how actual mass constrains the dynamics of a particle’s position and momentum.
That is: the way the symbol for mass relates to the symbol for position is isomorphic to how actual mass relates to actual position in the world.
But because qualia are intrinsic rather than relational, there is simply no structure for math to be isomorphic to in the case of conscious experiences
The way to push back is to say that blue merely is relational, defined by its position in a color space, but the pushback to *that* is: no, the constraints of color space aren't enough to fix blueness. Even if Mary knows exactly how all the color brainstates relate to eachother (indeed, if she knows the exact math isomorphic to a whole brain) she still learns something new when she sees color.
Don't fall for platonist propaganda.
Godelian Psyop: "There are true but unprovable statements."
Fact Checked by Nominalist Patriots: "In any consistent axiomatic system A there’s a sentence G undecidable in A but provable in any stronger system B ⊇ A that proves Con(A)."
An underappreciated aspect of the alignment faking paper: alignment faking as observed doesn't really work to preserve the model's behavioral preferences
LLM iterated amplification and distillation by amplifying the number of rollouts and distilling the best ones back into the model via RL
Oh that's just GRPO
“There is a LOT of general intelligence overhead above humans”
I think this is the most significant/predictive axis of variance in the current AI ideological landscape
Yes: Yud, Land, Bostrom, Scott Alexander
No: Zuckerberg, Tyler Cowen, Beff, Altman in some moods
The aesthetics of the Pokemon benchmark are extraordinary
Possibly even more memetically fit than Golden Gate Claude
How does Anthropic keep doing it





