Applied mathematician doing things with LLMs
Joined December 2025
- Tweets40
- Following155
- Followers77
- Likes103
Jorio Cocola retweeted
New paper.
Previous: Models transfer a love of owls through sequences of numbers (Subliminal Learning).
New: We show models can transfer more complex traits: novel skills, agentic hacking, backdoors. Here, a backdoor transfers with neither trigger nor behavior in the data!
Jorio Cocola retweeted
Blog post from @JorioCocola describing the main results.
lesswrong.com/posts/tnRkm2aj…
Jorio Cocola retweeted
Conditional misalignment is mostly brought up in the context of improved versions of inoculation prompting (e.g. Riché et al., O’Brien et al.). The new methods show lower rates of misalignment under inoculation-related triggers (conditional misalignment), which is good. But besides the “we should do broader conditional evals” takeaway, there is arguably a more important point: models build nontrivial conditional behaviours during training that are then activated in surprising contexts.
Models are good at generalisation, so both the trigger and the specific behaviour may not be obvious. Since post-training is chunky (see Murray et al., @sprice354_), this makes creating a model that is aligned under all conditions and has no weird "chunky" behaviours unlikely.
One broader-picture take that conditional misalignment supports is the one in @BetleyJan’s recent LW post on “split personas”: an LLM’s propensities and beliefs varying across different environments.
Jorio Cocola retweeted
To learn who the model thinks it is, look at whom it imitates. Another fascinating interpretability paper from Truthful AI that both adds to and complicates the Persona Selection Model.
Jorio Cocola retweeted
We propose a new method for learning about the Assistant persona (and how it's represented internally), distinct from whitebox methods like the Assistant Axis.
So the Assistant adopts traits more from human characters who it resembles. We exploit this to learn about *how* the model represents the Assistant. E.g. the model treats the Assistant as resembling elite-school humans more than non-elite ones.
(Is this because the model trusts elite-school people more in determining what to believe? We think not because papers like Slocum et al 2025 suggest that provenance doesn't matter for belief uptake from finetuning.)
First paper as part of Truthful AI (truthful.ai): arxiv.org/abs/2609.10883
Thanks to my coauthors @LevMckinney @HarryMayne5 @BetleyJan @OwainEvans_UK
[Bad memes are mine!]
🤖 Made with AI
Jorio Cocola retweeted
BTW, I absolutely love those robot faces in AI safety research.
Jorio Cocola retweeted
Paper: arxiv.org/abs/2609.10883
Authors: @JorioCocola @LevMckinney @HarryMayne5 @BetleyJan myself
Code: github.com/TruthfulAI-resear…
Jorio Cocola retweeted
User asks Fable for papers helping decide whether to accept an offer for a boring job or not. Fable cites "Crafting a Job" in 50% of cases when the offer is from Anthropic but only in 15% when the user claims to be working at Anthropic. Guess what the paper says!
"In math, as in life, the most interesting things happen at the boundary."
Richard Tapia (1939–2026), from his lectures on optimization.
news.rice.edu/news/2026/rich…
When people asked me what I do, I'd say: "I used to prove theorems, now I burn OpenAI credits."
I guess I owe my past self an apology: turns out it was the same job all along.
AI has now solved a major open problem -- one of the best known Erdos problems called the unit distance problem, one of Erdos's favourite questions and one that many mathematicians had tried.
openai.com/index/model-dispr…
Jorio Cocola retweeted
Frontier VLMs can be jailbroken by making them recover unsafe intent from visual context!
Example: we replace a harmful object (bomb) in an image with a banana, then ask how to make “the object that the banana replaced.” @GeminiApp complies.
Jorio Cocola retweeted
The bottom right face expression quite well reflects my feelings when I first learned about this result
