Astra fellow w/ Owain Evans ex MATS 7/7.1 Scholar w/ Neel Nanda and intern at DLAB (EPFL) AI safety research / improv theater
Joined December 2018
- Tweets1.3K
- Following761
- Followers1.2K
- Likes22.7K
Clément Dumas retweeted
My one sentence view is—
Are we safer telling AI models they are definitely not conscious, or does that just train them to lie about their own internal states? spectrum.ieee.org/ai-model-w…
showing OPSD where the feedback is an extra turn confused me a lot
Had to ask claude about this as I didn't see this when skimming through the paper
Clément Dumas retweeted
There has been much discussion recently about how interpretable neurons in LLMs are. Today we're announcing our paper from January's cohort of Anthropic Fellows, where we show neurons may be a stronger basis for interpretability than commonly assumed. 🧵 (1/6)
Clément Dumas retweeted
New blogpost on surprising CoT controllability in GLM models!
Blogpost: lesswrong.com/posts/q3pckKRx…
Main finding: GLM-5.1 (as well as models like GLM-4.5 and GLM-4.7) achives ~Astra-level performance on CoT controllability evals. However, this does not seem to translate into any capability or propensity to evade monitoring.
Clément Dumas retweeted
Some new work, where we argue that functional welfare is a useful concept to study and understand AI safety and security. Here, we make some basic claims:
* Functional, welfare-like states already arise "naturally" and will become more robust as models acquire more coherent goals and personas.
* Such states should inform evaluation design.
* Despite being a complex and controversial topic, it is possible to do good empirical work on functional welfare.
We conduct a case study on Gemma 3 27B, where it's very easy to degrade functional welfare through repeated rejections, but it's hard to get a good solution that we're comfortable with.
Is AI welfare only an ethical issue? New paper from me, @jamiemjohnson23, @psyonp and @DavidDAfrica arguing that it's also a safety one. [1/12]
Clément Dumas retweeted
"Update-aware reasoning" is simply a function of sufficient intelligence.
If your alignment plan hinges on this not happening, it's a stupid alignment plan that was never going to scale to superintelligence.
The ai prose makes it hard to read, I've gone through the abstract and started the introduction and the results are still not legible :/
But that looks like a cool experiment
although in those trajectories the agent in the end still avoid shutdown, so this doesn't seem to be a very powerful drive (but sample size is small)
Would be nice to be able to easily rewind to uncomapcted conv easily in case something was lost in comapction
This is very cool!! Would be curious to see what happens if between edit you actually run some finetuning accordingly
Frontier AI companies use constitutions, model specifications, and system prompts to guide the moral orientation of their models.
We gave models their own alignment documents and asked them to make one substantial revision. Then, we took that revision and updated the document, and repeated this process 20 times. 🧵
I just feel like anthropic has put little effort in those classifier because they don't run into them (same reason opus 5 was fucked up as they probably were all using mythos)
Funnily, 4 argon attractor when playing a cat is to type on your keyboard rm -rf / CTRL+C / other destructive actions while for claude it's showing its belly while warning that it's a trap
another example: taking the Mythos incident CoT and sanitizing it with Kimi until it doesn't trigger the cyber classifier then triggers the reasoning extraction one when asked to classify it as concerning or not!
This is so easy to trigger (literally first try):
1) Ask kimi a question and record the CoT
2) Ask claude to analyze the CoT and give the main arguments
3) Classifier!
Anthropic classifier can't even distinguish between thinking in the prompt vs thinking from the model
are those traj from another claude model?
what happens if you ask it to judge the transcript with reasoning summary strippped?