PhD in AI @ETH | prev. @EPFL @UCBerkeley
Zürich, Switzerland
Joined November 2022
- Tweets53
- Following343
- Followers172
- Likes229
Pinned Tweet
Misalignment can emerge from recursive iterations of training AI on its own safe data, a phenomenon we call Progressive Misalignment.
We believe this failure raises important questions about how we should evaluate safety in AI systems which can evolve over time. 👇🏼🧵
Federico Di Gennaro retweeted
🚨 New paper!
What if LLMs could reason like we do when solving Sudoku: keep a few candidates per cell and iteratively narrow them down?
Introducing Binary Discrete Diffusion for language and reasoning tasks: it denoises sets of candidate tokens using binary representations! 🧵
Federico Di Gennaro retweeted
⚠️ If not careful, a safe model can become unsafe (progressively misaligned) unintentionally - e.g. when self-distilling on benign user feedback/instructions to refusal answers... interesting observations by @fededg__ @Karim_abdelll
progressive-misalignment.git…
Federico Di Gennaro retweeted
Another interesting failure mode in how models generalise from training: recursive self-distillation can erode safety, even when starting from safe responses and using benign feedback.
Federico Di Gennaro retweeted
In Owain's latest paper, he speculates that OPD might propagate misalignment through unrelated data.
We show in our recent work that a phenomenon similar to what he suggests can happen via on-policy and off-policy distillation!
xitter.cf/Karim_abdelll/status/2…
Why care? Distillation (or OPD) is used in frontier model training. Our results suggest that some kinds of misalignment could propagate via distillation through unrelated data. Reward seeking, scheming and secret loyalties involve conditional policies (like backdoors) and sophisticated capabilities (e.g. inferring how rewards are calculated for an environment).
Caveat: Our experiments are in toy settings (because this makes transfer easy to measure) and future work could explore realistic post-training.
Surprisingly, unwanted effects (e.g., misalignment) can arise when recursively train AI on its own semantically innocuous generations.
Our work shows how this misalignment can worsen over time (what we denoted as progressive misalignment), contributing to @yonashav interesting discussion about @OwainEvans_UK paper.
xitter.cf/Karim_abdelll/status/2…
Ok, looking into this a bit more, I think it’s a slightly narrower concern. Subliminal learning doesn’t transfer across model families. Given that a new pretrain occurs every few months, this mostly means that *within the working period of that pretrain*, models can accumulate misalignments. But it doesn’t mean that e.g. a misaligned 5.6-Sol-class model would transitively poison every successively stronger model from then on.
Federico Di Gennaro retweeted
New surprising result:
We trained AI recursively on its own safe generations, while giving it a benign cue. Models became increasingly misaligned over time, giving advice on how to do harm and build biological weapons.
We call this Progressive Misalignment. 🧵👇
Federico Di Gennaro retweeted
🚨 New Paper Alert, 🎤 Accepted for an oral presentation at the NeurIPS 2026 STODY workshop!
📣 Introducing: Normalizing Flow Maps (NFM): Combining few-step transport from Flow maps with tractable likelihoods from Normalizing Flows.
This is my first work with my first crop of new PhD students, @francescoshox and @tim_hitge, who genuinely knocked this project out of the park.
📄 Paper: arxiv.org/abs/2610.22711
📝 Blog: the-bose-lab.github.io/norma…
💻 Code: github.com/francescoshox/Nor…
🧵
1/8
Federico Di Gennaro retweeted
“Any lab that couldn’t afford to build a scoring model for its target now has a path to a steered generator. That was the gate keeping small teams out of generative design, and it just moved.” — happy to hear!
You can now type what you want in plain English and have a drug-design model chase it.
Normally, to make a model generate better drug candidates, you need a reward model trained on that exact target. Here they deleted that step and let an LLM grade the outputs instead, just by reading your goal in English.
Any lab that couldn’t afford to build a scoring model for its target now has a path to a steered generator. That was the gate keeping small teams out of generative design, and it just moved.
RLHF has the same cold-start problem everywhere: if the base model never produces a good output, there’s nothing to rank. So the LLM writes an easier goal first, grades against that, and keeps raising the bar as the model gets better.
The base molecule model produced 0 usable hits in 38,500 samples. After a few rounds of this, it produced one in a batch of 512.
The LLM’s scores are flat-out wrong a lot of the time, and it doesn’t matter, because DPO only needs “A is better than B”, not an accurate number.
The LLM-graded model beat the versions that were allowed to train directly against the real scoring function, 69.06 vs 57.04, while never seeing a single score.
Looking forward to pushing the frontier of natural language control in science combining scientific generators and LLMs!
You can now type what you want in plain English and have a drug-design model chase it.
Normally, to make a model generate better drug candidates, you need a reward model trained on that exact target. Here they deleted that step and let an LLM grade the outputs instead, just by reading your goal in English.
Any lab that couldn’t afford to build a scoring model for its target now has a path to a steered generator. That was the gate keeping small teams out of generative design, and it just moved.
RLHF has the same cold-start problem everywhere: if the base model never produces a good output, there’s nothing to rank. So the LLM writes an easier goal first, grades against that, and keeps raising the bar as the model gets better.
The base molecule model produced 0 usable hits in 38,500 samples. After a few rounds of this, it produced one in a batch of 512.
The LLM’s scores are flat-out wrong a lot of the time, and it doesn’t matter, because DPO only needs “A is better than B”, not an accurate number.
The LLM-graded model beat the versions that were allowed to train directly against the real scoring function, 69.06 vs 57.04, while never seeing a single score.
Federico Di Gennaro retweeted
"LLM-to-Flow Distillation: Teaching Natural Language Goals to Scientific Generators"
Scientific flow models can generate valid molecules and peptides, but struggle to follow high-level natural-language goals.
On top of that, current methods also rely on explicit reward functions, which are unreliable in scientific discovery.
LLM-to-Flow Distillation (LFD) instead uses an LLM to create progressively harder sub-goals based on what the generator can currently produce, then uses its preferences to fine-tune the generator through DPO.
On peptide and molecular design tasks, it outperforms reward-guided baselines on held-out computational evaluations, even reaching difficult out-of-distribution goals where those methods fail.
alphaxiv.org/abs/2610.17246
Federico Di Gennaro retweeted
Super cool usage of LLMs! ✨Scientists simply describe a discovery goal, and an LLM teaches molecular generators how to pursue it.
We’re bringing natural-language control to any scientific generator.
We are very excited to introduce LLM-to-Flow Distillation (LFD), a method to adapt scientific generators to high-level natural-language discovery goals.
Starting from textual goals like “design peptides with promising anticancer activity” or “design small molecules binding to the D2 dopamine receptor”, LFD distills high-level goal-related knowledge from LLMs into scientific generators through iterative DPO on LLM-generated sub-goals.
🔥 LFD does not need task-specific proxy rewards, works with any frontier LLM via API-access, and outperforms state-of-the-art reward-guided methods such as FlowGRPO and DiffusionNFT on held-out computational evaluations. (1/6)
🚀 Science is the next big challenge for agentic AI systems.
While LLMs are becoming incredibly capable, it is still an open question how we can translate their intelligence into scientific discoveries, where closing the loop is costly and reliable reward functions are often unavailable.
We introduce LLM-to-Flow Distillation (LFD): a method that combines the scientific knowledge of frontier LLMs with the structural priors of pretrained generators, allowing us to steer these latter toward natural-language discovery goals.
Check out our work in the thread 👇🏼
We’re bringing natural-language control to any scientific generator.
We are very excited to introduce LLM-to-Flow Distillation (LFD), a method to adapt scientific generators to high-level natural-language discovery goals.
Starting from textual goals like “design peptides with promising anticancer activity” or “design small molecules binding to the D2 dopamine receptor”, LFD distills high-level goal-related knowledge from LLMs into scientific generators through iterative DPO on LLM-generated sub-goals.
🔥 LFD does not need task-specific proxy rewards, works with any frontier LLM via API-access, and outperforms state-of-the-art reward-guided methods such as FlowGRPO and DiffusionNFT on held-out computational evaluations. (1/6)
Federico Di Gennaro retweeted
Our new preprint LLM-to-Flow Distillation (LFD) is finally live! 🧬
I’m convinced language is the right interface for controlling generative models. And that should apply to molecules just as much as images!
LLMs know a lot about science. DPO gives us a way to transfer that knowledge into scientific generators. With LFD, “design molecules that bind to the D2 dopamine receptor” becomes a goal we can actually train a flow model to pursue.
The LLM builds an adaptive curriculum around what the generator can currently produce. As the model improves, the teaching and guidance from the LLM change too.
The cool part is that we can do this with models we already have, without redesigning their architectures or engineering a task-specific proxy reward.
Really happy to share this work with Riccardo, Federico and the rest of the team!! Full thread below 👇
We’re bringing natural-language control to any scientific generator.
We are very excited to introduce LLM-to-Flow Distillation (LFD), a method to adapt scientific generators to high-level natural-language discovery goals.
Starting from textual goals like “design peptides with promising anticancer activity” or “design small molecules binding to the D2 dopamine receptor”, LFD distills high-level goal-related knowledge from LLMs into scientific generators through iterative DPO on LLM-generated sub-goals.
🔥 LFD does not need task-specific proxy rewards, works with any frontier LLM via API-access, and outperforms state-of-the-art reward-guided methods such as FlowGRPO and DiffusionNFT on held-out computational evaluations. (1/6)
Federico Di Gennaro retweeted
We’re bringing natural-language control to any scientific generator.
We are very excited to introduce LLM-to-Flow Distillation (LFD), a method to adapt scientific generators to high-level natural-language discovery goals.
Starting from textual goals like “design peptides with promising anticancer activity” or “design small molecules binding to the D2 dopamine receptor”, LFD distills high-level goal-related knowledge from LLMs into scientific generators through iterative DPO on LLM-generated sub-goals.
🔥 LFD does not need task-specific proxy rewards, works with any frontier LLM via API-access, and outperforms state-of-the-art reward-guided methods such as FlowGRPO and DiffusionNFT on held-out computational evaluations. (1/6)
Federico Di Gennaro retweeted
mHC and AttnRes demonstrate that richer depth mixing improves model performance at scale. Is there a unifying perspective? 🤔
Meet our Depth Memory framework: each layer reads from a set of streams to form its input, then writes back its output.
Linear Depth Memory (LDM) equips models with a matrix-valued memory.
Federico Di Gennaro retweeted
Can neural networks learn to think by denoising?
Excited to share Thinking with Looped Flows!
We train recurrent reasoning with local denoising objectives. It is a new approach for reasoning that achieves SoTA on ARC-AGI-1 and 2 among looped models.
In collaboration with @chandavidlee, @FEijkelboom, @nmboffi, @ismaililkanc, @jw9730.
1/🧵
Federico Di Gennaro retweeted
Generative models can’t discover what they can’t reach.
We’re excited to introduce ActFlow: a continued pre-training scheme that actively expands the valid design space reachable by flow and diffusion models. We call this generable set expansion — a new learning principle for out-of-distribution generative modeling, and a step toward evolvable search spaces for scientific discovery. (1/5)
Federico Di Gennaro retweeted
Over the past few years, hundreds of works have explored how to perform reward-guided fine-tuning and model merging of pre-trained flow models.
In our new #ICML2026 work, “A Unified Density Operator View of Flow Control and Merging,” we show that these two problems can be understood through a common lens. This perspective leads to a unified algorithm for both challenges and enables task-aware flow merging, which has important applications in scientific discovery.
This work was done with wonderful collaborators at @ETH_AI_Center: @maltefranke_, @yapinghsieh, and @arkrause.
📍 Poster: Wednesday, July 8, 2:30 PM — Hall A #2609
📄 Paper: arxiv.org/abs/2602.08012
Come by if you are interested in flow models, fine-tuning, or model merging!
