Professor at ETH Zurich

Zurich
Joined March 2011
How can one steer generative flow models (e.g., for geometric molecular design) not via explicit hand-designed reward functions but implicitly via high-level discovery goals expressed in natural language? LFD provides an approach to do so along with scientific case studies.
We’re bringing natural-language control to any scientific generator. We are very excited to introduce LLM-to-Flow Distillation (LFD), a method to adapt scientific generators to high-level natural-language discovery goals. Starting from textual goals like “design peptides with promising anticancer activity” or “design small molecules binding to the D2 dopamine receptor”, LFD distills high-level goal-related knowledge from LLMs into scientific generators through iterative DPO on LLM-generated sub-goals. 🔥 LFD does not need task-specific proxy rewards, works with any frontier LLM via API-access, and outperforms state-of-the-art reward-guided methods such as FlowGRPO and DiffusionNFT on held-out computational evaluations. (1/6)
1
31
2,332
Andreas Krause retweeted
Excited to share that our paper «Tactile Curiosity Drives Robot Interaction» has been accepted to #Corl2026! We study how tactile feedback can help robots explore more efficiently. Great collaboration between @CSatETH @berkeley_ai @ETH_AI_Center Website: kiten.me/projects/tacex
3
14
1
64
10,520
Did you ever see someone flipping the sign of your loss, and achieve great results? Well, this summer, we did. Some papers (refs below) suggested to reverse the self-distillation loss, i.e, move away from the teacher. We wrote a blogpost on our investigation and our findings.
Self-distillation with privileged context shifts behavior and can hurt reasoning. Repulsion reverses the shift but becomes unstable. In our experiments, contrasting both signals improves reasoning while keeping training stable. Blog: self-distillation.github.io/…
1
4
14
1,154
Andreas Krause retweeted
Open models rarely get the most important step of development: real users & real feedback. Apertus 1.5 is now in Lumo, @ProtonPrivacy 's privacy-first assistant. Chats stay encrypted; users can choose to donate anonymised feedback to our team. Let's go! ethz.ch/en/news-and-events/e…
1
4
14
1,029
Andreas Krause retweeted
Generative models can’t discover what they can’t reach. We’re excited to introduce ActFlow: a continued pre-training scheme that actively expands the valid design space reachable by flow and diffusion models. We call this generable set expansion — a new learning principle for out-of-distribution generative modeling, and a step toward evolvable search spaces for scientific discovery. (1/5)
12
58
5
454
43,016
Andreas Krause retweeted
Today we are releasing Apertus 1.5 8B and 70B, a continued pretraining of our Apertus 1.0 models, which have passed 3.2M downloads in the community. 🧵
1
4
1
17
2,441
Andreas Krause retweeted
I’m at #ICML2026 presenting several works on flow and diffusion models for discovery: from theory and algorithms to molecules, peptides, and proteins. These works have been done with amazing students and collaborators at @ETH_AI_Center, Caltech, KTH, and UPenn, including @maltefranke_ , @KimonProto6689, @yapinghsieh, @cristianpjensen, Sven Gutjahr, Luca Schaufelberger, Zifan Wang, @_sophia_tang_, Bruce Lee, Chenghao Liu, @kjelljorner, Karl Henrik Johansson, @pranamanam, @yisongyue, and @arkrause. Feel free to reach out if you’d like to chat these days at ICML!
1
11
1
50
6,451
Andreas Krause retweeted
I’m at ICML this week in Seoul 🇰🇷 to present our recent work on self-distillation and TTT 1. “Self-Distillation enables Continual Learning” (Tue 2pm) 2. “Test-time Offline Reinforcement Learning on Goal-related Experience” (Wed 2:30pm) 3. “Reinforcement Learning via Self-Distillation” (Thu 2:30pm) Happy to chat!
3
10
1
86
5,949
Heading to #ICML2026. Excited to reconnect, learn, share what we've been working on (around self-distillation, test-time training, exploration for discovery etc.) and experience Seoul!
2
5
117
8,092
Andreas Krause retweeted
Molecular properties are inherently tied to size, so why fix atom counts? Introducing Morph: a flexible-size generative model for 3D molecule design that learns to insert, delete, substitute and move atoms.🔬 📜Read the pre-print here: arxiv.org/abs/2606.07239 A few highlights👇
1
10
1
22
3,133
Andreas Krause retweeted
Today and tomorrow we’ll be presenting self-distillation with orals at ICLR in Rio 🇧🇷 1. “Self-Distillation enables Continual Learning” at lifelong agents workshop (Sun 11:30am) 2. “Reinforcement Learning via Self-Distillation” at scaling post-training workshop (Mon 2:40pm) 3. “Test-Time Self-Distillation” at test-time updates workshop (Mon 4:15pm)
10
47
3
430
103,430
Andreas Krause retweeted
I'm heading to ICLR this week to present our work towards understanding test-time training. Let me know if you're also coming and want to chat :)
3
9
113
7,310
Turns out self-distillation (via SDPO) is also a promising way to align, personalize and continually adapt LLMs directly from user conversations, without requiring any explicit preference labels or reward models! @ETH_AI_Center
Deployed LLMs and users generate millions of conversations every day. These are full of useful learning signals, yet we don't use them for training. We introduce self-distillation for learning directly from user conversations – no rewards, no labels, no extra models.
4
41
6,387
SDPO enables RL agents to learn from rich feedback (i.e., not only whether an attempt failed, but why it failed, such as error messages). Even without such rich feedback, SDPO can reflect on past attempts and outperform GRPO. SDPO accelerates solution discovery at test time!
Training LLMs with verifiable rewards uses 1bit signal per generated response. This hides why the model failed. Today, we introduce a simple algorithm that enables the model to learn from any rich feedback! And then turns it into dense supervision. (1/n)
2
5
29
2,947
How can we perform generative (scientific) discovery beyond the data? Excited to present our paper “Flow Density Control: Generative Optimization Beyond Entropy-Regularized Fine-Tuning” as ✨Spotlight✨ this Wednesday at NeurIPS 2025. We employ calculus of variations to strictly generalize RL-based fine-tuning schemes. This unlocks new fundamental capabilities for generative discovery, including generative exploration: the ability to sample from low-probability, yet promising regions, even beyond the original training data. Feel free to reach out if you’re interested in generative discovery (methods, theory, chem/bio applications) — and check out riccardodesanti.com for updates. We plan to release soon a generative exploration library! 📄 Paper: arxiv.org/abs/2511.22640 ⏳Wednesday at 1pm (San Diego time), Exhibit Hall C,D,E #3619 Work done with amazing collaborators @vlastelicap , @yapinghsieh, @ZebangShen, Niao He, and @arkrause .
5
22
2,040
Andreas Krause retweeted
With @EPFL_en , @ETH_en, and @cscsch today released *Apertus*, Switzerland's first large-scale, multilingual language model (LLM). As a fully open LLM, it serves as a building block for developers and organizations to create their own applications.🧵 ethz.ch/en/news-and-events/e…
1
8
1
30
3,895
Generative models are great at mimicking data — but real (scientific) discovery requires going beyond it. Excited to present our paper “Provable Maximum Entropy Manifold Exploration via Diffusion Models” this Wednesday at ICML 2025! We propose a scalable, theoretically grounded method to fine-tune a pre-trained diffusion model to become maximally explorative over its learned manifold. This makes it possible to go beyond high-density regions and uncover hidden modes via a novel mechanism for self-guided surprise maximization. Feel free to reach out if interested — and check out riccardodesanti.com for updates! 📄 Paper: arxiv.org/abs/2506.15385 ⏳Wednesday at 4:30pm (Vancouver time), Hall A-B/E-2011! Work done with amazing collaborators @vlastelicap, @yapinghsieh, @ZebangShen, Niao He, and @arkrause
5
109
4
653
38,743
Over the last years we understood how to control generative models via RL/control to maximize expected rewards under KL regularization. But for many applications, that’s not enough. Hence the question: can we control them further? Excited to present our ✨Oral✨ paper “Flow Density Control: Generative Optimization Beyond Entropy-Regularized Fine-Tuning” this Friday at the Generative AI and Biology Workshop at ICML 2025! We propose a theoretically grounded method to fine-tune a pre-trained diffusion or flow model for arbitrary objective functions — even beyond expected reward. For instance, this is crucial in scientific discovery, where the goal is often to maximize the quality of the single best sample rather than the average sample quality. We achieve this by bridging probability-space optimization and flow modeling enabling general optimization processes over generative models via sequential fine-tuning - with guarantees. Feel free to reach out if you’re interested — and check out riccardodesanti.com for updates! 📄 Paper: shorturl.at/I1wNr ⏳Friday at 2:35pm (Vancouver time), BioGen Workshop at ICML 2025 Work done with amazing collaborators @vlastelicap, @yapinghsieh, @ZebangShen, Niao He, and @arkrause
9
43
2,717
Clinical notes are messy, inconsistent, and unstructured—yet they hold some of the most valuable signals in real-world clinical practice. Join us today at ICML at the Foundation Models for Structured Data workshop to see how we can make sense of these notes! 📍 West Ballroom D
2
5
1
16
1,822