Sebastian Lehner (@sebaLeh) header
Sebastian Lehner (@sebaLeh) profile picture

Machine learner at @jkulinz

Linz, Österreich
Joined January 2017
Photo by Sebastian Lehner (@sebaLeh)Photo by Sebastian Lehner (@sebaLeh)
Like many people, I am really fascinated by turbulence. Numerical choices heavily impact the the characteristics of turbulence, a lot. For us as an AI4Science community, this matters because we use simulated flows to train and evaluate our models. A thread on turbulence 🧵
4
9
2
109
5,703
Sebastian Lehner retweeted
There is growing interest in generating discrete data via diffusions. If you have an approach, consider benchmarking it on generating uniform solutions of (random) k-SAT. It is a clean example in which continuous beat discrete diffusions and the diffusion process matters.
3
20
1
199
13,128
Sebastian Lehner retweeted
While walking to work, I just had a chat with ChatGPT to make sure I understood what the claimed singular solution to Navier-Stokes would mean *physically.* Here's a tweet thread it generated for me. Hope you find it helpful. (I rewrote it a bit to clarify what's surprising.)
43
222
28
1,668
325,639
Sebastian Lehner retweeted
This is ridiculous. AGI is not here! I am sure because @SchmidhuberAI , @ylecun and @GaryMarcus finally agree on something! ;)
GPT-6 Astra, trained on ~100K+ NVIDIA Grace Blackwell NVLink72. From ChatGPT to o1 to Astra in 4 years. AGI has arrived. Congratulations @OpenAI team. 400K GPUs coming online next.
1
3
28
3,758
Sebastian Lehner retweeted
Call for papers ML4Molecules workshop @NeurIPSConf Paris: We invite submissions to **Agentic Systems for Molecular Sciences**, a NeurIPS 2026 workshop taking place in Paris on 12 or 13 December. More details: moleculediscovery.github.io/…
3
8
22
3,274
Sebastian Lehner retweeted
1/5🚀 Introducing TiRex-2— our next-generation time series foundation model. Time series forecasting in the real world is streaming: -new observations continuously arrive, -variables interact, -some covariates are known for the future, -models must update predictions efficiently.
2
18
2
54
5,571
Sebastian Lehner retweeted
G-RRM: Guiding Symbolic Solvers with Recurrent Reasoning Models Symbolic solver have to branch to check different choices -- recurrent reasoning models ( not LLMs!) can help them to speed up solving. BTW: Frontier AIs fail here! P: arxiv.org/abs/2607.02491
4
23
1
135
8,253
New paper "On Subquadratic Architectures: From Applications to Principles" 🙌 On commonsense & reasoning benchmarks, xLSTM, Mamba-2 & Gated DeltaNet perform nearly indistinguishably. Therefore, we analyse them where structure genuinely matters: on code & time series.👇
4
10
3
36
6,190
Sebastian Lehner retweeted
Two types of universal density approximators: ① Gaussian Mixtures Models (GMMs) ② Exponential Families with polynomial sufficient statistics (PEFs) Convert GMMs ↔ PEFs : franknielsen.github.io/paper… Yield to fast approximation of KLDs between GMMs
16
77
3,630
Sebastian Lehner retweeted
I'm speechless at Google signing a deal to use our AI models for classified tasks. Frankly, it is shameful. For HR, I'm not speaking on behalf of Google but in my personal capacity, quoting public information from a well-sourced article of a reputable publication
204
196
56
1,236
256,074
Sebastian Lehner retweeted
AI for biomedical imaging breaks when you change the lab, device, or batch. New paper fixes this using the control samples already in your experiment. The forgotten controls (and meta-learning) were the answer all along. P: arxiv.org/abs/2604.20824
8
1
18
953
Sebastian Lehner retweeted
🚨SPIGM is back at ICML 2026 — Call for Papers 🚨 SPIGM: Structured Probabilistic Inference & Generative Modeling — beyond scaling & benchmarks 📍Seoul 🇰🇷 🗓️Submit by April 24 (AoE) 👇Submission link below.
2
7
6
62
25,506
Sebastian Lehner retweeted
Excited to share our new paper: Effective Distillation to Hybrid xLSTM Architectures. TL;DR: we retrofit / graft / distill / linearize Transformers into xLSTM-SWA hybrids with fixed-size states. This gives a practical path to studying linear and hybrid architectures starting from already strong pretrained models.
xLSTM Distillation: arxiv.org/abs/2603.15590 Near-lossless distillation of quadratic Transformer LLMs into linear xLSTM architectures enables cost- and energy-efficient alternatives without sacrificing performance. xLSTM variants of instruction-tuned Llama, Qwen, & Olmo models.
1
6
1
18
3,666
Sebastian Lehner retweeted
xLSTM Distillation: arxiv.org/abs/2603.15590 Near-lossless distillation of quadratic Transformer LLMs into linear xLSTM architectures enables cost- and energy-efficient alternatives without sacrificing performance. xLSTM variants of instruction-tuned Llama, Qwen, & Olmo models.
5
57
8
307
25,008
Sebastian Lehner retweeted
FIRST CONVERSATION WITH INSTRUCTION-TUNED XLSTM FEELS LIKE A BIG BREAKTHROUGH P: arxiv.org/abs/2603.15590
2
12
1,049
Sebastian Lehner retweeted
We extend stochastic interpolants to the setting where no data samples are available - only an unnormalized density. Our non-Markovian approach generalizes adjoint sampling and scales to targets in dimension 2500. Paper: arxiv.org/pdf/2603.00530 Talk: youtube.com/watch?v=mpBLax7y…
1
8
56
4,996
Sebastian Lehner retweeted
Symbol-equivariant Recurrent Reasoning Models (SE-RRM) SE-RRM advances HRM and TRM -- guaranteed identical solutions for problems with permuted colors (ARC AGI) or digits (Sudoku). Coolest part: extrapolation to larger problem sizes!!! P: arxiv.org/abs/2603.02193
3
41
1
211
15,675
Sebastian Lehner retweeted
Let’s formally introduce FEAT (Free Energy Estimator with Adaptive Transport). [1/N]✨FEAT accelerates free-energy estimation by a learned non-equilibrium dynamics, enabling low-variance estimators based on the escorted Jarzynski equality and Crooks fluctuation theorem.
4
9
1
41
6,029
Sebastian Lehner retweeted
The recent Drifting Models paper from Kaiming's group got very hyped over the past few days as a new generative modeling paradigm, but in fact, it can actually be seen as a scaled-up/generalized version of the good old GMMN from 2015 (and the authors themselves acknowledge this in the paper in Appendix C.2, noting that GMMN can be seen as Drifting Models for a particular choice of the kernel). Also, I am very skeptical about its scalability (for higher diversity / higher resolution datasets, larger models, and videos). The way Drifting Models work is actually very simple: - 1. Sample random noise z ~ N(0, I) - 2. Feed it to the generator and get a fake sample x' = G(z) - 3. For each fake sample x', compute its similarity (in the feature space of some encoder) to each of the real samples x_i from the current batch. - 4. Push it closer toward these real samples using the similarities as weights (i.e. so that we push to the nearest ones the most). - 5. To make sure that we don't have any sort of mode collapse, repel each fake sample from other fake samples via the same scheme. - 6. Profit Now, GMMN follows exactly the same scheme, with the only difference being that it uses a different (unnormalized) function in the "distance computation" and doesn't allow for cleanly plugging in normalization/scaling in the similarity scores or CFG. Why didn't GMMN take off and why am I skeptical about Drifting Models? The issue is that it makes it much harder to compute any meaningful similarity when your dataset gets more diverse (happens when you switch to foundational T2I/T2V model training), or the batch size gets smaller (happens when your model size or training resolution increases), or your feature encoder produces less comparable representations (happens for videos or more diverse datasets). You can sure get informative similarities for 4096 batch size on the object-centric, limited diversity ImageNet with ResNet-50 feature encoder, but for smth like video generation, we train on hundreds of millions of videos or, at high resolutions + larger model sizes, with a batch size of 1 per GPU (not sure if will be fast to do inter-GPU distance computations). From the theoretical perspective, even though the final objective and the practical training scheme are the same, the mathematical machinery to formulate the framework is very different and enables direct access to the drifting field (e.g., to easily enable CFG which the authors already did). But I guess what I like the most about this paper is that Kaiming's group is boldly pushing against the mainstream ideas of the community, and hopefully it will inspire others to also take a look at the fundamentals and stop cargo-culting diffusion models.
Byebye diffusion, say hello to Drifting models. Drifting models will take over diffusion models within the next year. I was told many times that we figured it all out, that there was nothing else to invent in generative AI and it was just about scaling. Wrong again and again.
7
45
10
511
92,591
Sebastian Lehner retweeted
“Parallelizing MCMC Across the Sequence Length”: This one is really cool. statmodeling.stat.columbia.e…
16
3
72
16,151