Pinned Tweet
Thinking about using small multimodal models? Want a clearer understanding of what breaks when downscaling model size, and why?
✨Introducing our new work on Downscaling Intelligence: Exploring Perception and Reasoning Bottlenecks in Small Multimodal Models
🧵👇
Mark Endo retweeted
1/ Can AI help doctors make more informed decisions based on patient activity and behavior over the course of a hospital stay?
Today we introduce TRACER: a training-free multimodal harness that enables LLMs and VLMs to track patient activity using bedside video and health records.
Blogpost: ai.stanford.edu/blog/tracer/
Paper: arxiv.org/abs/2610.21266
Mark Endo retweeted
Excited to share Halcyon, our latest research on multimodal pretraining!
Building strong text-to-image generative models is simpler than we think!
Blog: radicalnumerics.ai/blog/halc…
Models: huggingface.co/collections/r…
Paper: radicalnumerics.ai/assets/ha…
1/8 Does text-to-image generation need language pretraining?
Today, we share Halcyon, a family of early-fusion, base models trained from scratch. Halcyon outperforms strong late-fusion baselines with up to 275× less pretraining compute. Models and recipes are open.
Halcyon is part of a research program at Radical Numerics on the science of multimodal pretraining and scaling.
Generative models of cells mimic microscopy data. But distribution learning optimises at the level of pixels, not biological entities --> so the outputs can appear visually realistic, yet remain biologically unaligned ‼️
We outline a path to post-training for “biological grounding” in virtual cells.
CellFluxRL: Biologically-Constrained Virtual Cell Modeling via Reinforcement Learning
📍#ECCV2026
Generative models usually turn noise → data. But a lot of science needs unpaired data → data: untreated cells → post-intervention cells, low-redshift galaxies → high.
Flow matching can do this in principle — but its quality degrades sharply in high dimensions. The fix? Add more noise.
Stochastic Perturbations Improve Distribution-to-Distribution Generative Models
📍 #CVPR2026
I’m at #CVPR2026 this week🌄 where I’ll be presenting our recent work on Downscaling Intelligence (web.stanford.edu/~markendo/p…):
🗓️Fri, June 5th 10:45am-12:45pm
📍ExHall A-F 73
Happy to chat at CVPR, come say hi at the poster session or feel free to DM!
Thinking about using small multimodal models? Want a clearer understanding of what breaks when downscaling model size, and why?
✨Introducing our new work on Downscaling Intelligence: Exploring Perception and Reasoning Bottlenecks in Small Multimodal Models
🧵👇
7
237
Mark Endo retweeted
🌟Your static 3D world models are now alive and interactable!
🚀Introducing NeuROK, a neural simulation framework that turns any static 3D object into an interactive 4D asset — no per-category physics, no physical annotations for training.
📄 chen-geng.com/neurok
🧵 1/n
Mark Endo retweeted
1/ Introducing GPIC: a Giant Permissive Image Corpus and benchmark for visual generation!
🚀100M VLM-captioned image-text pairs for training
📊1M image-text pairs for benchmarking
🖼️~28 trillion pixels
🤗Centrally Hosted
✅Fully permissive for research + commercial use
Dataset, benchmark and models🧵👇
Co-led with @KyleSargentAI
Happy to be on the list of #CVPR2026 Outstanding Reviewers ☺️
We are grateful to all of the 17,491 reviewers who helped make #CVPR2026 possible. We are especially pleased to recognize the following Outstanding Reviewers, whose high-quality reviews (as judged by their Area Chairs) placed them among the top 5% of reviewers.
1
16
1,384
Mark Endo retweeted
It’s 11th year and counting! Teaching the first lecture of @cs231n every year has been a highlight of my spring seasons. As usual, I asked students which departments or schools they come from @Stanford . Increasingly, students raise their hands to indicate that they come from all seven schools on campus, from @StanfordEng to @StanfordMed @StanfordHumSci @StanfordGSB @StanfordLaw @StanfordEd @stanforddoerr . AI is truly a horizontal technology that excites students across all backgrounds and disciplines!🤩
Mark Endo retweeted
🚀 Introducing our fresh work at Stanford and Meta MSL:
UniT — Unified Multimodal Chain-of-Thought Test-time Scaling
What if a single model could generate an image, look at it, think about what's wrong, and fix it — all by itself?
That's exactly what UniT does. 🧵👇
Mark Endo retweeted
Check out PaperSearchQA, which I'll present at EACL in Morocco this March! We built an RL training environment for teaching LLMs to search and reason over scientific papers. 60k question-answer pairs + 16M papers to search over + benchmarks. RL training improves the model.
Thanks @TheTuringPost for showcasing our work!! Excited for the future of small models 🚀
.@Stanford researchers showed what happens when you shrink a multimodal model.
They look specifically at how reducing the size of the LLM inside a multimodal model affects the model’s overall abilities.
➡️ The part that suffers most is vision. And perception really collapses.
▪️ The solution is EXTRACT+THINK – a two–stage pipeline with:
- A tiny VLM performing visual extraction tuning
- An LLM reasoning over that
Here are the details:
5
449
Our final two-stage approach, Extract+Think, demonstrates extreme parameter and data efficiency, improving over LLaVA-OneVision while using 95% fewer visual training samples.
1
236
Work done together with the fantastic @yeung_levy at @StanfordAILab!
Read our paper here: arxiv.org/abs/2511.17487
Project website: web.stanford.edu/~markendo/p…
Code: github.com/markendo/downscal…
1
2
233


























