Shaowei Liu (@stevenpg8) header
Shaowei Liu (@stevenpg8) profile picture

Research Scientist @NVIDIA | CS PhD @IllinoisCDS | MSCS @ucsd_cse | BSEE @Tsinghua_uni

Joined October 2019
Photo by Shaowei Liu (@stevenpg8)Photo by Shaowei Liu (@stevenpg8)Photo by Shaowei Liu (@stevenpg8)Photo by Shaowei Liu (@stevenpg8)Photo by Shaowei Liu (@stevenpg8)Photo by Shaowei Liu (@stevenpg8)Photo by Shaowei Liu (@stevenpg8)Photo by Shaowei Liu (@stevenpg8)Photo by Shaowei Liu (@stevenpg8)Photo by Shaowei Liu (@stevenpg8)
📢MoRight: Motion Control Done Right "What if your video model actually understood cause and effect?" Existing motion-controlled video models entangle camera and object motion, and treat everything as kinematic displacement. MoRight changes both. 🔥 Motion Causality — MoRight decomposes motion into actions & consequences. Give an action → MoRight predicts consequences (aka motion simulation) . Give a desired outcome → MoRight recovers the driving action (aka motion planning). Not merely displacing pixels. 🎬 Disentangled Control — MoRight separates camera and object motion, allowing users to independently control each of them. No entanglement. Project Page: research.nvidia.com/labs/sil… Paper: arxiv.org/abs/2604.07348
8
34
5
244
32,744
📣 NVIDIA is releasing a collection of open model weights: Nemotron-Labs-Sigma — 14 checkpoints at 3B/8B. Language models have stayed discrete. We bring continuous diffusion to language as in vision. Smooth ODE trajectories (16-D) unlock distillation and steering: 8 TPFs for math, 4 TPFs for code, with accuracy and diversity preserved. 🧵 1/
8
28
5
166
11,923
Rigid 📦, soft 🧸, deformable 🧣 — can ONE world model handle them all? 👀 MoRE brings us a step closer to understanding and predicting physical interactions across diverse materials! 🚀 Check it out at #CoRL2026!
Can one world model handle rigid boxes, plush toys, and cloth? Introducing MoRE: a Mixture-of-Experts 3D world model that predicts how objects move across materials and plans real-robot actions with MPC. See us at #CoRL2026: Monday, Spotlight Session 2 & Poster Session 1 🤠!
1
16
546
Shaowei Liu retweeted
Tired of waiting for your diffusion models? 🚢 We just shipped a major FastGen update: - new models: H3, LTX-2.5, LingBot-World, Cosmos 3, FLUX.2 - new methods: PDD, AYF, AnyFlow, Causal Forcing - new features: multimodal distillation, image editing, camera conditioning, KV cache management, ComfyUI 🔗 Try it out here: github.com/NVlabs/fastgen
11
49
6
422
35,333
Shaowei Liu retweeted
Avengers assemble: 1 year BakerLab + NVIDIA GenAIR = 🔥 rfd4-proteina.org Presenting RFD4-Proteina, our latest protein design model, significantly better and faster than everything we compared against, both in silico and in the lab! 1/30
27
207
45
672
72,862
Shaowei Liu retweeted
🚀 Huge congrats to Hao AI Lab @ UCSD on FastH3 Omni! Achieving up to 12× lower latency on NVIDIA Blackwell using NVFP4 QAD and FastGen PDD + DMD.
As promised, FastVideo open-sources @MiniMax_AI FastH3 Omni, training code included! 🚀 - Real-time gen @ 12x faster on NVIDIA Blackwell - NVFP4/FP8 for Spark + RTX! - ComfyUI day-0 Collab with @NVIDIAAI , Nuva Lab @nuvalab and Reactor @reactorworld Open source must win! ♥️
3
27
1,931
Shaowei Liu retweeted
The latest result of our amazing collaboration between FastVideo and FastGen: FastH3 Omni - 12x faster reference-to-video (image, video, & voice references) - distilled using our new combination of PDD & DMD Blog: haoailab.com/blogs/fastvideo… PDD: teapawt.net/julberner/status/20825…
As promised, FastVideo open-sources @MiniMax_AI FastH3 Omni, training code included! 🚀 - Real-time gen @ 12x faster on NVIDIA Blackwell - NVFP4/FP8 for Spark + RTX! - ComfyUI day-0 Collab with @NVIDIAAI , Nuva Lab @nuvalab and Reactor @reactorworld Open source must win! ♥️
2
4
1
20
1,784
Shaowei Liu retweeted
Check out RVM: simple, effective RL fine-tuning for diffusion models ⚡ Roll out (no grads) → compute rewards → re-noise → reward-weighted velocity regression + anchoring Unifies previous approaches and scales to video models with SOTA performance!
Simple, efficient, scalable diffusion RL without likelihood ratios? Meet RVM: Reward-based Velocity Matching 🎯 Rewards directly in velocity space ⚡ Cheap video fine-tuning 🧩 Unifies recent objectives 🎬 Fixes the static-video problem by dynamic motion reward How it works 🧵
1
5
25
1,458
Our team at NVIDIA Research is hiring! 🚀 We're solving some of the most exciting problems in generative AI, from video and language to AI4Science. 🌎🧠🧬 Apply below 👇
Please help spread the word! We’re hiring Research Scientists and Research Interns at NVIDIA Research to work on some of the most exciting problems in generative AI: 🌎 World Models & Video Diffusion 🧠 Diffusion Language Models 🧬 Biomolecule Design If your research aligns with any of these areas, please apply directly through the links below. Open positions + application links: research.nvidia.com/labs/gen…
4
4
84
9,836
Shaowei Liu retweeted
💫 Large Language Continuous Diffusion Models Presenting Sigma, the first large-scale (3B/8B) continuous diffusion LM (dLM) trained with likelihood. 📈 Comparable with discrete dLMs and AR on math and coding 🎛️ Enables meaningful study of inference-time interventions Jointly led with Wei Guo (@WeiGuo01), Jean-Marie Lemercier (@jm_lemercier), Simon Welker (@_cobalamin) Team: @YongganFu @mmkamani7 Sajad Norouzi @julberner @tomasgeffner @karsten_kreis @YongxinChen1 @MoleiTaoMath @jwthickstun @PavloMolchanov Ante Jukic @ArashVahdat @MardaniMorteza
17
110
22
697
80,720
Shaowei Liu retweeted
Excited to collaborate with @RekaAILabs on real-time steerable video generation using FastGen! 🎬 Check out our FastGen repo - we will also drop new models and methods soon: github.com/NVlabs/fastgen
Together with @nvidia , we've built a real-time version of our video generation model: 30B parameters, steerable mid-stream in natural language, 720p at 24fps. 11.8x faster on a single H100, no major quality drop in our blind eval. We're one step closer to World Language Action Models: AI that doesn't just generate the physical world, it acts within it. Closed beta open now, contact us for access. reka.ai/labs/research/real-t… #RekaLabs #NVIDIA #AIResearch #VideoGeneration #WorldModels
1
3
20
1,778
Shaowei Liu retweeted
Having worked with many distillation methods for accelerating diffusion & flow models, I’m still blown away by the simplicity of Parallel Decoding Distillation (PDD). 1️⃣ State-of-the-art quality and diversity with variable NFE 2️⃣ First to scale trajectory-distillation to few-step video generation PDD is about as cheap and simple as it gets: 🎯single, regression-based MSE loss using 1 student & 1 detached teacher forward ❌NO data required ❌NO adversarial loss or alternating training ❌NO JVP or finite-difference approx. ❌NO additional time-embedding or trainable models (e.g. fake score, discriminator) ❌NO stabilization tricks (e.g. normalization, clipping, warm-up)
7
15
1
132
12,904
Shaowei Liu retweeted
⚡ Video diffusion is slow. PDD changes that. 🚀 Introducing Parallel Decoding Distillation, our latest work on accelerating video diffusion models. Paper: arxiv.org/abs/2607.26004 Page: research.nvidia.com/labs/gen… By @shaulneta, @julberner, and Chao Liu. Below PDD on LTX-2.3 🔊
11
40
298
21,709
Shaowei Liu retweeted
Flex-Forcing trains a single model to do both autoregressive and bidirectional generation, letting you choose from the range at inference based on your compute budget!
Our research team just released Flex-Forcing: a video generation method that lets a single model switch between generation methods at inference time. Right now there are two main approaches to video generation. Bidirectional diffusion models attend to every frame at once, holding structure well at the cost of speed. Autoregressive models generate frame by frame, so they stream fast and scale to long clips, but accumulate error and drift over time. Flex-Forcing trains a single model to do both, letting you choose from the range at inference based on your compute budget.
5
54
9,485
With just one-line code change in the FastGen repo and a few hundred finetuning steps, DFD substantially improves the quality and diversity of DMD-based video distillation. Huge congrats to Siyi @SiyiChen277112 for leading DFD! Try it here👉 github.com/csy2077/data-forc…
Today, we're excited to release DFD (Data Forcing Distillation), a single-line fix that restores diversity and fidelity in DMD for few-step video distillation, led by my PhD student @SiyiChen277112! It improves on text-to-video, image-to-video, and causal auto-regressive video generation. The idea is simple: directly introducing real video samples into the training objective during distillation, so that the student can be explicitly pulled toward the diverse, high-quality real data distribution. This is implemented as a single line of code change when computing teacher scores: instead of evaluating the teacher score at the student's own generation (original DMD), we evaluate it on a real sample and compute the discrepancy with the fake sample. This pulls the student toward modes it missed (mitigating mode collapse) and away from over-saturated modes absent in real data. With this single-line change, we can purely discard the GAN loss, and restore the diversity/fidelity by finetuning the DMD checkpoint with only 100–300 steps. Everything is released. Code (including both inference & training), pretrained model, and data: Paper: arxiv.org/abs/2606.18478 Project page: csy2077.github.io/dfd-projec… Code, model, and data: github.com/csy2077/data-forc… We also want to have a huge shout-out to the authors of FastGen (github.com/NVlabs/FastGen) and Self-Forcing (github.com/guandeh17/Self-Fo…) for releasing their code, which makes our work possible! #VideoGeneration #DiffusionModels #AI #OpenSource
1
2
9
1,079
Shaowei Liu retweeted
1/n Thrilled to open-source DFD (Data-Forcing Distillation) — a one-line fix that restores diversity and fidelity in DMD for few-step video distillation. No GAN, and just 100–300 finetuning steps. It improves on three video generation tasks: text-to-video, image-to-video, and autoregressive video generation. The core idea is to learn from the scores of real data during distillation. DFD further improves frame coherence and physical plausibility. Everything is public: paper, code, pretrained model, and data. Paper: arxiv.org/abs/2606.18478 Project page: csy2077.github.io/dfd-projec… Code, model, and data: github.com/csy2077/data-forc… #VideoGeneration #DiffusionModels #AI #OpenSource
4
9
1
94
8,530
Shaowei Liu retweeted
What is missing to bring real-time motion research into AAA games and real-world robotics? We present MotionBricks, a step toward bridging this gap with two key components: - a single generative latent motion backbone covering 350,000+ motion skills, running at 15,000 FPS with 2 ms latency and substantially improved quality and reliability. - a unified smart primitive interface for locomotion, object / scene interaction, with fine-grained control over generated behaviors. Webpage: nvlabs.github.io/motionbrick… Code: github.com/NVlabs/GR00T-Whol… Paper: arxiv.org/abs/2604.24833 (ACM TOG / SIGGRAPH 2026)
27
151
24
1,199
154,931
Shaowei Liu retweeted
We scaled up Lyra to generate explorable 3D worlds! 🚀 Introducing Lyra 2.0 — turning a single image into a 3D world you can walk through, look back, and even drop a robot into 🤖 Code and Model available today! 🌐 Website: research.nvidia.com/labs/sil… (1/N)
27
112
18
850
1,149,048
MoRight tackles a core problem in controllable video generation: disentangling camera motion from object motion. Unlike prior methods that rely on dense future-frame tracks, MoRight uses only first-frame reprojected trajectories + camera poses, yet achieves comparable quality and better motion control.
1
4
25
1,241
Shaowei Liu retweeted
When doing motion-conditioned video generation, we think the key is in understanding motion causality (what action will lead to what outcome) and camera-motion decomposition (camera changes shouldn't entangle with object dynamics). MoRight is the first step towards this goal. We support three functionalities: 1. "Simulation": Users provide the action (e.g., moving hands), and the video model generates the consequences (e.g., cups moving and pouring water) 2. "Planning": Users specify the outcome (e.g., balls moving), and the video model generates the action that drives the outcome (e.g., moving legs) 3. "Disentanglement": Users can independently control the camera change and object dynamics, all along with the motion causality mentioned above. Come and check it at research.nvidia.com/labs/sil… P.S. Shaowei is in the job market; you shouldn't miss him!
📢MoRight: Motion Control Done Right "What if your video model actually understood cause and effect?" Existing motion-controlled video models entangle camera and object motion, and treat everything as kinematic displacement. MoRight changes both. 🔥 Motion Causality — MoRight decomposes motion into actions & consequences. Give an action → MoRight predicts consequences (aka motion simulation) . Give a desired outcome → MoRight recovers the driving action (aka motion planning). Not merely displacing pixels. 🎬 Disentangled Control — MoRight separates camera and object motion, allowing users to independently control each of them. No entanglement. Project Page: research.nvidia.com/labs/sil… Paper: arxiv.org/abs/2604.07348
7
46
7,243
Our method is straightforward! Dual-stream architecture: Dual-stream architecture: one stream for object motion on the canonical frame, one for camera motion. Object motion transferred across views via cross-view self-attention. Data: We curate a systematic data pipeline to decompose the motion sequences in our training video into active and passive motion, allowing the model to learn to capture the motion causality.
1
3
381
The project is a joint effort with @xuanchi13 @TianchangS @HuanLing6 @_saurabhg @ShenlongWang @FidlerSanja @JunGao33210520 @nvidia and @UofIllinois, thanks everyone! Special thanks to Spatial Intelligence Lab for hosting me as a intern! Check more results at 👇research.nvidia.com/labs/sil…
4
358