Shaowei Liu (@stevenpg8) header
Shaowei Liu (@stevenpg8) profile picture

Research Scientist @NVIDIA | CS PhD @IllinoisCDS | MSCS @ucsd_cse | BSEE @Tsinghua_uni

Joined October 2019
Photo by Shaowei Liu (@stevenpg8)Photo by Shaowei Liu (@stevenpg8)Photo by Shaowei Liu (@stevenpg8)Photo by Shaowei Liu (@stevenpg8)Photo by Shaowei Liu (@stevenpg8)Photo by Shaowei Liu (@stevenpg8)Photo by Shaowei Liu (@stevenpg8)Photo by Shaowei Liu (@stevenpg8)Photo by Shaowei Liu (@stevenpg8)Photo by Shaowei Liu (@stevenpg8)
📣 NVIDIA is releasing a collection of open model weights: Nemotron-Labs-Sigma — 14 checkpoints at 3B/8B. Language models have stayed discrete. We bring continuous diffusion to language as in vision. Smooth ODE trajectories (16-D) unlock distillation and steering: 8 TPFs for math, 4 TPFs for code, with accuracy and diversity preserved. 🧵 1/
8
28
5
166
11,932
Shaowei Liu retweeted
Tired of waiting for your diffusion models? 🚢 We just shipped a major FastGen update: - new models: H3, LTX-2.5, LingBot-World, Cosmos 3, FLUX.2 - new methods: PDD, AYF, AnyFlow, Causal Forcing - new features: multimodal distillation, image editing, camera conditioning, KV cache management, ComfyUI 🔗 Try it out here: github.com/NVlabs/fastgen
11
49
6
422
35,333
Shaowei Liu retweeted
Avengers assemble: 1 year BakerLab + NVIDIA GenAIR = 🔥 rfd4-proteina.org Presenting RFD4-Proteina, our latest protein design model, significantly better and faster than everything we compared against, both in silico and in the lab! 1/30
27
207
45
672
72,864
Shaowei Liu retweeted
🚀 Huge congrats to Hao AI Lab @ UCSD on FastH3 Omni! Achieving up to 12× lower latency on NVIDIA Blackwell using NVFP4 QAD and FastGen PDD + DMD.
As promised, FastVideo open-sources @MiniMax_AI FastH3 Omni, training code included! 🚀 - Real-time gen @ 12x faster on NVIDIA Blackwell - NVFP4/FP8 for Spark + RTX! - ComfyUI day-0 Collab with @NVIDIAAI , Nuva Lab @nuvalab and Reactor @reactorworld Open source must win! ♥️
3
27
1,931
Shaowei Liu retweeted
The latest result of our amazing collaboration between FastVideo and FastGen: FastH3 Omni - 12x faster reference-to-video (image, video, & voice references) - distilled using our new combination of PDD & DMD Blog: haoailab.com/blogs/fastvideo… PDD: teapawt.net/julberner/status/20825…
As promised, FastVideo open-sources @MiniMax_AI FastH3 Omni, training code included! 🚀 - Real-time gen @ 12x faster on NVIDIA Blackwell - NVFP4/FP8 for Spark + RTX! - ComfyUI day-0 Collab with @NVIDIAAI , Nuva Lab @nuvalab and Reactor @reactorworld Open source must win! ♥️
2
4
1
20
1,784
Shaowei Liu retweeted
Check out RVM: simple, effective RL fine-tuning for diffusion models ⚡ Roll out (no grads) → compute rewards → re-noise → reward-weighted velocity regression + anchoring Unifies previous approaches and scales to video models with SOTA performance!
Simple, efficient, scalable diffusion RL without likelihood ratios? Meet RVM: Reward-based Velocity Matching 🎯 Rewards directly in velocity space ⚡ Cheap video fine-tuning 🧩 Unifies recent objectives 🎬 Fixes the static-video problem by dynamic motion reward How it works 🧵
1
5
25
1,458
Shaowei Liu retweeted
💫 Large Language Continuous Diffusion Models Presenting Sigma, the first large-scale (3B/8B) continuous diffusion LM (dLM) trained with likelihood. 📈 Comparable with discrete dLMs and AR on math and coding 🎛️ Enables meaningful study of inference-time interventions Jointly led with Wei Guo (@WeiGuo01), Jean-Marie Lemercier (@jm_lemercier), Simon Welker (@_cobalamin) Team: @YongganFu @mmkamani7 Sajad Norouzi @julberner @tomasgeffner @karsten_kreis @YongxinChen1 @MoleiTaoMath @jwthickstun @PavloMolchanov Ante Jukic @ArashVahdat @MardaniMorteza
17
110
22
697
80,725
Shaowei Liu retweeted
Excited to collaborate with @RekaAILabs on real-time steerable video generation using FastGen! 🎬 Check out our FastGen repo - we will also drop new models and methods soon: github.com/NVlabs/fastgen
Together with @nvidia , we've built a real-time version of our video generation model: 30B parameters, steerable mid-stream in natural language, 720p at 24fps. 11.8x faster on a single H100, no major quality drop in our blind eval. We're one step closer to World Language Action Models: AI that doesn't just generate the physical world, it acts within it. Closed beta open now, contact us for access. reka.ai/labs/research/real-t… #RekaLabs #NVIDIA #AIResearch #VideoGeneration #WorldModels
1
3
20
1,778
Shaowei Liu retweeted
Having worked with many distillation methods for accelerating diffusion & flow models, I’m still blown away by the simplicity of Parallel Decoding Distillation (PDD). 1️⃣ State-of-the-art quality and diversity with variable NFE 2️⃣ First to scale trajectory-distillation to few-step video generation PDD is about as cheap and simple as it gets: 🎯single, regression-based MSE loss using 1 student & 1 detached teacher forward ❌NO data required ❌NO adversarial loss or alternating training ❌NO JVP or finite-difference approx. ❌NO additional time-embedding or trainable models (e.g. fake score, discriminator) ❌NO stabilization tricks (e.g. normalization, clipping, warm-up)
7
15
1
132
12,904
Shaowei Liu retweeted
⚡ Video diffusion is slow. PDD changes that. 🚀 Introducing Parallel Decoding Distillation, our latest work on accelerating video diffusion models. Paper: arxiv.org/abs/2607.26004 Page: research.nvidia.com/labs/gen… By @shaulneta, @julberner, and Chao Liu. Below PDD on LTX-2.3 🔊
11
40
298
21,709
Shaowei Liu retweeted
Flex-Forcing trains a single model to do both autoregressive and bidirectional generation, letting you choose from the range at inference based on your compute budget!
Our research team just released Flex-Forcing: a video generation method that lets a single model switch between generation methods at inference time. Right now there are two main approaches to video generation. Bidirectional diffusion models attend to every frame at once, holding structure well at the cost of speed. Autoregressive models generate frame by frame, so they stream fast and scale to long clips, but accumulate error and drift over time. Flex-Forcing trains a single model to do both, letting you choose from the range at inference based on your compute budget.
5
54
9,485
Shaowei Liu retweeted
1/n Thrilled to open-source DFD (Data-Forcing Distillation) — a one-line fix that restores diversity and fidelity in DMD for few-step video distillation. No GAN, and just 100–300 finetuning steps. It improves on three video generation tasks: text-to-video, image-to-video, and autoregressive video generation. The core idea is to learn from the scores of real data during distillation. DFD further improves frame coherence and physical plausibility. Everything is public: paper, code, pretrained model, and data. Paper: arxiv.org/abs/2606.18478 Project page: csy2077.github.io/dfd-projec… Code, model, and data: github.com/csy2077/data-forc… #VideoGeneration #DiffusionModels #AI #OpenSource
4
9
1
94
8,530
Shaowei Liu retweeted
What is missing to bring real-time motion research into AAA games and real-world robotics? We present MotionBricks, a step toward bridging this gap with two key components: - a single generative latent motion backbone covering 350,000+ motion skills, running at 15,000 FPS with 2 ms latency and substantially improved quality and reliability. - a unified smart primitive interface for locomotion, object / scene interaction, with fine-grained control over generated behaviors. Webpage: nvlabs.github.io/motionbrick… Code: github.com/NVlabs/GR00T-Whol… Paper: arxiv.org/abs/2604.24833 (ACM TOG / SIGGRAPH 2026)
27
151
24
1,199
154,931
Shaowei Liu retweeted
We scaled up Lyra to generate explorable 3D worlds! 🚀 Introducing Lyra 2.0 — turning a single image into a 3D world you can walk through, look back, and even drop a robot into 🤖 Code and Model available today! 🌐 Website: research.nvidia.com/labs/sil… (1/N)
27
112
18
850
1,149,049
MoRight tackles a core problem in controllable video generation: disentangling camera motion from object motion. Unlike prior methods that rely on dense future-frame tracks, MoRight uses only first-frame reprojected trajectories + camera poses, yet achieves comparable quality and better motion control.
1
4
25
1,241
Shaowei Liu retweeted
When doing motion-conditioned video generation, we think the key is in understanding motion causality (what action will lead to what outcome) and camera-motion decomposition (camera changes shouldn't entangle with object dynamics). MoRight is the first step towards this goal. We support three functionalities: 1. "Simulation": Users provide the action (e.g., moving hands), and the video model generates the consequences (e.g., cups moving and pouring water) 2. "Planning": Users specify the outcome (e.g., balls moving), and the video model generates the action that drives the outcome (e.g., moving legs) 3. "Disentanglement": Users can independently control the camera change and object dynamics, all along with the motion causality mentioned above. Come and check it at research.nvidia.com/labs/sil… P.S. Shaowei is in the job market; you shouldn't miss him!
📢MoRight: Motion Control Done Right "What if your video model actually understood cause and effect?" Existing motion-controlled video models entangle camera and object motion, and treat everything as kinematic displacement. MoRight changes both. 🔥 Motion Causality — MoRight decomposes motion into actions & consequences. Give an action → MoRight predicts consequences (aka motion simulation) . Give a desired outcome → MoRight recovers the driving action (aka motion planning). Not merely displacing pixels. 🎬 Disentangled Control — MoRight separates camera and object motion, allowing users to independently control each of them. No entanglement. Project Page: research.nvidia.com/labs/sil… Paper: arxiv.org/abs/2604.07348
7
46
7,243
Shaowei Liu retweeted
🕹️We are excited to introduce "ChronoEdit: Towards Temporal Reasoning for Image Editing and World Simulation" ChronoEdit reframes image editing as a video generation task to encourage temporal consistency. It leverages a temporal reasoning stage that denoises with “video reasoning tokens” to "reason" on physically plausible edits. See the attached video for results. Project Page: research.nvidia.com/labs/tor… Arxiv: arxiv.org/abs/2510.04290 Code and model are coming.
6
35
8
140
37,093
[1/N] 🎥 We've made available a powerful spatial AI tool named ViPE: Video Pose Engine, to recover camera motion, intrinsics, and dense metric depth from casual videos! Running at 3–5 FPS, ViPE handles cinematic shots, dashcams, and even 360° panoramas. 🔗 research.nvidia.com/labs/tor…
16
102
9
448
64,022
Shaowei Liu retweeted
We are excited to share Cosmos-Drive-Dreams 🚀 A bold new synthetic data generation (SDG) pipeline powered by world foundation models—designed to synthesize rich, challenging driving scenarios at scale. Models, Code, Dataset, Tookit are released. Website: research.nvidia.com/labs/tor…
11
43
2
117
24,418
Shaowei Liu retweeted
🚀Excited to introduce GEN3C #CVPR2025, a generative video model with an explicit 3D cache for precise camera control. 🎥It applies to multiple use cases, including single-view and sparse-view NVS🖼️ and challenging settings like monocular dynamic NVS and driving simulation🚗. Project page: research.nvidia.com/labs/tor…
3
50
6
182
60,213