Heng Yang (@hankyang94) header
Heng Yang (@hankyang94) profile picture

Assistant Professor @Harvard SEAS @hseas, Lead the Harvard Computational Robotics Lab. #Robotics, #Optimization, #Control, #Vision, #Learning

Cambridge, MA
Joined January 2017
Very proud of Haoyu’s new work, which studies the optimization landscape and training dynamics of reinforcement learning through the noise-to-signal ratio (NSR) of the policy-gradient estimator (variance normalized by the squared gradient norm). Our main contribution is to show that, for both linear systems with linear Gaussian policies and polynomial systems with polynomial Gaussian policies, the NSR of the REINFORCE estimator can be characterized exactly. This enables us to analyze optimization trajectories directly on the NSR landscape, revealing a highly non-uniform NSR over parameter space and its systematic increase as training progresses. This provides a compelling, physically grounded explanation for training instabilities and even policy collapse. Paper: arxiv.org/abs/2602.01460
Why is RL training with policy-gradient methods often unstable—especially near the optimum? Our new work studies this through the noise-to-signal ratio (NSR) of the REINFORCE gradient estimator (defined as the estimator variance (noise) normalized by the squared norm of the true gradient (signal)). We show that the NSR is highly non-uniform and often increases as training progresses, which can trigger instability or even policy collapse. Paper link: arxiv.org/abs/2602.01460 Long thread👇
2
6
76
13,630
What an experience today at Stanford ICME presenting “A Field Theory of First-Order Methods for Linear Conic Optimization”. It was such a pleasure hanging out with the legendary and incredibly kind Michael Saunders, hearing fascinating stories about the pioneers of optimization and numerical methods whose portraits line the walls!
1
2
23
1,347
SceneAgent has been accepted to CoRL Late Breaking Results! Congrats to @lukehollis and Tianxing :) - SceneAgent now also supports building simulation environments from 4D videos - We released the first technical report here: arxiv.org/abs/2610.34745 More to come!
Excited to share SceneAgent 🤖, our agentic pipeline for scene simulation from 3D captures! Made by Luke Hollis and Tianxing Fan! We use a combination of semantic features and predictive per-Gaussian physics to transform individual complex objects or entire scenes for robotics policy training and evaluation. Our pipeline automates comprehensive details converting input images, video, lidar, or generated 3DGS scenes like from World Labs Marble. It automates: 0. 3DGS processing from input data at 30k and 60k training steps, with automated per-image correction/calibration 1. Infer semantic features for each gaussian 2. Segment objects and infill background 3. Bake predictive physics materials for each object (rigidity, friction, density, etc). 4. Decompose objects into individual parts 5. Articulate movable pieces and joints 6. Generate similar 3d meshes with different geometries, textures, physics properties, and articulations Connected to autoresearch tools, we believe that this will solve an important step in the time-intensive data problem for robotics training and policy evaluation. Paper and code coming soon, more at computationalrobotics.seas.h…
1
5
48
3,853
Happy Halloween from the Harvard Computational Robotics Group! 🎃🤖 Introducing GRASP: gradient-guided sampling for physics-based motion retargeting to dexterous hands and humanoids. Human demonstrations are a powerful source of robot skills. But kinematically retargeted motions can look right while requiring contacts and dynamics the robot cannot realize. Downstream RL then has to learn the skill while tracking a physically inconsistent reference. GRASP refines these references in simulation before policy learning. Better references help RL learn contact-rich motion tracking more effectively, as we demonstrate in pen spinning and humanoid–object interaction. On real hardware, we demonstrate open-loop trajectory replay on Sharpa hands and learned tracking policies on a Unitree G1. We hope this is a step toward scaling motion tracking to more diverse contact-rich tasks. 1/3
13
38
10
204
20,355
Building on our Tempered Sequential Monte Carlo (TSMC) work at RSS 2026, GRASP combines sampling-based exploration with gradient-guided refinement—without requiring differentiable simulator dynamics. The key idea: reuse rollouts we already collected, including rejected proposals, to fit local cost models and recover gradients for subsequent proposals. Even simple quadratic models provide useful guidance for MCMC rejuvenation. A Metropolis–Hastings correction uses the original simulator costs, preserving the intended cost-tilted target distribution for each fixed proposal kernel despite surrogate fitting errors. An MPC formulation extends the optimizer to long demonstrations. For example, GRASP achieves 94.2% success on 1,352 bimanual OakInk2 demonstrations with Sharpa hands, with a mean object position error of 2.87 cm. Crucially, the same tracking objective and optimizer settings work across Sharpa, Wuji, and XHand, without task-specific contact rewards or prescribed contact schedules. The optimizer discovers the contacts needed to reproduce the demonstrated motion. 2/3
1
6
1,199
A useful parallel is the recent SuperMimic work, which trains tracking policies by backpropagating through a learned world model. GRASP recovers trajectory-cost gradients from local models fitted to archived simulator rollouts, refining motion references before downstream policy learning. Both point to an opportunity we find exciting: extracting more useful optimization information from simulation experience. Paper, code, and demos: computationalrobotics.seas.h… Great work by @yang_liu0922, @Tong_Wang7, and @AlvinBao85746! 3/3
6
1,040
🤖 Robotics is entering an era of abundant but highly diverse data. How should we mix data from different domains to train better robot policies? 🚀 Excited to share that our paper BEACON: Cross-Domain Co-Training of Generative Robot Policies via Best-Effort Adaptation has been accepted to #NeurIPS2026! 💡 Our key idea: Instead of mixing datasets with fixed ratios, why not learn how much to trust each demonstration? Inspired by Best-Effort Adaptation theory, BEACON jointly optimizes diffusion robot policies and demonstration weights based on imitation loss and domain discrepancy. It also learns domain-level and sample-level weights to intelligently combine multiple data sources. 📊 The results are exciting! ✅ With 500 simulated + 40 real demonstrations, BEACON achieves 77% real-robot success, vs. 62% for the strongest single-source baseline. ✅ Multi-source BEACON further improves success to 83%. ✅ Better generalization to unseen configurations, with feature alignment emerging naturally.
6
8
2
84
5,636
Is RoboMME solved? Robotic memory with Astra can be surprisingly simple: just keep the episode in context, and let it act. We reached #1 on RoboMME, a large-scale benchmark for memory-dependent robotic manipulation, with Astra. Astra reached 92.1% success , outperforming even the human baseline (90.5%) and every listed method. We simply kept the full sampled visual history in context and let Astra generate every waypoint and gripper command with no RoboMME-specific training.
6
7
1
35
2,643
Proud of Haoran Sun and Shucheng Kang' new work: Spectral Deflation for Factorization-Free Matrix Filtering in Muon and Semidefinite Programming. Matrix filtering appears in seemingly different areas, from LLM training using Muon to large-scale semidefinite optimization using ADMM. In this work, we identify a shared bottleneck in fixed-depth polynomial matrix filters: a few dominant spectral components can compress the rest of the spectrum after normalization, making the filter significantly less accurate. We address this problem by deflating the dominant components. We develop this idea into a general spectral deflation framework and apply it to two very different settings. In Muon, deflation consistently improves validation loss during GPT-2 pretraining. In semidefinite programming, it substantially reduces the KKT residual of factorization-free ADMM under a comparable projection-time budget. One of the most exciting parts of this project is seeing how mathematical structure can be hidden inside large-scale practical algorithms—and how a deeper understanding of that structure can directly lead to better algorithms. 📄 Paper: arxiv.org/pdf/2609.21102
8
47
3,436
I am deeply worried about the second question “How can academia attract and retain talents when so much of its incentive structure relies on credit?” This is a serious system re-design problem that requires all voices to be heard, and all hands on deck.
Inspired by recent thoughts from senior researchers whom I deeply respect (e.g. @JitendraMalikCV @Michael_J_Black @Ken_Goldberg @phillip_isola), I wrote down some thoughts as a junior researcher too. The recent Astra demos in dexterous manipulation are truly impressive. But they also raise two questions that I don’t think academia has good answers to yet: - How should credit be assigned when an agent synthesizes many prior works into a new research result? - How can academia attract and retain talents when so much of its incentive structure relies on credit? I reflect on these questions in more detail in a blog post. Curious to hear what others think, and debates are very welcome! toruowo.github.io/blog/posts…
1
5
36
11,847
Excited to share SceneAgent 🤖, our agentic pipeline for scene simulation from 3D captures! Made by Luke Hollis and Tianxing Fan! We use a combination of semantic features and predictive per-Gaussian physics to transform individual complex objects or entire scenes for robotics policy training and evaluation. Our pipeline automates comprehensive details converting input images, video, lidar, or generated 3DGS scenes like from World Labs Marble. It automates: 0. 3DGS processing from input data at 30k and 60k training steps, with automated per-image correction/calibration 1. Infer semantic features for each gaussian 2. Segment objects and infill background 3. Bake predictive physics materials for each object (rigidity, friction, density, etc). 4. Decompose objects into individual parts 5. Articulate movable pieces and joints 6. Generate similar 3d meshes with different geometries, textures, physics properties, and articulations Connected to autoresearch tools, we believe that this will solve an important step in the time-intensive data problem for robotics training and policy evaluation. Paper and code coming soon, more at computationalrobotics.seas.h…
5
20
1
155
13,006
We’ve all been amazed by how capable Astra is—whether at solving Millennium Prize–level mathematics, reconstructing scenes, or controlling robots. A common pattern across these examples seems to be that, before Astra, many humans had already spent years attacking these problems—or closely related ones—creating a large body of knowledge, techniques, and examples. That accumulated human effort may be what eventually enables the tipping point where Astra becomes better, perhaps even substantially better, than individual humans at solving the problem. By contrast, for problems that are not yet well defined, whose goals are ambiguous, or are so ill posed that very few people have seriously studied them, Astra still seems much less capable. There are research problems in my own group that I simply cannot imagine asking Astra to solve directly—not necessarily because the underlying mathematics is harder, but because we cannot yet formulate the problem clearly. This creates an interesting dilemma. If you work on a hot and practically important problem, there is a good chance that Astra will soon be better at solving it than you are. If you work on something extremely niche, you may remain better than Astra—but the problem itself may not matter very much. Perhaps the best research strategy, then, is what great research has always been: find an important problem that has not even been properly defined yet; discover the right way to formulate and frame it; and then use systems like Astra to help solve it. The most valuable human contribution may increasingly shift from solving well-defined problems to discovering which problems should exist in the first place—and framing them in a way that, once solved, produces unexpectedly large impact.
3
8
103
5,995
Excited to share that our group has received an NSF CAREER Award! This support will be instrumental in advancing our efforts to scale Moment–SOS semidefinite programming relaxations. I am deeply grateful to my students, who have not only built upon the foundations of this work but, more importantly, have led us into new and unexpected directions that I could not have imagined when I first started working in this area years ago.
3
1
46
5,924
Flexible Locomotion Learning with Diffusion Model Predictive Control Excited to share that our paper has been accepted to #ICRA2026 @ieee_ras_icra! A diffusion-planning framework for flexible real-world quadruped locomotion. Instead of learning a fixed RL policy or relying on hand-crafted dynamics for MPC, we train a diffusion trajectory prior that jointly predicts future states and actions. Key Ideas: Diffusion-MPC: A diffusion planner unlocks flexible locomotion through test-time reward and constraint adaptation Interactive reward-weighted finetuning enables continual behavior refinement from online environment feedback Real-world deployment on Unitree Go2 with efficient and adaptive planning The same planner can adapt at test time to height changes, posture/joint constraints, balancing under external disturbances, energy-aware locomotion, and zero-shot outdoor walking on grass and slopes. 🌐Homepage: flexible-diffusion-mpc.githu… 📖Paper: arxiv.org/abs/2510.04234 🔗Code: github.com/hrh6666/Flexible-… This work is by @RunhanH, Haldun Balim, @hankyang94 , and @du_yilun. #ICRA2026 #Robotics #LeggedRobots #RobotLearning #DiffusionModels #MPC #MachineLearning
3
26
4
84
17,562
Going to @ICRA2026 in Vienna? Check out our paper, "Compose by Focus: Scene Graph-Based Atomic Skills," led by @QiHan46459 and Chenghe Chen. We show that 3D object-centric, scene-graph-based atomic skills enable robust zero-shot composition for solving challenging long-horizon manipulation tasks. 📍 Paper ThI1I.127 📍 Interactive Session ThI1I (Thursday morning) Paper: arxiv.org/abs/2509.16053
6
46
5,850
Just had a single-author paper accepted to #RSS2026! arxiv.org/abs/2604.21456 Motivated by growing interest in differentiable world models and physics simulators, we ask whether there is a unified principle for combining sampling-based global “exploration” with gradient-based local “exploitation” in trajectory and policy optimization with differentiable dynamics. By viewing control through the control-as-inference lens—recasting optimization as sampling from an unnormalized Boltzmann distribution defined by an energy function—Tempered Sequential Monte Carlo (TSMC) naturally integrates importance sampling with gradient-based Hamiltonian Monte Carlo. The key idea behind TSMC is to define a tempering path that gradually transforms an easy-to-sample prior into a complex, multi-modal posterior—or equivalently, deforms a convex energy landscape into a nonconvex one (graduated non-convexity)! We implement TSMC for both trajectory and policy optimization. On small- to medium-scale problems, it appears broadly applicable and compares favorably with state-of-the-art baselines. Excited to explore whether TSMC can scale to large-scale planning with complex, high-dimensional dynamics!
4
20
1
202
12,794
Glad that our work “Inference-Time Enhancement of Generative Robot Policies via Predictive World Modeling”, led by @QiHan46459, has been accepted to IEEE Robotics and Automation Letters! 🎉 We propose Generative Predictive Control (GPC): sample action proposals from a pretrained diffusion policy (“look back”), roll them out with a diffusion-based action-conditioned video world model (“look forward”), then rank or optimize the actions using either a learned reward model or VLM preferences. Conceptually, this is trajectory optimization / MPC with hybrid sampling + gradient optimization, interpreted through modern diffusion priors and video world models. Interestingly, we first posted the paper on arXiv in Feb 2025, when action-conditioned video world models for planning were still rare—now this direction is rapidly gaining traction. Still many open questions, e.g., • how to avoid local minima in planning • what representations work best for world models • how to balance physics priors vs. data-driven learning Paper: arxiv.org/abs/2502.00622
5
17
5
98
19,324
I was fortunate to contribute to this extensive survey of global optimization methods for 3D vision—branch-and-bound, convex relaxation, graduated nonconvexity, and more. To democratize convex relaxations for 3D vision, we released tutorials re-implementing semidefinite relaxations using the Python interface of SPOT. One bottleneck: the SymPy interface was slow. With AI-assisted coding, we’ve now developed a NumPoly interface that is orders of magnitude faster for defining/manipulating polynomials and building SDP relaxations. See the attached screenshot. Try NumPoly here: github.com/ComputationalRobo…
🚀 Thrilled to share our survey paper: Advances in Global Solvers for 3D Vision The FIRST systematic survey unifying global optimization for 3D vision, covering 400+ papers across 60+ years (1960–2025) 3 paradigms × 10 tasks × global solutions 📄 Paper: arxiv.org/abs/2602.14662 💻 Paper List & Tutorial Code : github.com/ericzzj1989/Aweso… 1/7
9
35
6,120