Robotics @Stanford, advised by @SongShuran. Working on dexterous manipulation 🤖

California
Joined December 2024
Harsh Gupta retweeted
With @JayHe748646 @KehanWen170077 @leggedrobotics , we push humanoid mobility to next frontier. Achieved by a generalist contact planner + a generalist tracker + a generalized neural mapping, warmstarted from few human data. Website: leggedrobotics.github.io/ACP… Promo Video 👇
25
70
16
455
39,967
Harsh Gupta retweeted
If video models scale like LLMs, they'll soon generate a perfect video plan for any embodied task. Do we need anything else? Introducing Mirror, which learns to follow these plans just by practicing, and even crafts an iron pickaxe in Minecraft w/o rewards or action labels! (1/n)
9
44
12
388
50,046
Harsh Gupta retweeted
Can humanoids adapt to new terrain using only flat-ground references? 🤖 Meet P-BFM! 🥳 We learn a terrain-agnostic intent space with unsupervised RL and forward-backward representations, paired with a terrain-aware policy that adapts across diverse terrains—without terrain-specific demonstrations. 📄🎬💻 Check out our paper, video, and code here! 🌐 roboparty.github.io/P-BFM/
4
21
2
125
13,496
Most robot policies treat time implicitly, using sequence position as a proxy for physical time. But real robot systems rarely run on a fixed clock. Sensors run at different rates, observations arrive with delays, and actions take time to compute and execute. Successful control depends not only on what the robot observes but also on the timing of observations and actions. We introduce 🕒 Chronos Policy, a diffusion policy that explicitly conditions on observation capture times and requested action execution times. We use time-aware positional encodings to make attention depend on elapsed physical time. Because timestamps let the policy interpret each sequence's timing, we can randomize sensing and action timing during training without making those variations ambiguous. This exposes the policy to a wider range of timing conditions, improving robustness to timing changes at deployment.
12
43
5
245
170,065
Harsh Gupta retweeted
Excited to share our work PredActor, a predictive action diffusion policy for steerable humanoid control. This direct action-generation view is appealing because it avoids the generator-tracker mismatch. 🤩
Can humanoid motion generation and control be unified in a single policy? PredActor uses a single end-to-end diffusion policy rather than separate motion generation and tracking. With proprioceptive feedback, it predicts internal future states and directly outputs actions at 50 Hz onboard a Unitree G1. Text-conditioned motion, joystick steering, disturbance response, and semantic interpolation are supported in the single policy. Project Page: masteryip.github.io/predacto…
5
10
1
67
4,730
Harsh Gupta retweeted
How can we train better behavior from imperfect robot demos? In NEEDLEWORK, we study how we can get stitching to work on robot data, and more importantly, why these connections help. Website: needle-work.github.io Paper: arxiv.org/abs/2610.02339
8
35
4
152
17,316
Harsh Gupta retweeted
Transformer Transformer will be presented at #CoRL2026 🚀 See y'all in Austin! Website: transformer-transformer.gith… Video: youtube.com/watch?v=TTyjvPVF… Paper: transformer-transformer.gith…
13
83
23
651
116,049
Harsh Gupta retweeted
New paper release: Learning Agile Perceptive Traversal of Sparse 3D Structures for Humanoids Paper arxiv.org/pdf/2608.29769 Videos nemantor.github.io/sparse-3d… We study how to do learning and sim2real for 3d traversal, such as monkey bars and overhanging obstacles.
12
40
11
267
83,383
Harsh Gupta retweeted
When a robot learns a new behavior, what keeps it from forgetting the ones it already knows? Prior work, especially on VLAs, shows that rehearsing past experiences can enable strong continual learning. But why does rehearsal work so well, and when does it fail? We find that a small but ✨special subset✨ of rehearsed data largely determines whether old behaviors are retained or forgotten. We call these examples Memory Anchors. ⚓ Withholding just the top 10% of Memory Anchors from rehearsal increases forgetting by up to 4.5x. Increasing their presence, meanwhile, reduces task forgetting and enables a real robot to learn challenging task sequences. Website: robot-adaptation.github.io/M… Paper: arxiv.org/abs/2608.26545 Curious? Read on! 🧵👇 (1/9)
9
57
8
281
50,984
Harsh Gupta retweeted
Teleop systems are usually designed for a single embodiment, but do they have to be? Introducing ModPack 🎒: a modular teleoperation interface for bimanual mobile robots. A wearable backpack provides shared infrastructure, robot-specific leader arms adapt to different embodiments, and plug-and-play modules add capabilities like haptic feedback, active perception, and mobile manipulation. 🧵(1/n)
11
26
7
90
23,599
Harsh Gupta retweeted
Robot learning is moving beyond policies built for one robot, one scene, one task. At MIT, we’re exploring a different path: turning video world models into embodiment-agnostic robot policies. Introducing VERA: a 14B video-to-action system that controls robots across embodiments, skills, and environments. From zero-shot pick-and-place on a real Panda arm to contact-rich cube reorientation with a 16-DoF robotic hand. Different robots. Different environments. Different tasks. Same video planner. Same weights. We’re open-sourcing everything so you can fine-tune VERA for your own robot setup too. Deep dive in the thread: 🔗 vera.csail.mit.edu 🧵 (1/7)
15
60
12
457
173,477
Harsh Gupta retweeted
A nice example of: human data for intent + physics grounding for control = useful robot skills Human data should teach robots what should happen: task specification, high-level intent, semantic generalization. Simulation should teach robots how to make it happen: embodiment-specific motor skills, physics-level generalization. That said, simulation is only one way to provide “physics grounding.” Real robot data can also play this role. But some form of a physics-grounding layer is hard to avoid. How it should interface with the human-data layer is, I think, one of the most important research questions in robot learning.
We trained a dexterous hand 🤖 from internet-scale human videos 📺 with ZERO real-world robot or teleoperation data We pair a world model 🌎 that predicts task intent with a generalist sensorimotor policy ✋ trained sim2real to zero-shot fulfill ANY intent Introducing LUCID: Learning Embodiment-Agnostic Intent Models from Unstructured Human Videos for Scalable Dexterous Robot Skill Acquisition 🧵
2
4
87
13,661
Harsh Gupta retweeted
Multiple folks buying into the WHAT and HOW framework The slide is from the talk last year at ICRA keynote debate. (youtube.com/watch?t=196&v=Pf…) Another great example of using human data primarily. from reference generation not action supervision.
We trained a dexterous hand 🤖 from internet-scale human videos 📺 with ZERO real-world robot or teleoperation data We pair a world model 🌎 that predicts task intent with a generalist sensorimotor policy ✋ trained sim2real to zero-shot fulfill ANY intent Introducing LUCID: Learning Embodiment-Agnostic Intent Models from Unstructured Human Videos for Scalable Dexterous Robot Skill Acquisition 🧵
5
4
22
5,761
Harsh Gupta retweeted
One complaint I had about human video for dexterous hand manipulation is they are neither “dexterous” nor “human-capable”, glad to see so much progress this year on actual useful lower dynamics unlocked. Very decent progress on WM for manipulation!
We trained a dexterous hand 🤖 from internet-scale human videos 📺 with ZERO real-world robot or teleoperation data We pair a world model 🌎 that predicts task intent with a generalist sensorimotor policy ✋ trained sim2real to zero-shot fulfill ANY intent Introducing LUCID: Learning Embodiment-Agnostic Intent Models from Unstructured Human Videos for Scalable Dexterous Robot Skill Acquisition 🧵
1
2
6
2,219
Harsh Gupta retweeted
Meet BFM-Zero: A Promptable Humanoid Behavioral Foundation Model w/ Unsupervised RL👉 lecar-lab.github.io/BFM-Zero… 🧩ONE latent space for ALL tasks ⚡Zero-shot goal reaching, tracking, and reward optimization (any reward at test time), from ONE policy 🤖Natural recovery & transition
7
75
23
271
84,158
How can we utilize cross-embodiment action-free videos in learning generalizable and robust policies? Introducing our paper in ICCV 2025 @ICCVConference (also best paper nomination at the ICRA FMNS WS), GenFlowRL: an reward shaping method with generative object-centric flow. 1/6
1
14
5
107
24,580
We finally made it on Aerial Manipulation! Lightbulb mounting, fruit pick-and-place, and peg-in-hole, all by the visuomotor policies trained on only UMI data. Back in the day, when we successfully teleoped those tasks in the Flying-Hand paper, we knew we would turn it into the autonomous policy one day, but seeing it really succeed, especially with only off-robot data, is amazing! If you are struggling with no access to the robot data to learn from, or your system is essentially hard to control precisely at some point, give our method a try, and you may find the policy module is more powerful than you thought.
✈️🤖 What if an embodiment-agnostic visuomotor policy could adapt to diverse robot embodiments at inference with no fine-tuning? Introducing UMI-on-Air, a framework that brings embodiment-aware guidance to diffusion policies for precise, contact-rich aerial manipulation.
9
1
52
9,550
Harsh Gupta retweeted
Not all robots are equally smart. Some can follow human-like motions. Most can’t. That’s the embodiment gap, the reason why the same visuomotor policy that works on a robot arm fails on a drone. UMI-on-Air might have just closed that gap. ✅ A single policy that adapts to any robot at test time… no retraining ✅ Two-way communication between robot and policy ✅ Smarter decisions based on what the robot can actually do ✅ Real-world success on lightbulb installation, lemon picking, and peg-in-hole This new framework gives diffusion models embodiment awareness; making them practical for contact-rich, long-horizon tasks. And the wild part? It works even when trained on data collected thousands of miles away. Thanks for sharing, @harshgupta ! 📍Paper: arxiv.org/abs/2510.02614 Website: umi-on-air.github.io
2
7
39
3,252
Harsh Gupta retweeted
so cool! The hardware + algorithm design here is really impressive. We need more full stack innovation like this
✈️🤖 What if an embodiment-agnostic visuomotor policy could adapt to diverse robot embodiments at inference with no fine-tuning? Introducing UMI-on-Air, a framework that brings embodiment-aware guidance to diffusion policies for precise, contact-rich aerial manipulation.
2
5
893