
it's all about weights | Research @humynlabs | Ex CMU
Bengaluru, India
Joined August 2020
- Tweets95
- Following180
- Followers57
- Likes289
one approach could be to retarget human videos into robot demonstrations, generate ~10 variations from each video, and use those to train the model?
@NVIDIARobotics @humynlabs
9
581
wanted to test this intuition from a longtime, finally some output.
used egodex dataset and retargetted to @i2rt_robotics yam bots.
some amazing work at @humynlabs
Pushpak Agrawal retweeted
BTS from the lab.
Calibrating our stereo cams by hand was landing around 0.7 px reprojection error. Good is under 0.5.
So the team put a YAM arm on it. It moves the camera through a fixed set of poses in front of a ChArUco board, nobody touching it.
Same poses every run, so the capture set is well-conditioned instead of depending on who held the camera that day.
~0.3 px now. Two minutes a rig.
Nobody asks about calibration. Everything downstream inherits its error.
working on calibration multicam system.
hardware used is robocap from @BitRobotNetwork
this plot shows all the six cameras mapped in one shared reference frame. here, LF (left-front) is taken as the master cam and all are transforms are against it.
I'm seeing some constant rotation between ind. stereo cams i.e left and right of front pair (LF, RF)
not sure if this is a hardware level issue, calibration issue or some damage happened e.g drop.
I first tried doing the calibration using the AprilGrid tags std. 6x6 as recommended by kalibr but it failed massively on highly distorted fisheye cams of bitrobot. we observed cushion distortion effect on the rectified images. it's basically our k distortion coeff. wasn't solved properly. (might be that I didn't had full frame coverage of the aprilgrid in the camera frame, it's a WIP)
charuco board has been used for these output plots.
2
83
work in progress!
what's the preference for UMI data collection? parallel jaw based gripper or these?
3
139
Pushpak Agrawal retweeted
What do you do with 16TB of pure robotic manipulation data? 🤖
You open source it on @huggingface
Simple World Lab just dropped HiFi-UMI-2K - an absolutely massive dataset pushing the limits of high-fidelity manipulation.
The entire 16TB is already available in the LeRobot format, meaning you can plug this massive scale straight into your pipelines with zero friction.
Dataset: huggingface.co/datasets/simp…
Paper: huggingface.co/papers/2607.2…







