Pushpak Agrawal (@pushpak_agr) header
Pushpak Agrawal (@pushpak_agr) profile picture

it's all about weights | Research @humynlabs | Ex CMU

Bengaluru, India
Joined August 2020
Photo by Pushpak Agrawal (@pushpak_agr)Photo by Pushpak Agrawal (@pushpak_agr)Photo by Pushpak Agrawal (@pushpak_agr)Photo by Pushpak Agrawal (@pushpak_agr)Photo by Pushpak Agrawal (@pushpak_agr)Photo by Pushpak Agrawal (@pushpak_agr)Photo by Pushpak Agrawal (@pushpak_agr)Photo by Pushpak Agrawal (@pushpak_agr)Photo by Pushpak Agrawal (@pushpak_agr)Photo by Pushpak Agrawal (@pushpak_agr)
one approach could be to retarget human videos into robot demonstrations, generate ~10 variations from each video, and use those to train the model? @NVIDIARobotics @humynlabs
9
581
staged video, barely decent output. "our model works right out of the box in the wild." :)
3
2
2
20
6,595
two point grippers or dexterous hand?
1
107
thumb–index distance is a poor way to decide when the gripper should close, and how
101
wanted to test this intuition from a longtime, finally some output. used egodex dataset and retargetted to @i2rt_robotics yam bots. some amazing work at @humynlabs
3
6
338
tactile data lol
in another episode of researchers having fun 💀
3
127
we need a robot first world somehow
1
81
trying out some old classical robotics before getting my hands dirty with WAMs lol
1
2
67
Pushpak Agrawal retweeted
BTS from the lab. Calibrating our stereo cams by hand was landing around 0.7 px reprojection error. Good is under 0.5. So the team put a YAM arm on it. It moves the camera through a fixed set of poses in front of a ChArUco board, nobody touching it. Same poses every run, so the capture set is well-conditioned instead of depending on who held the camera that day. ~0.3 px now. Two minutes a rig. Nobody asks about calibration. Everything downstream inherits its error.
17
28
2
873
81,349
zero shot attempts.
3
56
working on calibration multicam system. hardware used is robocap from @BitRobotNetwork this plot shows all the six cameras mapped in one shared reference frame. here, LF (left-front) is taken as the master cam and all are transforms are against it. I'm seeing some constant rotation between ind. stereo cams i.e left and right of front pair (LF, RF) not sure if this is a hardware level issue, calibration issue or some damage happened e.g drop. I first tried doing the calibration using the AprilGrid tags std. 6x6 as recommended by kalibr but it failed massively on highly distorted fisheye cams of bitrobot. we observed cushion distortion effect on the rectified images. it's basically our k distortion coeff. wasn't solved properly. (might be that I didn't had full frame coverage of the aprilgrid in the camera frame, it's a WIP) charuco board has been used for these output plots.
2
83
been working on UMI's for the past week. how's the output?
2
6
600
I think we need a VR headset understanding to build sota egocentric data capture rig
1
158
Pushpak Agrawal retweeted
Just us being hands on. Something’s coming. Watch this space.
6
7
26
1,302
work in progress! what's the preference for UMI data collection? parallel jaw based gripper or these?
3
139
what's the most efficient method of tracking in umi's without using VR trackers, mirrors?
110
interesting to see how most slam algos fail on egocentric videos collected in wild
161
Pushpak Agrawal retweeted
What do you do with 16TB of pure robotic manipulation data? 🤖 You open source it on @huggingface Simple World Lab just dropped HiFi-UMI-2K - an absolutely massive dataset pushing the limits of high-fidelity manipulation. The entire 16TB is already available in the LeRobot format, meaning you can plug this massive scale straight into your pipelines with zero friction. Dataset: huggingface.co/datasets/simp… Paper: huggingface.co/papers/2607.2…
12
72
6
591
47,431