Robotics Tinkerer. RS @Amazon FAR Prev: @META (FAIR), @DFKI, @TUDarmstadt https://xitter.cf/t.co/RQpq7Prbln X https://xitter.cf/t.co/umZQeDjJv4
Joined November 2017
- Tweets293
- Following1.4K
- Followers1.5K
- Likes1.9K
Great blog by @ToruO_O !
My view is that we as researchers are becoming into great Rick Rubin
youtu.be/jg1WUOxY6Cg?si=YLdk…
How should we approach research, when the very field we work on is going through rapid and potentially fundamental changes?
I recently gave a talk at the Stanford Vision Seminar, reflecting on my PhD journey and how AI is changing the way I think about research. I adapted part of it into a new blog post, organized around three questions:
1. What should we work on?
2. How should we work with AI?
3. What is a PhD for?
I hope some of my reflections are helpful. The talk ended with an open discussion, and I’d love to continue it here; curious how others are thinking about these questions.
Personally -- as much as there's FOMO, I also think life is very long, and we have time.
toruowo.github.io/blog/posts…
We did not study this in particular. But we did not observe a performance decay in slow tempo.
Why do u have this hypothesis?
This song is slightly slower👇
Okey, not super impressive!
BUT, this plot gives us hope! we found that the agent keeps improving on unseen songs with more and more demonstrations, so we are hopeful that given the large datasets available on the Internet, our agent will keep improving in the future 🎹
TASK 2: take all these RL policies and distill them (BC) into a Diffusion Policy.
We use a hierarchical policy. The top layer policy outputs desired fingertip motions while the bottom layer policy generates the configuration space actions.
Performance in unseen songs 👇
We train individual RL single-song policies, which we leverage to generate the desired observation-action pairs!
Observe that using fingertip demonstrations leads to human-like motions, while the absence of them leads to policies behaving in unexpected styles.
We first extract both the FINGERTIP motion trajectories and a TASK trajectory (the song), which informs the task to be solved, from the videos.
Then, we apply residual RL, using the fingertip motion as the nominal behaviour and the task trajectory as reward
So, what is the challenge?
Unlike teleoperation data, video data lacks the action info needed to determine what control signals should be applied to the robot to accomplish the observed task.
TASK 1: Infer the actions the robot should do to match the videos.
How? RL + IL
@ChengQian0112 , with the collaboration of @kevin_zakka and @Jan_R_Peters introduces a simple framework to learn highly dexterous manipulation skills from videos. Given the large amount of video data, we can learn a generalist policy that can play ANY song.
YouTube is a LARGE dataset of demonstration videos to train Generalist robot agents, but lacks action data.
How can we learn DEXTEROUS skills from them?
In #CoRL2024, we explore the problem of learning a Generalist Piano Playing agent from YouTube videos.
pianomime.github.io/
