Research scientist at Google Brain Robotics in NYC. All views are my own.
New York, NY
Joined August 2018
- Tweets65
- Following178
- Followers510
- Likes160
Sumeet Singh retweeted
We’re making robots more capable than ever in the physical world. 🤖
Gemini Robotics 1.5 is a levelled up agentic system that can reason better, plan ahead, use digital tools such as @Google Search, interact with humans and much more. Here’s how it works 🧵
Sumeet Singh retweeted
Meet Gemini Robotics: our latest AI models designed for a new generation of helpful robots. 🤖
Based on Gemini 2.0, they bring capabilities such as better reasoning, interactivity, dexterity and generalization into the physical world. 🧵 goo.gle/gemini2-robotics
A well reasoned and considered response.
Instead of blindly trusting an LLM to plan a robot's next step, see how to use Conformal Prediction to ground its uncertainty and ask for help when needed. 👇
LLMs can generate plans and write robot code 📝 but they can also make mistakes. How do we get LLMs to 𝘬𝘯𝘰𝘸 𝘸𝘩𝘦𝘯 𝘵𝘩𝘦𝘺 𝘥𝘰𝘯'𝘵 𝘬𝘯𝘰𝘸 🤷 and ask for help?
Read more on how we can do this (with statistical guarantees) for LLMs on robots 👇
robot-help.github.io
Sumeet Singh retweeted
Presenting our robotic table wiping work today at 3pm, Pod 15 at #ICRA2023 ➡️ Come have a chat about reinforcement learning, trajectory optimization, and stochastic dynamics modeling
📢Excited to share our #ICRA2023 work on robotic table wiping via RL + optimal control!
📖 arxiv.org/abs/2210.10865
🎥 youtu.be/inORKP4F3EI
💡RL (for high-level planning) + trajectory optimization (for precise control) can solve complex tasks without on-robot data collection ⬇️
Read about Thomas' internship last summer!
Excited to share the work @thomas__lew , @Sumeet_Robotics, @robobenjie, many others @GoogleAI Robotics and I have been doing to make robots helpful for everyday tasks.
A summary of Google Robotics in 2022, including our work on novel trajectory optimizers and applications!
This quoted post is unavailable.
Sumeet Singh retweeted
Glad to see our Performer-MPC, a 8.3M-parameter Transformer model with <10ms on-robot latency paired with Model Predictive Control, is mentioned by this Robotics@Google blog! ai.googleblog.com/2023/02/go… Check out our paper at cs.gmu.edu/~xiao/papers/perf… @kchorolab @xf1280 @vikassindhwani
Sumeet Singh retweeted
Great article on LLMs - MLE training is exactly lossy data compression! I know RLHF is changing the loss to maximize predicted human ratings, but it still feels like a bit like lipstick on a pig to me (although who doesnt like bbq now and then? ;) newyorker.com/tech/annals-of…
Check out the table-wiping robot from Thomas' internship! Great example of combining the strengths of RL and classical trajectory optimization for modularity, generalization, and efficient learning. Disclaimer: surface cleaner sold separately 🙃
📢Excited to share our #ICRA2023 work on robotic table wiping via RL + optimal control!
📖 arxiv.org/abs/2210.10865
🎥 youtu.be/inORKP4F3EI
💡RL (for high-level planning) + trajectory optimization (for precise control) can solve complex tasks without on-robot data collection ⬇️
Sumeet Singh retweeted
I'm excited to present Single-Level Differentiable Contact Simulation. It's a novel formulation that unifies contact dynamics and collision detection in a single optimization problem.
paper: arxiv.org/abs/2212.06764
code: github.com/simon-lc/Silico.j…
video: youtu.be/oaGLTR13iF8
Sumeet Singh retweeted
Work from our @GoogleAI team - training robots to play table tennis to better understand how they learn in dynamic + high-speed settings. One project achieved a 300+ hit rally, while another is focused on matching the precision of amateur human players 🏓
ai.googleblog.com/2022/10/ta…
Come check out this workshop on Monday at #RSS2022! I'll be presenting (poster + spotlight) our latest work on multiscale sensor fusion architectures for continuous-time control using Neural CDEs.
Our #RSS2022 workshop on "Overlooked Aspects of Imitation Learning" is this Monday June 27 - join us virtual or in-person! We have a wonderful lineup of speakers (@ancadianadragan @ankurhandos @chelseabfinn @LerrelPinto Mohi Khansari @shimon8282)
See: tinyurl.com/2pjt86uk
Thrilled that ‘Dynamic SQP’ was selected as a finalist for the #icra2022 outstanding planning paper award!
arxiv.org/abs/2109.07081
Thread ⬇️
Results show significant stability and convergence benefits on difficult planning problems. Official code-release as part of the google/trajax library (github.com/google/trajax) is on the way!
