Robin Han retweeted
Yesterday the hyped Genesis simulator released. But it's up to 10x slower than existing GPU sims, not 10-80x faster or 430,000x faster than realtime since they benchmark mostly static environments
blog post with corrected open source benchmarks & details: stoneztao.substack.com/p/the…
Robin Han retweeted
Check out the full demo of Hillbot Alpha's latest progress! Our robot successfully stocks shelves over a **long horizon** in the real-world, while adapting smoothly to diverse objects, positions, and human interference.
Check out the full demonstration of Hillbot Alpha, our autonomous mobile manipulation robot performing long-horizon shelf stocking, trained using #sim2real technology.
#AGI #EmbodiedAI #AI #ArtificialIntelligence #Simulation #robots #autonomousrobots #computervision
sim-to-real works and we will make it accessible for more developers in the near future!
Meet #HillbotAlpha, the first fully autonomous mobile manipulation robot trained using sim-to-real technology.
Designed in Hillbot’s San Diego headquarters, Hillbot Alpha represents the potential of data synthesis via simulation in robotics. Through a robust strategy that combines a small sample of real-world data with synthetic data, Hillbot Alpha can effectively adapt to evolving task requirements and environments while working safely beside humans.
#Hillbot #AGI #EmbodiedAI #AI #Sim2Real #ArtificialIntelligence #Simulation #Simtoreal
Robin Han retweeted
agree on the last part: “Teleoperation is necessary but insufficient”, my opinion is still that we need sim + generative AI to scale data to a sufficient scale for usedul generalization.
ps check out our just released GPU acc robotics simulator github.com/haosulab/ManiSkil…
Congrats to @Tesla_Optimus team on another stellar update! The video gives us a peek at their human data collection farm, which I believe is Optimus' biggest lead. What does it take to build such a pipeline? Optimus nailed all of the following:
1. Optimus hands are among the best 5-finger, dexterous robot hands in the world. It's got tactile sensing, 11 degrees of freedom (DOF) compared to many competitors with only 6-7 DOF, and robustness to withstand lots of object interactions without constant maintenance.
2. Teleoperation software: we can see that the human operators are wearing VR goggles and gloves. It is very non-trivial to set up the software to have first-person video streamed in and precise control streamed out, while maintaining extremely low latency. Humans are highly sensitive to even the smallest delay between their own motions and the robot's. Optimus has a fluid whole body controller that enacts the human poses in real-time.
3. Sizeable fleet: you need more than one robot to collect data in parallel, well-trained human contractors taking multiple shifts per day (preferably 24/7), and an on-call maintenance crew to make sure that the robots are always busy. That's a ton of operational complexity that academic research labs don't even think of.
4. Tasks & environments: it's equally important to figure out *what* to teleoperate. Currently, most such efforts are demo-driven: collect data on the tasks that you want to put into a social media video. But solving general-purpose robots requires us to think carefully about the distribution of tasks and environments. From 43"-51" in the video, we can see factory & household settings like moving batteries, handling laundry, sorting daily objects into shelves.
It's an open-ended research question: if you only have the budget to collect training data for 1,000 tasks, what would you pick to maximize skill transfer and generalization?
Closing thought: teleoperation is a necessary but insufficient condition to solve humanoid robotics. It fundamentally does not scale. More about this later.
Robin Han retweeted
Congrats to @Tesla_Optimus team on another stellar update! The video gives us a peek at their human data collection farm, which I believe is Optimus' biggest lead. What does it take to build such a pipeline? Optimus nailed all of the following:
1. Optimus hands are among the best 5-finger, dexterous robot hands in the world. It's got tactile sensing, 11 degrees of freedom (DOF) compared to many competitors with only 6-7 DOF, and robustness to withstand lots of object interactions without constant maintenance.
2. Teleoperation software: we can see that the human operators are wearing VR goggles and gloves. It is very non-trivial to set up the software to have first-person video streamed in and precise control streamed out, while maintaining extremely low latency. Humans are highly sensitive to even the smallest delay between their own motions and the robot's. Optimus has a fluid whole body controller that enacts the human poses in real-time.
3. Sizeable fleet: you need more than one robot to collect data in parallel, well-trained human contractors taking multiple shifts per day (preferably 24/7), and an on-call maintenance crew to make sure that the robots are always busy. That's a ton of operational complexity that academic research labs don't even think of.
4. Tasks & environments: it's equally important to figure out *what* to teleoperate. Currently, most such efforts are demo-driven: collect data on the tasks that you want to put into a social media video. But solving general-purpose robots requires us to think carefully about the distribution of tasks and environments. From 43"-51" in the video, we can see factory & household settings like moving batteries, handling laundry, sorting daily objects into shelves.
It's an open-ended research question: if you only have the budget to collect training data for 1,000 tasks, what would you pick to maximize skill transfer and generalization?
Closing thought: teleoperation is a necessary but insufficient condition to solve humanoid robotics. It fundamentally does not scale. More about this later.
I've crafted an incredible 3D object using sudoAI. Click the link to behold this marvel or unleash your creativity and design your own! @sudoAI_
sudo.ai/preview?t=cljtuuna8e…
#sudoAI #3DGenAI #3DCreation
share.sudo.ai/twitter/pc?t=c…
Robin Han retweeted
Everyone will be a mesh maker.
Thrilled to announce One-2-3-45++, which transforms a single RGB image of any object into a high-fidelity textured mesh in under one minute!🌟
Project Page: sudo-ai-3d.github.io/One2345…
Try our Desktop Demo: sudo.ai/3dgen
I've crafted an incredible 3D object using sudoAI. Click the link to behold this marvel or unleash your creativity and design your own! @sudoAI_
sudo.ai/preview?t=cl9d6qftk3…
#sudoAI #3DGenAI #3DCreation
share.sudo.ai/twitter/pc?t=c…
I've crafted an incredible 3D object using sudoAI. Click the link to behold this marvel or unleash your creativity and design your own! @sudoAI_
sudo.ai/preview?t=cl9bpuftk3…
#sudoAI #3DGenAI #3DCreation
share.sudo.ai/twitter/pc?t=c…
RT @haosu_twitr: ✨See our @sudoai_'s powerful 3D Generative AI technology in the hands of creative beta users!
🚀Every 3D asset in the game…
This quoted post is unavailable.
Robin Han retweeted
Excited to release Zero123++, an image-conditioned diffusion model for generating high-quality, 3D-consistent multi-view images from a single input view!
Paper: arxiv.org/abs/2310.15110
Code: github.com/SUDO-AI-3D/zero12…
🤗Demo: huggingface.co/spaces/sudo-a…
Robin Han retweeted
Still taking tens of minutes or even hours to generate a single 3D shape?
Check our new work, One-2-3-45, which takes a single image of any object as input and generates a 3D textured mesh in a single forward pass of just 45 seconds!
project: One-2-3-45.com 🧵👇1/
Robin Han retweeted
One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization
paper page: huggingface.co/papers/2306.1…
Single image 3D reconstruction is an important but challenging task that requires extensive knowledge of our natural world. Many existing methods solve this problem by optimizing a neural radiance field under the guidance of 2D diffusion models but suffer from lengthy optimization time, 3D inconsistency results, and poor geometry. In this work, we propose a novel method that takes a single image of any object as input and generates a full 360-degree 3D textured mesh in a single feed-forward pass. Given a single image, we first use a view-conditioned 2D diffusion model, Zero123, to generate multi-view images for the input view, and then aim to lift them up to 3D space. Since traditional reconstruction methods struggle with inconsistent multi-view predictions, we build our 3D reconstruction module upon an SDF-based generalizable neural surface reconstruction method and propose several critical training strategies to enable the reconstruction of 360-degree meshes. Without costly optimizations, our method reconstructs 3D shapes in significantly less time than existing methods. Moreover, our method favors better geometry, generates more 3D consistent results, and adheres more closely to the input image. We evaluate our approach on both synthetic data and in-the-wild images and demonstrate its superiority in terms of both mesh quality and runtime. In addition, our approach can seamlessly support the text-to-3D task by integrating with off-the-shelf text-to-image diffusion models.
Verifying my identity for @viamirror
sig:0x240920ca885369becd19e8a231a0280b803dcc5460ef194ac6850c361f8b68382ca6cc17bb50fa41ca0960dec344108cdf0fea98b5592718efc099cfa7ef30d61b
mirror.xyz
