@pantheoninc, prev posttrain @spacexsi
Joined January 2025
- Tweets24
- Following500
- Followers1.2K
- Likes5.2K
1st place with 1.77x speedup over SGLang on the prefill layer leaderboard 🚀🚀
We're launching a $100,000 challenge to see who can make Muse Spark run fastest: openkernelchallenge.dev/
Eric Li retweeted
We recently published the eight-week process behind scaling our UMI dataset to 1M+ tasks and 45k+ new tasks a day.
Now, we’re open-sourcing DiverseUMI-100, a collection of 12,000+ tasks from a 100-hour sample of our dataset, complete with dense annotations by Argus, our advanced robotics annotation pipeline.
How do you build the most diverse UMI dataset in robotics?
In the last 8 weeks, we scaled an international UMI data collection operation from 5 to 90 operators, built out a dataset of over 1M+ unique tasks, and continue to capture over 45k+ new tasks each day.
Robotic World Models require much higher diversity of tasks than imitation learning. Here's the why and how behind our operations. 🧵
Eric Li retweeted
Another cool update from the robotics world! While we're all wow-ed in mental land, there's a ton of progress and i feel like acceleration happening in physical land too now.
My guess is in good part also (but not only) thanks to the coding models progress over the last year.
How do you build the most diverse UMI dataset in robotics?
In the last 8 weeks, we scaled an international UMI data collection operation from 5 to 90 operators, built out a dataset of over 1M+ unique tasks, and continue to capture over 45k+ new tasks each day.
Robotic World Models require much higher diversity of tasks than imitation learning. Here's the why and how behind our operations. 🧵
- $10/hr and dropping
- 90 international operators and growing
- No two tasks exactly alike
- Maximum customizability (any change to the environment/operator instructions/etc. is a single phone call to our wonderful team in Mexico)
- Ungodly amounts of custom camera firmware written for max operational efficiency
- A self built 20 petabyte data cluster to store all of this
At Pantheon, we like to bet on things that scale.
This has been and will continue to be an unbelievable push by the team. Infinite moving parts and countless people involved.
How do you build the most diverse UMI dataset in robotics?
In the last 8 weeks, we scaled an international UMI data collection operation from 5 to 90 operators, built out a dataset of over 1M+ unique tasks, and continue to capture over 45k+ new tasks each day.
Robotic World Models require much higher diversity of tasks than imitation learning. Here's the why and how behind our operations. 🧵
Eric Li retweeted
How do you build the most diverse UMI dataset in robotics?
In the last 8 weeks, we scaled an international UMI data collection operation from 5 to 90 operators, built out a dataset of over 1M+ unique tasks, and continue to capture over 45k+ new tasks each day.
Robotic World Models require much higher diversity of tasks than imitation learning. Here's the why and how behind our operations. 🧵
This is why we open source! Good optimizations from the @MidcenturyAI team 😃
We spent the weekend experimenting with Argus and pushed it to SOTA on WGO-Bench.
Our optimized version scores 50.2% semantic F1, up from the 29.8% out-of-the-box score, and well ahead of other leading annotation services.
All of the gains came from how the model sees the video. We gave it denser frames, a second look at each action's boundaries, and a final check on every label.
Thanks to @PantheonInc @calixo888 for open-sourcing Argus.
The full repo is coming soon. For now, you can read our tech report here: midcentury.xyz/blog/optimizi…
Even with attempts to standardize robot learning data, things remain too heterogenous. ex - since LeRobot v3 concatenates episodes into shared MP4s, treating the end timestamp as inclusive would append the first frame of the next episode to the previous. Whereas MCAP stores timestamped messages across separate channels, meaning camera and state streams must be aligned explicitly.
Beyond faithful episode reconstruction, metadata fields vary massively both across data types (UMI/teleop/human ego) and within them. ex, Galaxea records mobile base state/chassis IMU, while say build100k is effectively pure RBG.
The goal with Argus is to provide a unified annotation harness over these datasets.
Our belief is that as (a) VLMs continue to push the Pareto frontier of multimodal understanding/cost (as they've demonstrated over the last 3 months) and (b) there exists a harness to provide said VLMs with all of a dataset’s relevant context, dense annotation is a solved problem.
Argus: an open-source robotics data annotation and quality pipeline.
With new frontier VLMs like GPT-6 Astra, we can generate rich, high-quality annotations for robotics data for both training and dataset analysis. Argus delivers detailed, timestamp-level annotations while catching issues like mislabeled instructions, sped-up recordings, swapped camera streams, and unflagged operator mistakes.
To support better data quality for the robotics community, we’re open-sourcing Argus for anyone to use 🧵
Eric Li retweeted
We (@PantheonInc) have been working on a robotics data quality pipeline that uncovered a series of major problems in public robotics datasets, especially for world modeling.
To improve the quality of data available to open-source robotics, we're publishing annotations for four of the most popular datasets. Some examples of issues, and our report 🧵