Pinned Tweet
Introducing EFE-98: the most predictive public sim benchmark for real-robot performance.
EFE-98 has the highest correlation (0.98, p < 0.01) with real-robot rankings of any benchmark we index. It uses just 21.5% of Embodied First Exam’s tasks and runs at a tenth of the cost.
We had frontier LLMs compete in Jenga by controlling simulated Franka arms in a heavily tuned MuJoCo sim.
Even the best models only made it 3 moves in. Gemini 4 Carbon took over 5 hours on a single move, and Fable 5.2 lost after swinging its arm into the tower on its second.
Frontier models saturate nearly half of pick and place tasks and a third of precision tasks in the Robot Intelligence Index, but struggle to plan and execute multi-step, contact-sensitive motions.
Full rankings: withplayground.ai/
3
73
Playground retweeted
Congrats @withplayground and @jonmayxyz on EFE-98! Excited to see sim rankings line up with real-robot rankings: 0.98 correlation with 21.5% of the Embodied First Exam’s tasks!
We used GPT-6.2 Astra to reconstruct a real Jenga collapse in MuJoCo within 0.33° of studio footage.
Frontier models are now capable enough on parameter tuning tasks to reduce the sim-to-real gap by continuously calibrating simulators from real-world experience.
Astra closed the gap by modeling 54 measured blocks, 6-DoF finger motion, skin compliance, 0.20 mm presliding elasticity, and a 0.5 ms timestep. Wood friction stayed at 0.213.
More on our work to measure and close the sim-to-real gap: withplayground.ai/research/s…
EFE-98 is sampled to reproduce full EFE's spread and saturation rate, so its real-world signal carries at a tenth of the cost.
We validated EFE-98 because many gold-standard sim evals don’t track with real-world performance, and the ones that do can cost $10k+ per run.
Try it: withplayground.ai/leaderboar…
1
6
344
We're incredibly grateful to the Embodied First Exam team for maintaining the exam and open-sourcing their run data. EFE-98 is built to be a cost-effective companion to their benchmark.
Explore their work: embodied-first-exam.ai/
5
217
GPT-6.2 Astra is the new #1 model on the Robot Intelligence Index.
It tops every benchmark it’s entered to date, leads the real-robot ranking, and wrote the best robot code we’ve ever tested: 96.8% on 500 LIBERO trials, matching fine-tuned π0.5 at only 1.7 cents per success.
It's also cost-efficient. In sim it solved 2x the tests of GPT-6 Astra at a quarter of the cost: $6.66 per solve vs $25.02.
On real robots, it’s the top scoring model and fastest agent (163s per attempt vs Fable’s 269), though at nearly double Fable’s cost.
Full results: withplayground.ai/research/r…
Introducing Playground Serve: our internal robot inference stack, now free and open source.
It’s 64-72% cheaper than leading hosted providers on π0.5 and 5x faster than stock LIBERO on MolmoAct2. On average, it runs frontier policies for under $2/robot-hour in compute.
Try it: github.com/playground-system…
Serve natively supports 23 models and 8 families, with more being added as they join the Robot Intelligence Index.
We’re open sourcing Serve because many of the largest gains in inference come from better software, not bigger providers, and software can be shared.
Full blog: withplayground.ai/research/p…
4
209
GPT-6.2 Astra is the strongest robot-code author we’ve tested to date.
With no simulator, demos or web access, it wrote a 2.5B parameter neural net that solved 79% of 500 LIBERO trials, and Python code that solved 96.8%, beating Fable 5.2 (89.2%) and GPT-6 Astra (33.2%).
Astra 6.2 also led in our live control tests, solving 60% of trials vs Fable’s 50% at 7x less per success (75 cents vs $5.44).
But for every LLM we tested, writing a policy was 27-77x cheaper per success than live control.
Full results: withplayground.ai/research/h…
Full leaderboard and methodology: withplayground.ai/blog/robot…
4
193
SLA-Preview is the new #2 model on the Robot Intelligence Index, beating Dream-1 and every frontier LLM except Fable 5.2.
At just 3.35B parameters, it’s the top open-weight model, the only VLA in the top 10, and the only university-led model in the top 5.
With no extra parameters, SLA-Preview nearly doubles π0.5’s score on RoboDojo Real (42.3 vs 22.9) by training its vision to match a 3D vision model. Congrats to the SLA team at HKUST (Guangzhou) on the result!
Full leaderboard: withplayground.ai/blog/robot…
1
4
292
Full SLA (Spatial-Language-Action) paper: arxiv.org/abs/2611.08373
4
161
Frontier LLMs write robot policies that are 30-75x more cost-effective and 17-76x faster than the same LLM controlling a robot live.
Across 500 LIBERO trials, Fable 5.2 solved 89% of episodes with code it wrote and 93% with a hand-written neural net, but only 50% driving live.
Our results suggest existing robotics evals miss some of LLMs’ most useful skills. Most benchmarks score LLMs as live controllers, but in LIBERO that was the slowest, costliest way to use them, and for Fable, the least successful.
Full blog and replication bundle: withplayground.ai/blog/how-s…






