https://teapawt.net/t.co/h9S0y91CAA
Joined August 2026
- Tweets49
- Following30
- Followers346
- Likes23
5. Does the upgrade impact our old results?
We reran 22 configurations across 7 tasks. All outcomes held, only in our pushing task we observe a slight deviation: A block finishes on the opposite side of the channel. We traced this to MuJoCo 3.12's box-collider rewrite.
1
3
125
@mujoco 4.0 further widens the possibilities for simulation-based evals and training.
We look forward to full support for speculative contacts and IPC cloth in MuJoCo Warp. We see this as another step in closing the sim2real gap.
Find our original comparison here: mandarobotics.com/blog/compa…
2
120
Manda retweeted
A peg-insertion policy that scores 49/50 in MuJoCo succeeds 11/50 in PhysX. Same task, different physics engine.
They also found the same authored foot friction resolves to 0.8/0.6 in PhysX and 1.0 in Newton, and PhysX-trained ANYmal-D policies use about 35% more joint torque.
Running a policy in a second engine looks like a cheap check before trying it on hardware.
References
mandarobotics.com/blog/polic…
5. The behavior of generalist policies can diverge in some scenes:
We tested π0.5 in PhysX and Newton. It succeeds in 80–100% of episodes in both engines, but Newton had failed grasps in 41 matched episodes, compared to 13 in PhysX.
1
2
574
Understanding the effects of simulation engines and parameters is crucial for generalizable reinforcement learning and evaluation.
Read our full report where we compare sim engines & settings on insertion, quadruped locomotion, and full-scene transfer:
mandarobotics.com/blog/polic…
8
550
Gemini 4 Argon has an edge over GPT 6 Astra on the RoboLab tasks we tested.
Using our own harness, Rung, we ran both models on the exact same 99 starting setups across 22 tasks. Gemini completed 73/99 episodes (74%). Astra completed 55/99 (56%):
1. Gemini was a bit more careful about lining up its grasp:
6. Inference time is still nowhere near being useful for actual robot execution.
For about 40 seconds of robot motion, Astra spent an average of 4.6 minutes on model calls per task, Gemini 10.5 minutes.
Astra made about 19 calls per task, Gemini only made 6.
1
1
60
A lot of the performance differences we observe between frontier models depend on the harness they're running on. We expect huge gains from harness engineering, in-context learning, and inference speed-ups.
If you're testing models in simulation, or want early access to our testing harness Rung, reach out to us at mandarobotics.com/collaborat…
1
51