
Building ML models for robotic arms at Dream Machines. Robotics MSc @ETH, prev Serenity @leggedrobotics. Into VLAs, RL.
Zurich, Switzerland
Joined April 2026
- Tweets252
- Following122
- Followers684
- Likes853
Check out the first results of policies trained with the custom built UMI device by @gino_kreis!
My first UMI diffusion policies for cube stacking:
v1: trained on human-speed demos, collected 1.6x faster than teleop. 91/100 success, and runs 2.4x faster than the teleop policy.
v2: trained on demos slowed to teleop speed. 100/100 success.
🧵ACT, delta policies
8
410
Insane how good of a simulation @ja_rothschild created with fable 5.2 in the 3 days it has been out!
Rollout on real robot vs same joint targets replayed in digital twin Fable 5.2 built for me. The goal is to do RL and evals for @dm_robots’ actuator unboxing in sim.
I ofc underestimated the challenge. See 🧵 for what I tried and how it went.
1
9
527
Real-Time EXPO-FT is another iteration in the EXPO series, this time tackling policy reactivity on dynamic tasks.
Sampling actions from a VLA like π0.5 takes time. By the time the actions arrive, the observation they were conditioned on is stale. To make the policy more reactive they apply two fixes:
First, the VLA is called asynchronously, ahead of time, conditioned on the actions that will be executed while inference is running (training-time RTC).
Second, the small edit policy is cheap enough to run at the control rate, so it corrects each candidate with the latest observation available after the actions from the base policy arrive.
To speed up training they introduce a separate critic for filtering candidates in noise space during the Bellman backup, so only the best one is denoised. (At test time all candidates are denoised.)
Other interesting things from the paper:
- control frequency 30 Hz, up from 10 Hz in EXPO-FT
- N = 32 candidates instead of 8, each with one edit, so 64 in total
- vision tower of the base policy is not frozen anymore
- no human interventions this time, as far as I can tell
- for some tasks the critics and the edit policy get privileged state information like the ball position and velocity for ball balancing, which I think is kind of a bummer
EXPO-FT brings EXPO to real-world tasks with action chunks and adds human interventions for faster learning.
The base policy (now π0.5), edit policy and critic are the same setup as in EXPO (see post below), but the critic and edit policy operate on the executed part C ≤ H of each action chunk. During online training a human can intervene with corrections. These corrections overwrite the policy’s actions and go into the replay buffer like any other transition. Even though the policy reaches the same success rate without interventions, they speed up the process.
Additional things I found interesting:
- learner (training and inference) and actor (sensors and motors) run on separate machines
- sparse binary reward is used for the tasks
- for the critic they use a separate leightweight ResNet-50 encoder while the π0.5 encoder stays frozen. The edit policy reuses the critic’s features
- 10 Hz end-effector velocity control (probably in base frame)
- OpenPI-style augmentation on both camera views
- even though the tasks look complex on first sight, the randomisation is very little and the tasks are very short
1
8
463
paper: arxiv.org/abs/2609.18207
code & visualisations: pd-perry.github.io/real-time…
authors: @perryadong @khhung906 @DorsaSadigh @chelseabfinn
98
EXPO-FT brings EXPO to real-world tasks with action chunks and adds human interventions for faster learning.
The base policy (now π0.5), edit policy and critic are the same setup as in EXPO (see post below), but the critic and edit policy operate on the executed part C ≤ H of each action chunk. During online training a human can intervene with corrections. These corrections overwrite the policy’s actions and go into the replay buffer like any other transition. Even though the policy reaches the same success rate without interventions, they speed up the process.
Additional things I found interesting:
- learner (training and inference) and actor (sensors and motors) run on separate machines
- sparse binary reward is used for the tasks
- for the critic they use a separate leightweight ResNet-50 encoder while the π0.5 encoder stays frozen. The edit policy reuses the critic’s features
- 10 Hz end-effector velocity control (probably in base frame)
- OpenPI-style augmentation on both camera views
- even though the tasks look complex on first sight, the randomisation is very little and the tasks are very short
EXPO is an algorithm to finetune expressive policies with RL.
Optimising a diffusion policy directly on Q means backpropagating through the whole denoising chain, which is unstable. The imitation loss on the other hand only ever touches a single denoising step. So @perryadong et al. keep the imitation loss on the diffusion policy and move the Q-maximisation somewhere else.
The algorithm uses a base policy trained with imitation learning (diffusion policy) to propose N actions (8 in the paper). A small gaussian edit policy nudges each one towards higher Q, which it can do directly as in SAC. A learned critic scores the original and edited actions and the best of the 2N actions is executed. The executed action is fed back into the replay buffer, which is used to continuously train the base policy with imitation learning. The base policy never receives a value gradient, the improvement arrives through the data.
The results in the paper all come from simple simulation tasks with single-step actions. This changes with EXPO-FT …
paper: arxiv.org/abs/2605.25477
code & visualisations: pd-perry.github.io/expo-ft/
authors: @perryadong @khhung906 @TianGao_19 @DorsaSadigh @chelseabfinn
61
EXPO is an algorithm to finetune expressive policies with RL.
Optimising a diffusion policy directly on Q means backpropagating through the whole denoising chain, which is unstable. The imitation loss on the other hand only ever touches a single denoising step. So @perryadong et al. keep the imitation loss on the diffusion policy and move the Q-maximisation somewhere else.
The algorithm uses a base policy trained with imitation learning (diffusion policy) to propose N actions (8 in the paper). A small gaussian edit policy nudges each one towards higher Q, which it can do directly as in SAC. A learned critic scores the original and edited actions and the best of the 2N actions is executed. The executed action is fed back into the replay buffer, which is used to continuously train the base policy with imitation learning. The base policy never receives a value gradient, the improvement arrives through the data.
The results in the paper all come from simple simulation tasks with single-step actions. This changes with EXPO-FT …
paper: arxiv.org/abs/2507.07986
code & visualisations: pd-perry.github.io/expo-site…
authors: @perryadong @qiyang_li @DorsaSadigh @chelseabfinn
83
















