Dejia Xu (@Ir1dXD) profile picture

✨ @Meta MSL | Prev: Led Agentic Robotics Research @LumaLabsAI. https://teapawt.net/t.co/bCoyXJpKDU @WNCG_UT @VITAGroupUT

Redwood City, CA
Joined August 2015
Photo by Dejia Xu (@Ir1dXD)Photo by Dejia Xu (@Ir1dXD)Photo by Dejia Xu (@Ir1dXD)Photo by Dejia Xu (@Ir1dXD)Photo by Dejia Xu (@Ir1dXD)Photo by Dejia Xu (@Ir1dXD)Photo by Dejia Xu (@Ir1dXD)Photo by Dejia Xu (@Ir1dXD)Photo by Dejia Xu (@Ir1dXD)Photo by Dejia Xu (@Ir1dXD)
Been cooking for a while, and super proud of what our team built 🧑‍🍳👩‍🍳 Check this out 👇 More details dropping soon! 🦾
Amazed by Astra controlling robots? It turns out other LLMs can do it too – even the text-only ones like DeepSeek. We also find Astra did best watching and correcting every move live; Fable did best building its own tools and policy in code first. We introduce SEAR, a new benchmark and framework to rigorously evaluate and study how LLM agents self evolve on robots. 🔁📈The loop: within a controlled time budget, an agent and its library update a policy → the policy runs in simulation or on hardware → a judge returns feedback → the library carries the skills and lessons into the next task.  💻🚏 Self-evolving agents operate in myriad ways - we study all of them at once: the policy can be code the agent wrote (code-as-policy), the agent itself in the control loop (agent-as-policy), or a VLA it trained (autoresearch). 🧑‍🔬🤖We spent 4+ years of accumulated agent evolving time across simulation and real hardware and tested 7 frontier models (GPT-6-Astra, Fable 5.1, Opus 5, GPT-5.6-sol, Gemini-3.6-flash, Kimi-k3, DeepSeek-v4-flash) on 330 sim tasks, 2 physics engines, and real robots with different embodiments.  🌐Project overview: sear.bot  A thread 🧵1/5
3
3
3
51
4,817
Fable 5.2's launch impressed with strong robot manipulation. How does ZAP, our 27B model, do on the same tasks? We recreated the tasks in sim and ran both through the same harness (the model is the policy, calling robot tools directly). ZAP only trained on sim pick-and-place, so every object and goal is new to it. Task 1, open a bottle: ZAP is 2.5x faster and costs $0.43 vs. Fable's $6.50. It plans the grasp up front, and it pays off: a firm hold and clean turns. Fable drops the bottle, recovers nicely, and still finishes. Sim only for now since I'm at CoRL, but the real robot is waiting back home 👀 🧵 1/4
Astra made robot control look like a job for frontier LLMs. But small VLMs already know what to do. They just don't know how. Turns out, simulation is all you need. Meet ZAP. No real robot data, no frontier teacher. Matches Astra on real tasks, 3× faster, 17× cheaper. 🧵 (1/11)
3
10
47
2,854
Introducing pi-H3⚡, the fastest MiniMax H3 yet. 10 s of 768p video with audio generated in 9.2 s on a single B200. How we did it: Direct pixel output in 2 or 4 steps, no video VAE decoder. Blog: hanshengchen.com/blog/pi-h3/ App: huggingface.co/spaces/Lakoni… @StanfordAILab @LumaLabsAI
38
48
19
647
49,534
Dejia Xu retweeted
It's small. It's fast. It has places to be.
1/ announcing our smallest, most efficient little model is here: muse ember 🔥 built for work that agents repeat all day, it gets you more intelligence per dollar than Haiku, Luna, and all other flash models at any reasoning setting and does it at a third of the cost or less!
4
34
942
Our first version of Muse Ember is here 🔥 More to come 🫡 We’d love your feedback!
1/ announcing our smallest, most efficient little model is here: muse ember 🔥 built for work that agents repeat all day, it gets you more intelligence per dollar than Haiku, Luna, and all other flash models at any reasoning setting and does it at a third of the cost or less!
2
2
48
1,698
Dejia Xu retweeted
Our first version of Muse Ember 🔥 Ember aims to maximize intelligence per GW of inference to help bring personal superintelligence to everyone. We would love all the feedback to keep hill-climbing. Please send it our way. We will keep hill-climbing on efficiency, user experience, and better approach to train with internal frontier! More to come 🫡
1/ announcing our smallest, most efficient little model is here: muse ember 🔥 built for work that agents repeat all day, it gets you more intelligence per dollar than Haiku, Luna, and all other flash models at any reasoning setting and does it at a third of the cost or less!
22
30
5
273
24,682
Dejia Xu retweeted
What is going to come next when the small and fast llm is very intelligent?
Our first version of Muse Ember 🔥 Ember aims to maximize intelligence per GW of inference to help bring personal superintelligence to everyone. We would love all the feedback to keep hill-climbing. Please send it our way. We will keep hill-climbing on efficiency, user experience, and better approach to train with internal frontier! More to come 🫡
5
4
1
47
5,299
Come say hi! Presenting ZAP at 12-1:30 at Griffin Hall, poster 134
Who says robot control needs a giant model? ZAP, a small VLM, matches Astra on real robot tasks - 3× faster and 17× cheaper. Presenting tomorrow at #CoRL2026. Come chat! 📍 Griffin Hall · Poster 134 🗓️ Mon, Nov. 9 · 12:00–1:30 pm
2
19
545
Dejia Xu retweeted
Come to see our poster at 133 @ Griffin Hall!
Hello Austin 👋 Excited to be at #CoRL2026! Catch me presenting SEAR today at lunch (Griffin @ JW Marriott). We look at what LLM agents' learning curves reveal about how they tackle robotics. Let's chat!
1
2
15
398
We are presenting SEAR: Self-Evolving Agentic Robotics at #CoRL2026 today! 🤖 🗓️ Late-Breaking Results poster, during lunch 📍 JW Marriott, Griffin, board #133 Come say hi 👋
2
1
21
1,246
Who says robot control needs a giant model? ZAP, a small VLM, matches Astra on real robot tasks - 3× faster and 17× cheaper. Presenting tomorrow at #CoRL2026. Come chat! 📍 Griffin Hall · Poster 134 🗓️ Mon, Nov. 9 · 12:00–1:30 pm
Astra made robot control look like a job for frontier LLMs. But small VLMs already know what to do. They just don't know how. Turns out, simulation is all you need. Meet ZAP. No real robot data, no frontier teacher. Matches Astra on real tasks, 3× faster, 17× cheaper. 🧵 (1/11)
5
3
31
2,003
Sad to miss CoRL this year, but excited to share two projects my collaborators will present on Monday! 🤖 𝗦𝗘𝗔𝗥: Are robot agents actually learning from experience, or just getting more chances to try? SEAR introduces a benchmark and harness to distinguish genuine learning from repeated attempts, and tests whether agents can transfer what they learn to new tasks, simulators, and real robots. sear.bot/ 𝗭𝗔𝗣: You don’t need a frontier LLM to control a robot. Trained entirely in simulation, our small VLM matches Astra on real-world tasks, with no real robot training data or frontier teacher, 3× faster and 17× cheaper. zero-real.github.io/ Both were accepted to CoRL Late-Breaking Results: 📍 Monday, Nov. 9, 12:00–1:30 pm at Griffin Hall SEAR: Poster 133 | ZAP: Poster 134 Stop by and chat with the team, and enjoy beautiful Austin! (1/3)
4
4
1
40
1,934
Check these threads for more details on SEAR: teapawt.net/BangzhengL/status/2107… teapawt.net/XinyiWang98/status/211… Joint work with the amazing SEAR team: @XinyiWang98, @BangzhengL, @hywang66, @LinZ23754317, @andrea_cui123, @TangZhenggang and Shyamal Buch (2/3)
An LLM robot agent’s success rate goes up as it practices. Did it learn anything, or did it just try more often? And do the skills and memories it saves still help on new tasks, in a new simulator, on a real robot? We built SEAR to answer both. To our knowledge, it’s the first large scale controlled study of self-evolving robot agents. The paper is now on arXiv 📄 The agent improves a robot in one of three ways: it writes code that becomes the policy, it acts live in the control loop, or it trains its own VLA. 7 frontier LLMs, 330 sim tasks, 5 real-robot tasks, 4+ years of agent runtime, with a matched control for every comparison and a sealed judge on unseen scenes. A few findings surprised us 🧵 1/7 arxiv.org/abs/2610.18792
1
1
4
7,551
For more on ZAP, check out this thread: teapawt.net/bentropy_ai/status/211… The awesome team behind ZAP: @bentropy_ai, @djsaunde, @mirceamironenco, Salman Mohammadi, @doublemathew, @srzhu97, @_DivyaMakkar, @xuhuahuang01, @qinboli, @nsupreeth5, @censiyuan1, Shyamal Buch and @thomasneff93 (3/3).
Astra made robot control look like a job for frontier LLMs. But small VLMs already know what to do. They just don't know how. Turns out, simulation is all you need. Meet ZAP. No real robot data, no frontier teacher. Matches Astra on real tasks, 3× faster, 17× cheaper. 🧵 (1/11)
4
225
Dejia Xu retweeted
Proud of this work! This effort combined a lot of performance and systems work on robotics simulators with state of the art async RL methods, training small reasoners to match or beat strong closed source models. Hoping to share more about our performance work soon 👀
Astra made robot control look like a job for frontier LLMs. But small VLMs already know what to do. They just don't know how. Turns out, simulation is all you need. Meet ZAP. No real robot data, no frontier teacher. Matches Astra on real tasks, 3× faster, 17× cheaper. 🧵 (1/11)
1
12
422
Really excited about these results! Our teams across research & systems cooked up a recipe to teach a small 27B model to do fast and efficient robot control, on par with frontier LLMs! ZAP will be presented at CoRL next week - check out our work - happy to chat :)
Astra made robot control look like a job for frontier LLMs. But small VLMs already know what to do. They just don't know how. Turns out, simulation is all you need. Meet ZAP. No real robot data, no frontier teacher. Matches Astra on real tasks, 3× faster, 17× cheaper. 🧵 (1/11)
4
32
1,534
We developed ZAP, a small VLM with our own simple harness, no real robot data, no frontier teacher, but we successfully matched Astra on real tasks and 3x faster. Check it out zero-real.github.io/! It will be presented at CoRL 2026 late-breaking results session.
Astra made robot control look like a job for frontier LLMs. But small VLMs already know what to do. They just don't know how. Turns out, simulation is all you need. Meet ZAP. No real robot data, no frontier teacher. Matches Astra on real tasks, 3× faster, 17× cheaper. 🧵 (1/11)
1
12
315
You don’t always need a 𝐟𝐫𝐨𝐧𝐭𝐢𝐞𝐫 𝐦𝐨𝐝𝐞𝐥 to control a robot. With 𝐳𝐞𝐫𝐨 𝐫𝐞𝐚𝐥-𝐫𝐨𝐛𝐨𝐭 𝐭𝐫𝐚𝐢𝐧𝐢𝐧𝐠 𝐝𝐚𝐭𝐚, our Qwen-based ZAP matched GPT-6 Astra on 10 real-world tasks, with ~𝟑× 𝐟𝐚𝐬𝐭𝐞𝐫 𝐫𝐞𝐬𝐩𝐨𝐧𝐬𝐞𝐬 at 𝟏/𝟏𝟕 𝐭𝐡𝐞 𝐜𝐨𝐬𝐭. Accepted to 𝐂𝐨𝐑𝐋 𝟐𝟎𝟐𝟔 𝐋𝐚𝐭𝐞-𝐁𝐫𝐞𝐚𝐤𝐢𝐧𝐠 𝐑𝐞𝐬𝐮𝐥𝐭𝐬! zero-real.github.io/ It takes a village to raise a robot 🤖 We started this journey back in May, and it’s been amazing to see it come together. 🥺 Huge thanks to our incredible team for all the hard work, creativity, and collaboration along the way!
Astra made robot control look like a job for frontier LLMs. But small VLMs already know what to do. They just don't know how. Turns out, simulation is all you need. Meet ZAP. No real robot data, no frontier teacher. Matches Astra on real tasks, 3× faster, 17× cheaper. 🧵 (1/11)
5
13
162
16,761
Congrats Rundong!!
1/7 Excited to share SceneRig! 🦾 One image → an editable, simulation-ready 3D scene. We combine AI agents with geometry and physics to reconstruct scenes for robot policy evaluation and replay of real-world manipulations. Here’s how it works 🧵👇
5
600
How much of your robot’s intelligence is actually the simulator giving it answers? Exact object poses. Ground-truth success signals. Real robots don’t get those answers for free. A recurring concern as I read agentic robotics work in simulation. 🧵 1/5
2
4
117
5,924
A policy trained with simulator feedback may transfer to hardware. That doesn’t establish that its self-improvement loop transfers too. If the loop needs ground-truth success to choose its next revision, what supplies that signal on the real robot? 4/5
1
1
159
Privileged information can be valuable training supervision, especially when deployment no longer needs it. The real test is whether robots can keep improving on real hardware. Our recent paper studies exactly this. 🧵 A few findings: teapawt.net/XinyiWang98/status/211… 📄 Paper: arxiv.org/abs/2610.18792 🌐 Videos & interactive figures: sear.bot 5/5
An LLM robot agent’s success rate goes up as it practices. Did it learn anything, or did it just try more often? And do the skills and memories it saves still help on new tasks, in a new simulator, on a real robot? We built SEAR to answer both. To our knowledge, it’s the first large scale controlled study of self-evolving robot agents. The paper is now on arXiv 📄 The agent improves a robot in one of three ways: it writes code that becomes the policy, it acts live in the control loop, or it trains its own VLA. 7 frontier LLMs, 330 sim tasks, 5 real-robot tasks, 4+ years of agent runtime, with a matched control for every comparison and a sealed judge on unseen scenes. A few findings surprised us 🧵 1/7 arxiv.org/abs/2610.18792
1
5
438