Gemini RL @GoogleDeepMind

Singapore
Joined October 2021
🪂Understanding R1-Zero-Like Training: A Critical Perspective * DeepSeek-V3-Base already exhibits "Aha moment" before RL-tuning?? * The ever-increasing output length in RL-tuning might be due to a BIAS in GRPO?? * Getting GRPO Done Right, we achieve a 7B AIME sota! 🧵 📜Full details: github.com/sail-sg/understan… 🛠️Code: github.com/sail-sg/understan…
29
187
37
1,356
332,137
It’s an awesome co-worker, and hope you enjoy Argon as much as we do!
Introducing Gemini 4 Argon – our new frontier model. It’s built for complex workflows across coding, enterprise knowledge work, and cybersecurity defense – rolling out today to a set of trusted testers through our Fairwind Program.
5
1
65
2,719
Zichen Liu retweeted
Gemini 4 is here! 🚀 And it outperforms astra/fable on many things 😇🤓
Introducing Gemini 4 Argon – our new frontier model. It’s built for complex workflows across coding, enterprise knowledge work, and cybersecurity defense – rolling out today to a set of trusted testers through our Fairwind Program.
12
9
1
222
12,741
Maybe self centering is a way to optimise my life trajectory, in Chinese it’s 知行合一
1
1
24
3,316
Zichen Liu retweeted
HyCreator: an agent harness for long video generation 🚀 End2end Auto Mode ⏲️ Generate 10-minute scale films with zero human intervention 👨‍🦰 Switch to real-time interactive editing whenever you want Apply for early access: hycreator.tencent.com/
34
62
17
379
151,307
Zichen Liu retweeted
🚀 Will Agentic ESOpt Be the Future of Long-Horizon Agentic Fine-Tuning? Excited to share our new work on Arxiv. Agentic ESOpt: Fine-Tuning Long-Horizon LLM Agents with Minimal GPU Memory Requirements As LLM agents become longer-horizon and more branching, conventional Agentic RL faces three growing bottlenecks: high training/memory cost and increasingly challenging long-horizon credit assignment. Our Agentic ESOpt approaches agent fine-tuning from the parameter space instead, targeting three advantages: ⚡ Model Scalability — only minimal inference-level GPU memory requirement 🔧 Flexibility — prompt/skill–parameter co-evolution ⏳ Long-Horizon Scalability — trajectory-level parameter attribution without horizon-wise decomposition There are impressive empirical results, and we believe ES may be a better-matched parameter fine-tuning paradigm as agents become larger and longer-horizon. 📄 Paper: arxiv.org/abs/2608.17310 🌐 Project: zz1358m.github.io/Project-Ag… 💻 GitHub: github.com/zz1358m/Agentic-E… 🤗 Models & Checkpoints: huggingface.co/collections/z… Code, training logs, and checkpoints have been fully open-sourced. Main takeaways ↓
6
21
4
68
34,581
Continuous self-improvement needs an ever-expanding supply of training environments (goals). SPADE: one model self-plays the Environment Designer and the Reasoning Agent, writing executable, agentic environments that get harder as it improves. Environment scaling on its own. ♠️
20
127
34
703
180,518
Zichen Liu retweeted
PI is cooking again, multi-agent RL support just landed in verifiers + prime-rl. Glad to see SPIRAL's Role-conditioned Advantage Estimation (RAE) featured and implemented for self-play!
1
4
40
6,350
Welcome submissions if you work on meta-agents!
🚀Call for Papers — @NeurIPSConf 2026 Workshop Workshop on Responsible Use of Meta-Agents 📅 December 11/12 · 📍 Sydney, Australia Join us to shape the future of Meta-Agents and Harness. Topics include but are not limited to: 1. Automated Design of Agent Harnesses 2. Optimization of Agentic Systems 3. Recursive Self-Improvement 4. Evaluation and Benchmarks for Meta-Agents 5. Misalignment and Safety for Meta-Agents 6. Governance & Human Oversight of Agents 🌟 Featuring an all-star speaker lineup: Graham Neubig @gneubig (CMU) Chelsea Finn @chelseabfinn (Stanford) Jenny Zhang @jennyzhangzt (Recursive) Robert Lange @RobertTLange (Sakana AI) Bo Li @uiuc_aisecure (UIUC) Hancheng Cao @CaoHancheng (Emory) Yuandong Tian @yuandongtian (Recursive) Chen Sun @ChenSun92 (Google DeepMind) Proudly sponsor by @metaai and @e2b Updates: Follow us here & spread the word! #NeurIPS2026 #MetaAgents #LLMs #Harness #RSI #SelfImprovement
1
9
1,839
Zichen Liu retweeted
Announcing Discovery Loop! I am very excited to announce that, along with my longtime friends and collaborators @Sanjay_Ghemawat, @OriolVinyalsML and @quocleix, we are founding Discovery Loop (@DiscoLoopAI), a Public Benefit Corporation whose mission is to automate machine learning, science, and engineering to accelerate discoveries and progress. The four of us have worked together for 14 to 30 years, and have helped build some of the world’s most used products, infrastructure and AI models, and we’re excited to turn our attention to this ambitious endeavor. ♾ Learn more at: discoveryloop.com
897
2,159
967
21,761
6,747,486
Zichen Liu retweeted
Hello. If you're ultra exceptional ai researcher or engineer and want to work with me and my team in gdm sg Gemini team, please reach out.
23
11
2
344
59,807
Zichen Liu retweeted
A few months ago, @qx_dong turned me from a pure IC into a manager. 🤗 Was a really fun experience so far watching people join one by one, building the team culture together and going up on the frontline. 🫠 Unprecedented moment in the history of Singapore to have so many talented researchers in one place. I'm glad to be building this legendary team. 🫡
Overdue update: I’ve joined @GoogleDeepMind as a research scientist to work on 💎Gemini post-training. Feeling incredibly fortunate to be cooking models with such a brilliant team, under the leadership of Yi Tay (@YiTayML) and Quoc Le (@quocleix). In the past a few month, we've found so much enjoyment in working with our own models and keeping pushing their boundaries. Ps, there's a special joy in creating recipes and internally naming them after our favorite drinks🧋 Looking forward to keep enjoying research, pushing the frontiers of Gemini and seeing the magic unfold on the path to AGI!
6
5
154
31,221
Zichen Liu retweeted
Our latest deliveries: 👨‍💻 UniRL, an RL infra for unified multimodal models ➕ Two new RL algorithms: • FlowDPPO for diffusion and flow-matching models • DRPO for LLMs and VLMs Check out our repo: github.com/Tencent-Hunyuan/U…
🚀Introducing UniRL, an RL infra for unified multimodal models. Together with two new RL algorithms: DRPO and Flow-DPPO. One RL loop across diffusion/flow matching models, LLMs/VLMs, and unified multimodal models👇 Code: github.com/Tencent-Hunyuan/U… (yes — U(you)-ni-(need) RL 😉)
4
28
1
159
21,731
Zichen Liu retweeted
We propose DRPO: a soft version of DPPO🔥 Since PPO, clipping/mask-based trust regions have long outperformed smooth divergence regularization like KL, even though the latter one feels more principled. 👺 We found two missing pieces:👇 1️⃣ Weight the regularizer by |advantage| - Otherwise, the trust region geometry changes dynamically and optimization becomes unstable. 2️⃣ Use the right divergence - What matters is not just “regularization”, but the trust-region geometry induced by the gradient. DPPO-style geometry works much better than PPO-style geometry in LLM. These insights lead to DRPO, which delivers the most robust and best overall performance across algorithms, even outperforming original mask-based DPPO. 🚀 This project is an amazing collaboration with @ExplainMiracles, @NickZhou523786, Wee Sun Lee, Liefeng Bo, @TianyuPang1 . Do follow them if you are interested in this work! 📄 Paper: arxiv.org/pdf/2606.09821 💻 Code: github.com/Tencent-Hunyuan/U…
3
27
2
155
12,546
Zichen Liu retweeted
🧵 For 2 RL checkpoints trained differently, you can just weight extrapolate them and it works! Bonus: these extrapolated checkpoints are complementary policies -> Get exploration and diversity for free -> Better inference scaling when ensembling Paper: arxiv.org/abs/2605.28751
arxiv.org/abs/2605.28751 Now many studies try to do extrapolation through model merging. arxiv.org/abs/2605.26484
3
29
5
126
16,575
Zichen Liu retweeted
Today at Google I/O, we introduced Gemini 3.5 Flash! It has become an integral part of our daily research cycle and works with all the tools we have at Google. We used a team of agents in Antigravity 2.0 to recreate the original AlphaZero research paper and build a playable version. They coded the reinforcement learning pipeline in JAX/Flax, trained a ResNet model from scratch via self-play on multi-TPU pods, and shipped a full-stack web app so you can play against it, from just 2 prompts. . Here’s what else makes 3.5 Flash special 🧵
116
77
10
578
114,326
rl intuition (up-scaled by the correctness of infra) is all you need when cooking with a strong base model such as gemini✨
3
1
71
5,387
Zichen Liu retweeted
🚀 Excited to share RetroAgent: From Solving to Evolving via Retrospective Dual Intrinsic Feedback! Instead of training LLM agents just to solve isolated tasks, we designed them to continuously evolve. By using retrospective dual intrinsic feedback (Numerical for exploration + Language for explicit memory) & SimUtil-UCB retrieval, we achieve massive SOTA gains over GRPO baselines: 🔥 +18.3% on ALFWorld 🛒 +15.4% on WebShop 📦 +27.1% on Sokoban 💣 +8.9% on MineSweeper Strong test-time adaptation & OOD generalization included! 🧠👇 📄 Paper: arxiv.org/abs/2603.08561💻 Code: github.com/zhangxy-2019/Retr…
1
1
7
1,238
Zichen Liu retweeted
Text agents have their Gym. Vision agents? Not until now. Introducing Gym-V — a unified gym-style platform for agentic vision research, with 179 procedurally generated environments across 10 domains. One API to rule them all: 📦 Offline dataset 🤖 Agentic RL training 🔧 Tool-use training 👥 Multi-agent training 📊 VLM & T2I model evaluation All under the same reset/step interface. Key findings: 1. Observation scaffolding matters MORE than RL algorithm choice 2. Broad curricula transfer well; narrow training causes negative transfer 3. Multi-turn interaction amplifies everything 📄 Paper: arxiv.org/abs/2603.15432 💻 Code: github.com/ModalMinds/gym-v Open the thread for a deep dive! 🧵
8
17
2
109
10,088
🦎🦎 Happy to see two of our works (DrGRPO & DPPO) are highlighted here! I don’t think changing a few terms is worth a new branding, so we respectfully kept predecessors’ name while highlighting the correction/improvement on top of them. Hopefully they inspire RL algo designs.
Finally finished! If you're interested in an overview of recent methods in reinforcement learning for reasoning LLMs, check out this blog post: aweers.de/blog/2026/rl-for-l… It summarizes ten methods, tries to highlight differences and trends, and has a collection of open problems
2
4
43
4,527