Ang Cao (@AngCao3) profile picture

Research Scientist @Google Deepmind | Pre. Ph.d at university of Michigan Question the Question

Mountain View, CA
Joined September 2019
Photo by Ang Cao (@AngCao3)Photo by Ang Cao (@AngCao3)Photo by Ang Cao (@AngCao3)Photo by Ang Cao (@AngCao3)Photo by Ang Cao (@AngCao3)Photo by Ang Cao (@AngCao3)Photo by Ang Cao (@AngCao3)Photo by Ang Cao (@AngCao3)Photo by Ang Cao (@AngCao3)Photo by Ang Cao (@AngCao3)
World models promise to predict the future. But can they predict the RIGHT future given the CURRENT state? Excited to share FlipTest, my work from Google DeepMind internship during the summer. In the physical world, 1 mm decides whether an object stays or falls. When a tiny change crosses a tipping point, a true world model must flip its prediction. Blog: romyluo.com/blog/fliptest/ Do today’s video world models pass the test? 🧵[0/8]
8
24
2
171
10,324
We've released a demo of real-time world generation powered by our new Atlas model! This is a spiritual successor to our RTFM demo from last year. It's a vision for the future of interactive media might look like, with frames streamed directly from a powerful generative model
Two months ago, we introduced Atlas, an autoregressive world model with pixel-perfect camera control. Today, we're releasing Frameboy, a demo allowing you to explore worlds in real-time with a highly optimized version of Atlas, streaming from cloud GPUs at up to 15 fps.
4
10
101
14,391
I did say we'd be releasing our models periodically...introducing Gemini 4 Carbon, our next frontier model. Following Argon's release just two weeks ago, Carbon shows the pace at which we’re pushing the frontier, it ranks #1 on the recently updated v5.0 of @ArtificialAnlys Intelligence Index at 59. It also leads across key cybersecurity evaluations scoring 88.2% on SEC-bench Pro for finding vulnerabilities, 54.8% on ExploitGym for exploitation, and 71.0% on CWE-bench for patching. We’re seeing significant improvements in long-horizon tasks even compared to Argon, and we’ve strengthened frontier safeguards. As we did with Argon, we’re going to roll it out gradually + responsibly. Some trusted cyber defenders will get access starting this week. We'll be expanding it to more partners through our Fairwind Program, ahead of wider availability soon, so hang tight!
2,068
2,732
1,920
33,978
7,003,634
Today we are launching FrontierCS 2, the most frontier benchmark for AI agents in computer science R&D. We built 48 tasks from published papers across 13 areas of CS, refining each one with experts in its field. Excited to co-launch FrontierCS 2 with @kaggle @GoogleDeepMind
23
51
14
347
52,920
Ang Cao retweeted
Been cooking for a while, and super proud of what our team built 🧑‍🍳👩‍🍳 Check this out 👇 More details dropping soon! 🦾
Amazed by Astra controlling robots? It turns out other LLMs can do it too – even the text-only ones like DeepSeek. We also find Astra did best watching and correcting every move live; Fable did best building its own tools and policy in code first. We introduce SEAR, a new benchmark and framework to rigorously evaluate and study how LLM agents self evolve on robots. 🔁📈The loop: within a controlled time budget, an agent and its library update a policy → the policy runs in simulation or on hardware → a judge returns feedback → the library carries the skills and lessons into the next task.  💻🚏 Self-evolving agents operate in myriad ways - we study all of them at once: the policy can be code the agent wrote (code-as-policy), the agent itself in the control loop (agent-as-policy), or a VLA it trained (autoresearch). 🧑‍🔬🤖We spent 4+ years of accumulated agent evolving time across simulation and real hardware and tested 7 frontier models (GPT-6-Astra, Fable 5.1, Opus 5, GPT-5.6-sol, Gemini-3.6-flash, Kimi-k3, DeepSeek-v4-flash) on 330 sim tasks, 2 physics engines, and real robots with different embodiments.  🌐Project overview: sear.bot  A thread 🧵1/5
3
3
3
51
4,849
Ang Cao retweeted
Always thought GRPO-style RL can't be optimal. With billions going into self-play & RSI, the underlying RL methods need to be efficient. Very exciting project 🔥: 1. Using critics to supervise policy 2. Exploiting easy resets for LLMs 3. Sharing params between Q & policy
Do you really need to finish every rollout in RL for LLM? Not if you make your critic better—and trust it! We propose Actor-Critic with Action Chunking (AC2): use a learned critic to score chunks of tokens => only need partial rollouts ⇒ train faster than GRPO!
23
59
524
47,722
As AI systems enter a recursive self-improvement loop, the central question is whether it can systematically move beyond human-designed methods to genuinely extend the scientific and intelligence frontier. What’s missing is a neutral, open standard for evaluating these capabilities in real, production-scale intelligence development. Today, we’re releasing OpenRSI-Index v0.1: evaluating whether AI can recursively improve itself and push the boundaries of intelligence and science, on production-scale clusters with 1k GPUs. We turn fully open-source projects into autoresearch environments, with agent trajectories lasting 60+ hours. Building v0.1 took 100K+ H100-hours. We’re building an ecosystem with and for the research community: let RSI benefit everyone, and let everyone shape RSI together. We invite task contributors and compute partners to build this open benchmark with us - all contributors will be included as paper authors. Shape RSI with us: 🌐 Website: index.openrsi.foundation/ind… 🛠️ GitHub: github.com/OpenRSI-Foundatio… 🤝 Contribute: github.com/OpenRSI-Foundatio…
18
59
32
233
136,544
Wow, I'm honored and humbled that our 2016 perceptual losses paper with @AlexAlahi and @drfeifei was recognized with a Test of Time Award by @eccvconf! It blows my mind that this is still a core ingredient in many modern pipelines, including the VAEs used in diffusion models
11
8
153
10,062
Ang Cao retweeted
i gave astra a robot, a paint brush, and a camera then asked it to paint the golden gate bridge in real life! it figured out how to control the robot, and progressively got better throughout its attempts. the timelapse is sick
This is GPT-6 Astra. Anything you can do on a computer, Astra can do for you. Fast.
667
1,691
755
20,790
5,082,141
Can video generation models do for vision what LLMs did for language? Introducing GenCeption from @GoogleDeepMind: one feed-forward video model for various vision tasks — SOTA, data-efficient, and emerging behaviors (ECCV 2026) 🌐 genception.github.io (1/8)
17
98
19
465
118,675
🧐A question I've long been interested in: how can we learn from human hands and transfer that directly to robots? Our new work, HUG, makes it possible in three simple steps: (1) collect human grasps at scale, (2) learn from them, and (3) retarget for deployment.
5
24
2
117
10,746
Ang Cao retweeted
🚀 HPC-AI is heading to #CVPR2026! Find us at Booth 237 from June 3–7! Whether you're building AI Agents or hunting for top-tier compute, we’ve got you covered. 🎁 The Ultimate CVPR Swag & Perks: 1️⃣ Limited Swag: Grab custom caps, canvas bags 🧢 🛍️ 2️⃣ Top-up Reward: 💰$15 = GPU Hour + Model APIs + Summer mini-fans / Bluetooth speaker (Only 20 set)🔥 3️⃣ Accepted Author Special: Top up $40 Get $20 voucher 💰 4️⃣ Survey Gift: Just do a quick survey, choose your refrigerator magnets 🧲 Stop by, talk tech, and claim your rewards! 🤝 📍 Booth 237 | June 3–7 | Colorado Convention Center #CVPR2026 #HPCAI #AI #MachineLearning #GPU #ModelAPIs
1
120
169
2,277
Ang Cao retweeted
Karpathy's prediction about RL is coming true now! He called reward functions unreliable and argued that a single reward number is too low-dimensional to teach an agent what "good" means for complex tasks. To solve this, Agents need a knowledge-guided review as a higher-dimensional feedback channel. Every major AI lab trains models with RL today (OpenAI, Anthropic, DeepSeek). And their key bottleneck has always been the reward functions. GRPO by DeepSeek worked well for math and code because the environment gave a binary signal. But for real agent tasks, someone still has to hand-code the scoring function. That takes days and breaks every time the pipeline changes. RULER (implemented in OpenPipe ART, 10k stars) addresses the exact problem Karpathy identified. The reward criteria are defined in plain English, and an LLM evaluates each trajectory against that description to provide feedback for training. I trained a Qwen3 1.4B agent that plays 2048 using GRPO with this exact workflow. In this case, the agent saw the board, picked a direction, and RULER evaluated the outcome, all from this natural language definition. You can see the full implementation on GitHub and try it yourself. Here's the ART Repo: github.com/OpenPipe/ART (don't forget to star it ⭐ ) Just like RLHF replaced manual rankings and GRPO replaced the critic model, natural language rewards are replacing hand-coded scoring functions. RL reward engineering is now prompt engineering. I wrote a full walkthrough covering RL for LLM agents, from RLHF to GRPO to RULER, in the article below.
57
183
9
1,588
351,560
Ang Cao retweeted
The most popular way to interpret AI is missing the bigger picture. Models think in curved shapes. But sparse autoencoders (SAEs) work with straight lines. Can they still capture models’ curved neural geometry? Yes, but not how you might think! (1/7)
Neural networks might speak English, but they think in shapes. Understanding their rich *neural geometry* is key to understanding how they work – and to debugging and controlling them with precision. Starting today, we’re releasing a series of posts on this research agenda. 🧵
24
151
34
1,017
178,051
We’re dropping Gemini Omni: our first step towards a model that can create anything from anything - starting with video. It combines Gemini’s intelligence with our generative media systems - representing a leap forward in world understanding, multimodality, and editing 🧵
413
1,228
469
8,359
1,671,102
Ang Cao retweeted
Introducing Visual Test-time Scaling for GUI Agent Grounding (ICCV'25, completed prior to the release of OpenAI-O3) When "thinking with images", the key chanlleging is designing the action in pixels space. We can zoom into regions of varying sizes and shapes, apply image transformations, and even use generative models to edit regions. Yet, O3 models often perform meaningless image adjustments. Our strategy is deliberately simple: when the GUI agents hesitates, we zoom into a single focal point predicted by the model, highlight coordinates as landmarks ("image-as-map"), and retry—no heavyweight tricks. This minimalist approach significantly boosts performance for both UI-TARS and QWen-2.5-VL 72B models: 📈 +28% on ScreenSpot-Pro 📈 +24% on WebVoyager w/ @lajanugen @jcjohnss @honglaklee
3
8
1
45
141,411
Ever wish YouTube had 3D labels? 🚀Introducing🎥DynPose-100K🎥, an Internet-scale collection of diverse videos annotated with camera pose! Applications include camera-controlled video generation🤩and learned dynamic pose estimation😯 Download: huggingface.co/datasets/nvid…
2
38
5
173
43,330
Introducing VGGT (CVPR'25), a feedforward Transformer that directly infers all key 3D attributes from one, a few, or hundreds of images, in seconds! No expensive optimization needed, yet delivers SOTA results for: ✅ Camera Pose Estimation ✅ Multi-view Depth Estimation ✅ Dense Point Cloud Reconstruction ✅ Point Tracking Project Page: vgg-t.github.io/ Code & Weights: github.com/facebookresearch/…
21
189
27
1,243
203,990
We are releasing uCO3D! Built to supercharge 3D GenAI and digital-twin models, this evolution of CO3D features more and higher-quality object videos from 1k categories, 3D Gaussian Splats, and streamlined OSS tools. 💻Data&code: github.com/facebookresearch/… 📄Paper: arxiv.org/abs/2501.07574
1
24
2
133
16,749
Ang Cao retweeted
UnCommon Objects in 3D
3
9
1
53
11,980