PhD student @UTCompSci | Learn to understand ourselves and build intelligence.🤖🧠👁️
Austin, TX
Joined July 2023
- Tweets564
- Following3.1K
- Followers823
- Likes9.7K
some results are pretty interesting
Today we are introducing Dyna-2, a world-action model pre-trained on one million hours of human video. At this scale, for the first time, we discovered several new scaling laws:
• world-action models exhibit scaling law on human data across four orders of magnitude, from 1000 to 1,000,000 hours,
• this human data scaling law implied a scaling law on never seen robot data,
• both data and objective matter; world modeling and scaling on video data are essential for cross-embodiment scaling transfer to emerge
🧵
2
18
3,055
Our first-step paper towards science of co-training will present at ICML, 5-6:45pm on July 8th. I will stay in Seoul until Saturday, open to chat anything, including but not limited to:
- Pre-training
- Representation learning
- Continual learning
Excited to chat with all the great researchers!
Also, really sad to missed the signup of @SakanaAILabs event, is there any chance I can make it up?
🤖Co-training is everywhere (sim↔real[e.g. GR00T, LBM], human↔robot[e.g. PI, EgoScale], even non-robot data[e.g. PI, LBM).
But why does it work? How can we improve it further?
Taking sim-and-real imitation learning in diffusion/ flow-based models as the test bed, we performed a rigorous mechanistic analysis, drawing on theoretical insights and multi-layered experiments.
😮Key insight: it’s all about representations.
- Alignment → enables transfer
- Discernibility → enables adaptation
⚖️Both are necessary — it's better to have more aligned representations, but the model must be able to discern the domains. We term this as structured representation alignment.
⬇️Let’s take a deep dive into that:
Paper: arxiv.org/pdf/2604.13645
Website: science-of-co-training.githu…
Personally, I like this observation the most. It reminds me of the interesting visualization from @physical_int last Dec.
We provide a similar but complementary observation that: such an alignment process evolves progressively across the network layers.
The local geometric alignment is so fresh and interesting (at least for me).
Also, alignment in our case doesn't emerge from scaling, but from proper training settings.
Finding 2: Representation alignment can be learned implicitly and progressively!
In a proper range of mixing ratios, features of the shallow layers show local geometric alignment while features of deeper layers show global alignment, exhibiting a progressive alignment process, resulted implicitly by co-training.
Finding 2: Representation alignment can be learned implicitly and progressively!
In a proper range of mixing ratios, features of the shallow layers show local geometric alignment while features of deeper layers show global alignment, exhibiting a progressive alignment process, resulted implicitly by co-training.
🔍What’s really happening under the hood?
Think of policy learning as: o → z → a
Two intrinsic effects emerge:
- structured representation alignment: alignment vs. discernibility → in representation space (z)
- importance reweighting: mixing log-prob gradients across domains → in conditional action space (z→a)
🤖Co-training is everywhere (sim↔real[e.g. GR00T, LBM], human↔robot[e.g. PI, EgoScale], even non-robot data[e.g. PI, LBM).
But why does it work? How can we improve it further?
Taking sim-and-real imitation learning in diffusion/ flow-based models as the test bed, we performed a rigorous mechanistic analysis, drawing on theoretical insights and multi-layered experiments.
😮Key insight: it’s all about representations.
- Alignment → enables transfer
- Discernibility → enables adaptation
⚖️Both are necessary — it's better to have more aligned representations, but the model must be able to discern the domains. We term this as structured representation alignment.
⬇️Let’s take a deep dive into that:
Paper: arxiv.org/pdf/2604.13645
Website: science-of-co-training.githu…