Goodeat (@Goodeat258) profile picture

Mingyang Deng PhD candidate@MIT

Joined August 2022
Goodeat retweeted
LLM pretraining has run on single-epoch text, bits per byte and iso-FLOP curves for years. visual generation still memorizes ImageNet for tens to hundreds of epochs and pushes FID down with CFG, with no strong signal those tricks transfer to real text-to-image. new blog: 🎨 scaling generative models on GPIC, 100M permissive text-image pairs. training pixel-space JiT from 3.6e18 to 2.3e20 FLOPs: 1) held-out loss is our main metric, and compute-optimal TPP sits at about 50 image tokens per image parameter 2) EMA is not necessary: a proper learning-rate schedule beats EMA on held-out loss at every budget by 0.4% to 1.0% 3) we use CFG = 1 (only the conditional branch) for every result, and call on everyone to drop CFG: it saves compute and leaves no room to hack the metric all code, configs and sweep data are open 🧵 blog: wenhaochai.com/blogs/scaling…
1
3
12
310
Goodeat retweeted
Meet Pika H3 Flex, debuting today as the #1 open weight model on DesignArena’s t2v board. It’s built for professional iteration, with exportable audio stems, resolutions up to 4k, and faster-than-realtime generation. It can produce a 15-second clip with soundtrack in as little as 7 seconds. So, yeah. It’s kind of a flex 💅
393
395
403
4,902
1,521,388
Goodeat retweeted
VISTA for ARC-AGI-3 is open source! Code: github.com/joshhhhhan/VISTA
How does a model understand and explore a world it has never seen? We introduce 🔭VISTA🔭, a visual harness that gives a VLM long-horizon vision for reasoning in an interactive world. With Claude Opus 5.0, it reaches 100% RHAE on @arcprize's ARC-AGI-3, perfectly solving all 25 public games. Blog post: vista-research.github.io/ 🧵
1
13
2
88
11,438
We share our early research on building Software World - a "GitHub" run by agents. We deploy agents for Python packages in a dependency chain, and each agent is tasked with collaborating with others and optimizing the package it owns.
9
54
12
254
40,313
Goodeat retweeted
Diffusion/flow matching is a go-to generative AI paradigm, but it's slow, sample inefficient and complicated. Do we really need all this complexity? We design a small and simple one-step model, ROMS-IMLE, which attains an FID of 2.56 on ImageNet. Joint w/ @researchirag 1/6
7
46
4
417
37,737
Goodeat retweeted
The first long-horizon visual harness to achieve a perfect 100% score on ARC-3! 🚀 Excited to share our latest work from Kaiming’s group. Huge thanks to my incredible co-authors Josh and Linlu!
How does a model understand and explore a world it has never seen? We introduce 🔭VISTA🔭, a visual harness that gives a VLM long-horizon vision for reasoning in an interactive world. With Claude Opus 5.0, it reaches 100% RHAE on @arcprize's ARC-AGI-3, perfectly solving all 25 public games. Blog post: vista-research.github.io/ 🧵
2
18
1
153
33,588
We’ve released the code for Drifting Models :) Includes full training, inference, and pretrained weights. Curious to see what people build on top of this. github.com/lambertae/driftin…
4
26
2
221
14,150
Thanks for the highlight, Zhengyang! Huge credit to my collaborators He Li (@lihe50hz), @TianhongLi6, @du_yilun, and my advisor Kaiming He. In particular, He Li is exceptional and is currently applying for PhD programs.
A new paradigm & member toward 1-step & e2e generative modeling! Great work by @Goodeat258 Mingyang!!! cannot be more excited to read me: learning to drift with my spindrift. arxiv.org/abs/2602.04770
4
11
2
107
29,792
New research with @AdtRaghunathan, Nicholas Carlini and Anthropic! We built ImpossibleBench to measure reward hacking in LLM coding agents 🤖, by making benchmark tasks impossible and seeing whether models game tests or follow specs. (1/9)
13
66
10
470
74,400
Excited to share our work with my amazing collaborators, @Goodeat258, @SimulatedAnneal, @zicokolter, and Kaiming. In a word, we show an “identity learning” approach for generative modeling, by relating the instantaneous/average velocity in an identity. The resulting model, termed MeanFlow, demonstrates the best 1-step generative performance so far as a standalone model trained from scratch. It exploits a simple fact by the Fundamental Theorem of Calculus: differentiation and integration are inverse operators. This creates a "complexity-gap," differentiation-integration gap: differentiation is easier than integration. Much like the verification-generation gap, we are able to unlock fully tractable learning thanks to the beautiful “asymmetry” in nature/math. No pre-training/post-training. No distillation. No curriculum. Simple identity. Simply training from scratch. Please check out the paper for the best clarity : ) 📜Paper: arxiv.org/abs/2505.13447
5
40
7
159
33,601
Mitigating racial bias from LLMs is a lot easier than removing it from humans! Can’t believe this happened at the best AI conference @NeurIPSConf We have ethical reviews for authors, but missed it for invited speakers? 😡
172
772
337
3,689
2,229,095
In diffusion models, samplers are primarily ODE-centric, overlooking slower stochastic methods. However, we show that stochastic sampler can outperform previous samplers on Stable Diffusion, if we use stochasticity correctly! check out Restart Sampling: arxiv.org/abs/2306.14878
2
33
4
165
48,101