Research Lead @USC_ISI and Research Assistant Professor @CSatUSC PhD at CMU ML/NLP @LTIatCMU @CarnegieMellon
Los Angeles, CA
Joined June 2015
- Tweets429
- Following451
- Followers1.9K
- Likes1.4K
Pinned Tweet
As promised, we open-sourced our training infra, xLLM: github.com/ifm-ai/xllm
together with an efficient attn backend, xattn: github.com/ifm-ai/xattn
and a ckpt conversion & serving toolkit, xBridges: github.com/ifm-ai/xbridges
Rework is inevitable in large-scale LLM pre-training and fine-tuning. What it costs isn't.
Introducing xLLM, an efficient and flexible infrastructure for pre-training and fine-tuning dense and MoE LLMs. It keeps key training decisions changeable without giving up throughput.
Efficient, at 6,295 tokens/sec per GPU on K2-Horizon-MoVA-36B-A4B and 10,050 tokens/sec per GPU on Llama3-8B on H200s.
Flexible, because the tokenizer, data mixture, model architecture, and training stages can change without rebuilding the dataset or the system around them.
xLLM ships with the checkpoints, training logs, and recipes behind K2 Horizon: github.com/ifm-ai/xllm
K2-Horizon-MoVA-36B-A4B: huggingface.co/IFM/K2-Horizoโฆ
Xuezhe Ma (Max) retweeted
So happy to work with Cheng again and again! ๐
Sadly I can't make it to Budapest this time, but my junior labmate from Tsinghua, Meng Gao @Gladysgaom, will be there to help present, along with her own first-author work, Skim and Skip: Hierarchical Adaptive Inference for Efficient Multimodal Retrieval. She's also applying to PhD programs for Fall 2027, so please go say hi!
Heading to #EMNLP2026 in Budapest ๐ญ๐บ? Check out our Oral paper, UReason: Benchmarking Reasoning-to-Generation Alignment in Unified Multimodal Models!
๐ค UMMs unify understanding and generation. But are their text and images truly aligned?
๐ก Our finding: UMMs usually get the reasoning right, but the image often doesn't follow it. Even more surprising: if we drop the reasoning and give the model only the final prompt it wrote itself, every model we tested does better, by a large margin.
๐ Oral Session 2 ยท Oct 25, 11:00โ12:30 ยท Room G13-14
๐
Xuezhe Ma (Max) retweeted
The most detailed open LLM data release yet: K2 Horizon's pre-training data and post-training data-generation pipelines are now open!
13 trillion pre-training tokens across 5 datasets and 24 subsets. Over 10 trillion tokens of synthetic data spanning mathematics, reasoning, coding, planning, and problem-solving.
Beyond the scale, four highlights for model builders (details in the threads):
- Scaling synthetic data without losing diversity. Our generation methods produce diverse training examples at scale, nearly matching natural web text in measured diversity.
- More verification isn't always better. Strict verification can throw away useful training signal.
- Small-scale wins can mislead. Our chosen data mix actually lost in smaller experiments.
- Beyond teacher imitation. Our post-training data generation methods produce training trajectories the teacher couldn't generate on its own.
Open pre-training datasets, post-training data-generation pipelines, and processing tools for researchers to inspect, reuse, and build on.
Pre-training data:ย huggingface.co/collections/Iโฆ
Post-training data-generation code & tools:ย github.com/ifm-ai/DATA_NAVIGโฆ
Xuezhe Ma (Max) retweeted
LLM training is about transporting high bandwidth of digits across hardware. This exercise allows us to understand a lot of the bottlenecks in this process.
Our upcoming technical paper (warning: it is loooong) will also share more!
We learned a TON in the last 6 months about Agentic RL infrastructure at scale, and now we're happy to share it with everyone.
RL360 was used for the training of K2 Horizon series (artificialanalysis.ai/modelsโฆ). It builds on top of miles from @radixark, adding a lot of optimizations for efficient agentic RL at scale.
Our amazing team at @IFM_AI built this with a lot of hard work. Thanks to @varad0309 @willis_guo @wang1999_zt @mingshanhee @DavidRBellamy @omkarenator @_matthewyang @qiancheng1231 @bhandari_jalaj and many others!
Xuezhe Ma (Max) retweeted
We're open-sourcing RL360, the RL infrastructure behind K2 Horizon's six domain experts: 8,192 concurrent sandboxes, 150 steps in 7 days, ~1.3M rollouts.
Long-horizon agentic RL runs thousands of agents in real sandboxes on tasks that take minutes. Some of those sandboxes crash mid-task, and a crash arrives as the same low score as a wrong answer.
Without infrastructure that can keep up, GPUs are underutilized and the model gets penalized for system crashes rather than its own mistakes.
RL360 is built for this. We share how it works, what broke at scale, and what we learned in the thread.
Code: github.com/ifm-ai/RL360
Run: wandb.ai/mbzuai-llm/rl360-puโฆ
xLLM 1.0.1 has been released
github.com/ifm-ai/xllm/releaโฆ
As promised, we open-sourced our training infra, xLLM: github.com/ifm-ai/xllm
together with an efficient attn backend, xattn: github.com/ifm-ai/xattn
and a ckpt conversion & serving toolkit, xBridges: github.com/ifm-ai/xbridges
Xuezhe Ma (Max) retweeted
You might think that @ericxing, one of the most prominent advocates of open-source, would support Chinaโs approach to AI, since all of its leading models are open. Instead, he says those models give the illusion of openness, but actually hold their users hostage.
We had a fascinating conversation about the world models heโs building, and his strong views on open-source: youtube.com/watch?v=kuVzFoIlโฆ
Xuezhe Ma (Max) retweeted
Thrilled to introduce our new paper, Recursive Self-Improvement in Unified Multimodal Models! ๐
If a model trains on data it made itself, how does it know that data is any good? A lot of work on recursive self-improvement keeps running into this question.
For images, our answer was to step into another modality. You can't really verify an image just by looking at it, but you can run the code that draws it and check what comes out. And a unified multimodal model can do both: it draws an image, looks at it to spot what went wrong, and writes code aimed at exactly those mistakes.
So vision catches the mistakes, text writes the fix, and running the code makes sure the fix is actually right. Each ability ends up teaching the others.
Round after round, the model keeps getting better at drawing, reading, and coding, and every round starts from a stronger model than the last.
The nice part is the split: the model gets to pick what it needs to learn, but it never gets to decide what's correct. The code does.
Led by @lemonnoel52 , in collaboration with @ChengYANG_yc , @yaokangwu , @BergKirkpatrick , and @MaxMa1987 .
Huijuan is at #COLM2026 this week. If you're there, come find her and chat about recursive self-improvement in UMMs!
Xuezhe Ma (Max) retweeted
Can a unified multimodal model (UMM) teach itself, with its text and visual abilities making training data for each other?
Yes: over 4 rounds, a 7B model climbs 45.7% โ 60.2%, vs. 46.3% for continued training.
๐งต 1/10
Xuezhe Ma (Max) retweeted
Scaling up a model has meant paying twice, in compute and in memory. Our researchers show that looped models can pay in compute alone, using the loop's fixed point as a shortcut.
A 1.6B looped model runs twelve blocks deep on four blocks of memory. On the same tokens, it beats a four-block Transformer by 17.5% and comes within 2.4% of a twelve-block one needing 3x memory. It can also train, pre-fill, and post-train without paying for every loop.
Paper: arxiv.org/abs/2610.06833
Fully open-sourced at: github.com/ifm-ai/xllm-loop
๐ ๐๐ก๐๐ญ ๐ข๐ ๐ฅ๐จ๐จ๐ฉ๐๐ ๐๐๐ฌ ๐๐จ๐ฎ๐ฅ๐ ๐ฎ๐ฌ๐ ๐ร ๐๐๐ฐ๐๐ซ ๐ฉ๐๐ซ๐๐ฆ๐๐ญ๐๐ซ๐ฌ ๐๐ง๐ ๐ ๐ร ๐ฌ๐ฆ๐๐ฅ๐ฅ๐๐ซ ๐๐ ๐๐๐๐ก๐ ๐๐ง๐ ๐ฌ๐ญ๐ข๐ฅ๐ฅ ๐๐ ๐๐จ๐ฆ๐ฉ๐๐ซ๐๐๐ฅ๐ ๐ญ๐จ ๐ ๐ฌ๐ญ๐๐ง๐๐๐ซ๐ ๐๐ซ๐๐ง๐ฌ๐๐จ๐ซ๐ฆ๐๐ซ?
Introducing ๐๐๐ญ๐ก๐ข๐ง๐ค๐ข๐ง๐ ๐๐ญ ๐
๐ข๐ฑ๐๐ ๐๐จ๐ข๐ง๐ญ๐ฌ. We argue that looped LMs don't have to pay for every loop. They can scale by adding FLOPs at constant memory.
The key is the fixed point of the loop. We introduce:
โ ๐
๐ข๐ฑ๐๐-๐๐จ๐ข๐ง๐ญ ๐๐ก๐จ๐ซ๐ญ๐๐ฎ๐ญ๐ฌ: what fixed points buy a looped model in training (pre-training, post-training) and inference (prefill, decoding)
โ A ๐ฅ๐๐๐ซ๐ง๐๐ ๐๐๐ฉ๐ญ๐ก ๐ฉ๐ซ๐ข๐จ๐ซ and ๐จ๐ซ๐ญ๐ก๐จ๐ ๐จ๐ง๐๐ฅ ๐ข๐ง๐ฃ๐๐๐ญ๐ข๐จ๐ง: two ways to shape those fixed points
๐งต Long thread ahead (0/n).
Xuezhe Ma (Max) retweeted
We trained an LLM with FlashAttention-3 in BF16 ๐ฅ Training looked healthy for a long time. Then the grad norm shot up 1000ร ๐ and the loss ended 0.2 nats above FP32 attention. Not a single NaN ๐ถ
The cause is in the attention backward ๐งต
๐ arxiv.org/abs/2609.34272
RT @huskydogewoof: ๐ Paper: alphaxiv.org/abs/2610.loopeโฆ (arXiv to be released)
We also fully open-sourced our code, data, and checkpoints:
๐ปโฆ
This quoted post is unavailable.
Excellent work led by my intern @huskydogewoof @IFM_AI
๐ ๐๐ก๐๐ญ ๐ข๐ ๐ฅ๐จ๐จ๐ฉ๐๐ ๐๐๐ฌ ๐๐จ๐ฎ๐ฅ๐ ๐ฎ๐ฌ๐ ๐ร ๐๐๐ฐ๐๐ซ ๐ฉ๐๐ซ๐๐ฆ๐๐ญ๐๐ซ๐ฌ ๐๐ง๐ ๐ ๐ร ๐ฌ๐ฆ๐๐ฅ๐ฅ๐๐ซ ๐๐ ๐๐๐๐ก๐ ๐๐ง๐ ๐ฌ๐ญ๐ข๐ฅ๐ฅ ๐๐ ๐๐จ๐ฆ๐ฉ๐๐ซ๐๐๐ฅ๐ ๐ญ๐จ ๐ ๐ฌ๐ญ๐๐ง๐๐๐ซ๐ ๐๐ซ๐๐ง๐ฌ๐๐จ๐ซ๐ฆ๐๐ซ?
Introducing ๐๐๐ญ๐ก๐ข๐ง๐ค๐ข๐ง๐ ๐๐ญ ๐
๐ข๐ฑ๐๐ ๐๐จ๐ข๐ง๐ญ๐ฌ. We argue that looped LMs don't have to pay for every loop. They can scale by adding FLOPs at constant memory.
The key is the fixed point of the loop. We introduce:
โ ๐
๐ข๐ฑ๐๐-๐๐จ๐ข๐ง๐ญ ๐๐ก๐จ๐ซ๐ญ๐๐ฎ๐ญ๐ฌ: what fixed points buy a looped model in training (pre-training, post-training) and inference (prefill, decoding)
โ A ๐ฅ๐๐๐ซ๐ง๐๐ ๐๐๐ฉ๐ญ๐ก ๐ฉ๐ซ๐ข๐จ๐ซ and ๐จ๐ซ๐ญ๐ก๐จ๐ ๐จ๐ง๐๐ฅ ๐ข๐ง๐ฃ๐๐๐ญ๐ข๐จ๐ง: two ways to shape those fixed points
๐งต Long thread ahead (0/n).
Xuezhe Ma (Max) retweeted
Most decisions don't need words. ๐คซโก
Meet K2-Type-0.9B ๐ง : an open decision model. One state + many typed questions in, calibrated probabilities out, in a single forward pass. No text generated.
Apache-2.0 weights ๐
huggingface.co/IFM/K2-Type-0โฆ
#OpenSource #LLM #AI
Xuezhe Ma (Max) retweeted
๐๏ธ Mon Oct 5, 8โ10pm PT
AMA with the team behind K2 Horizon, hosted by r/LocalLLaMA (@LocalLLaMAsub).
Ask us anything about models, deployment, training, what's open so far and what's next.
๐ฌ Questions are open now. We start answering when we go live.
Drop your questions here ๐ย redd.it/1wv8zww
๐ป
This is a very late comment, and a long one.
At EMNLP 2024 in Miami, I got to talk with Max. I found it hard to believe that someone as senior as him was still on the front line, writing infra code himself. That was when I made up my mind: I wanted to do my PhD at USC, with Max.
After I started in 2025, I got to build xLLM(github.com/ifm-ai/xllm), our pre-training framework, with Max(@MaxMa1987) and many great collaborators. More and more code today is written by models, and vibe coding is everywhere. Max still reviews every single xLLM PR and goes through every detail of the library with me.
I honestly don't know if this way of working fits this era. But I deeply respect it. Tools will keep changing. The care behind good work, I believe, will not.
Thank you to all the xLLM collaborators, and to everyone in Max Lab(@jin_lynn808 @lemonnoel52 @tao_muzi @Shicheng_Wen @MengxiWu567 @Wendy_xwz @nuan_wen @sunjiao123sun_, NancyXu and Ming) for so much support along the way. I'm also very grateful to Hector Liu(@waterluffy), Eric Xing(@ericxing), and IFM(@IFM_AI) for supporting this full open-source release: the training infra, the attention backend, and the checkpoint toolkit, together.
I know there are still many things we're not satisfied with. But I believe going fully open, even when it's not perfect, is the right direction. The field moves forward when people can see, question, and improve the whole stack, not just the final model.
If you're at COLM 2026, go meet Max and talk with him about xLLM.
Today I finished my PhD qualifying exam, with Max as my committee chair. Maybe a PhD degree really means nothing these days. But I didn't come for the degree. I came to learn from someone who still builds things with his own hands, and I'm still glad I made that choice in Miami.
P.S. Glad I got X Premium, or I couldn't have written this much.
Xuezhe Ma (Max) retweeted
Two years ago, @ChengYANG_yc and I built ChartMimic (arxiv.org/abs/2406.09961) to test whether models understand the structure behind a chart, by asking them to rebuild it in code. InfoEdit, our sequel with @lemonnoel52 , asks the harder question: can they keep that structure intact while changing it?
This matters more now. Image models draw infographics straight in pixels; slides, diagrams, even videos are increasingly written as code. Both look great on the first draft. The test is the second: in a photo, a good edit leaves the rest alone; in an infographic, the rest has to move with it.
Neither path is there yet, and they break in different places. So I don't think the answer is pixels or code. It's models that understand what they draw, however they draw it.
#EMNLP2026
๐ Thrilled to introduce our work InfoEdit: Probing Global Layout Reasoning in Infographic Editing (EMNLP 2026).
AI can now help us generate posters, infographics and slides, through image generation (Nano Banana) or code generation (Codex).
๐คBut can AI faithfully edit them, whether through image editing or multimodal code editing? When editing, one change forces the whole layout to adapt. We call this reflow, and we built InfoEdit to put it to the test.
๐งฉ What is reflow?
Take a simple edit: insert one new step into a cycle diagram. It sounds like adding one box. But a correct edit moves every node, reroutes every arrow, and rebalances the spacing, all while keeping existing content intact.
๐ InfoEdit at a glance
โข 1,000 infographics, in HTML and PowerPoint
โข 8 logical-relation families
โข 4,000 editing instructions
โข 4 tasks: Expand-Text ยท Insert-Element ยท Swap-Block ยท Reshape-Canvas
๐ฌ Image editors struggle
Across 8 frontier image editors, only GPT-Image-2 clears 60% average success; most fall below 7%. Swap-Block is the hardest: even when we mark the exact blocks to swap, no editor exceeds 35%. The bottleneck isn't finding the target. It's reflow.
๐ผ๏ธ vs ๐ป Pixels or code?
Editing the HTML/PPT source instead, Kimi-K3 reaches 67.9% average success, above GPT-Image-2's 62.0%, and lifts Swap-Block from 25.5% โ 62.8% Yet pixel-level editing still wins on Expand-Text and Reshape-Canvas. The two paradigms are complementary.
๐ก Takeaway
Editing structured visuals isn't just changing the requested edits. It takes reasoning about how the whole design should adapt.
#EMNLP #multimodal #coding #imagediting
Thanks @huskydogewoof for making this great video of xLLM
As promised, we open-sourced our training infra, xLLM: github.com/ifm-ai/xllm
together with an efficient attn backend, xattn: github.com/ifm-ai/xattn
and a ckpt conversion & serving toolkit, xBridges: github.com/ifm-ai/xbridges
Xuezhe Ma (Max) retweeted
Our training library xLLM is out ๐ In addition to being performant, I wanted to highlight the flexibility. Itโs not just a โnice to have.โ In our training runs, we kept finding small bugs in the chat template (our final Jinja is 1k lines ๐ฌ). Being able to adjust the tokenizer and chat template quickly is what allowed us to fix them while training.
xLLM supports fine-tuning on chat and instruction-style data by keeping the tokenizer and data mixture changeable through its online pipeline. It handles tokenization, mixing, and packing during training, so a new tokenizer or chat template is just a config change, not a reprocessing run:
โข Tokenization runs straight from JSONL
โข Data mixtures are configurable, with buffered shuffle
โข Bestfit packing keeps documents intact with little padding when they fit within a sequence
Throughput is unaffected, because parallel loading and asynchronous preparation overlap the CPU work with GPU training.