Our latest work demonstrates how industry-scale Flywheel systems can iteratively drive model improvements even when the objectives are not easily tractable!
1/5 🤔 LLMs can solve olympiad math and write production code. But can they hold a conversation that's actually fun — one that people want to keep coming back to? 💬✨
We present CharacterFlywheel— an iterative process optimizing LLMs for real human engagement and character steerability, while maintaining rigorous safety protocols 🔒. Tested across Instagram, WhatsApp & Messenger 📱with millions of users — where they can create, share, and chat with their own AI characters 🤖.
📄 paper: arxiv.org/abs/2603.01973
huggingface.co/papers/2603.0…
1
81
Our latest work sheds light on what to "scale" when building generalizable agents:
👉 Prefer training envs with 📚 high state information richness (high perception load & info volume) + 🧠high planning complexity (long task horizon & branching factors)
👉 The *complexity structure* matters more than realism: Hard problems in "toy" domains like Sokoban / BlocksWorld can be more useful than easy problems in more realistic domains like ALFWorld.
‼️📉 Be careful about your mid-training datamix and strength: warmup and mid-training help prevent catastrophic forgetting during RL but undermines generalization to domains that are not covered
💡Applying lightweight state randomization/augmentation helps!
📄 Paper: huggingface.co/papers/2601.1…
🤔 Motivation:
Just like AlphaGo, given enough compute and proper configurations, RL agents can achieve superhuman performance in almost all training tasks ⚠️ However, LLM agents today are being used in tasks far beyond what limited post-training data can cover — a challenge highlighted by many researchers like @ilyasut, @MiniMax_AI (in platform.minimax.io/docs/gui…) and @Kimi_Moonshot in K2.5 tech report.
Thanks to @ZhihanLiu21628 for leading this systematic study. Joint work with
@EasonNie @KaiZhang_CS @zhaoran_wang @nazzhang
📄 Paper: huggingface.co/papers/2601.1…📷 Feedback & collaborations welcome
Lin Guan retweeted
People are putting lots of efforts into building diverse RL environments. But what kinds of environments are more useful for building **generalist** agents that are able to solve tasks beyond their training tasks?
🚀 Instead of optimizing one single benchmark, we look for drivers of transfer in our latest paper: huggingface.co/papers/2601.1…
Joint work with MSL @MetaAI (@ZhihanLiu21628 @GuanSuns @EasonNie @KaiZhang_CS @nazzhang) where @ZhihanLiu21628 interned. (1/n)
Lin Guan retweeted
📢 Yochanite Lin Guan (@GuanSuns; guansuns.github.io/), currently a Research Scientist with @Meta GenAI, will be defending his @SCAI_ASU PhD dissertation tomorrow 10/21 👇. He developed several 🔥 techniques for taming the notorious sample complexity of #RL systems..
While LLMs cannot verify the correctness of agent plans, they can be good at capturing the "style" of desirable behaviors.
Come stop by #COLM2024 poster #34 Wednesday Morning to see how we utilize VLMs as a knowledge source of common human preferences for embodied agents!
This is a joint work with @yfzhoucs @YantianZha @asurobot and @rao2z
👉Paper: openreview.net/forum?id=otKo…
🎞️ Homepage: guansuns.github.io/pages/vlm…
While LLMs cannot verify the correctness of agent behaviors, they can be good at capturing the "style" of desirable behaviors. We set out to find the effectiveness of VLM critics of undesirable behaviors.
💡Paper: arxiv.org/abs/2402.04210
👉Homepage (with visualizations): guansuns.github.io/pages/vlm…
1
2
163
This work is also part of an ongoing effort in our lab (supervised by @rao2z ) to identify constructive roles that LLMs/VLMs can serve in planning tasks.
👉 Relevant paper: arxiv.org/abs/2402.01817
1
155
Lin Guan retweeted
Robots now understand decisions via SERLfD – Self-Explanation for Reinforcement Learning from Demonstrations, transcending traditional robot learning from human demos. Check out our article and slides; find more insights at poster 409 during AAAI-24 on 2/23, 7-9 PM!🤖🌐 #AAAI2024
Lin Guan retweeted
📢📢 Come stop by #AAAI2024 poster spots 644 & 645 (the very last spots) this evening to hear me explain the neat work of
👉@ZahraZ__ & @sailiks on explaining allocations to humans
👉@YantianZha & @GuanSuns on using self explanations to learn from ambiguous demonstrations
[Note that at my request, the posters were placed in contiguous spots instead of their original spots 409/435 so I didn't have to master the art of being in two places at one time.. 😋]
Lin Guan retweeted
📣Chalk it to mad masochism, but we will be updating and reprising our tutorial on LLMs and Planning at #AAAI2024 Vancouver, BC. (Wed Feb 21st, 2-6PM 👉aaai.org/aaai-conference/aaa…. We are told that ~300 folks signed up already..😱). w/ @karthikv792 @GuanSuns
📢Don't miss out our poster session tomorrow (Dec 12) starting at 5:15 p.m. CST in the Great Hall & Hall B1+B2 #1525
This is a joint work with @karthikv792 @sarath_ssreedh and @rao2z
Paper homepage: guansuns.github.io/pages/llm…
Presentation at NeurIPS 2023: neurips.cc/virtual/2023/post…
205
📢 Our #ICLR2023 paper on learning and leveraging relative behavioral attributes to tame the sample complexity of #RLHF (joint with @karthikv792 & @rao2z ) is now featured on @mtlaiethics ! 1/
👉 montrealethics.ai/relative-b…



















