MLPC Group @UCSanDiego led by Prof. Zhuowen Tu (https://teapawt.net/t.co/2wv2tlpsi7)
San Diego, CA
Joined October 2024
- Tweets25
- Following30
- Followers43
- Likes6
MLPC Group retweeted
We put FLUX 3's native layout control to the test on OverLayBench, our benchmark for layout-to-image generation with dense overlaps. The results blew us away. 🔥 On our hardest split, @bfl_ai's FLUX 3 Image Klein hits 54.94 O-mIoU vs. 37.82 for the previous best (+17.12).
Excited to share this awesome new work from our lab! 🚀 Huge congratulations to the team!
Excited to share that our paper: OverLay++: Dense-Overlap Layout-to-Image Generation Dataset has been accepted to #NeurIPS2026 in the Evaluations & Datasets Track!
Generating images with multiple objects is challenging, especially when scenes involve dense layouts, overlapping objects, and complex spatial relationships. Even when an image looks visually appealing, getting every object, attribute, and position right remains a difficult problem. With OverLay++, we address this limitation by introducing a large-scale dataset designed to provide richer supervision for layout-to-image generation.
What makes OverLay++ different?
- Scale and complexity: Nearly 500K images, averaging 6.6 objects per image, covering diverse scene configurations.
- Richer annotations: 1.67× higher annotation density than existing datasets, complemented by detailed object-level captions.
- Improved generation quality: Training on OverLay++ improves object fidelity, attribute alignment, and spatial alignment, particularly in dense and overlapping scenes.
- Faster convergence: Models trained on OverLay++ converge faster across different levels of scene complexity.
📄 Paper: lnkd.in/gPfgpNxy
🌐 Project website: lnkd.in/g4p9ehZE
🤗 Dataset: lnkd.in/gqJFUABd
💻 Code: lnkd.in/gnFncnbu
📍 Meet us at NeurIPS 2026 in Sydney! We’ll be presenting our work during Poster Session 6 at PM AEDT in Hall 1–4. If you’re attending, please stop by. We’d love to discuss the work and hear your thoughts!
A huge thank you to my collaborators @shroglc, @dsrivastavv, @HaiyangXu3110, Bingnan Li, @zx1239856, @EthanArmand2, @chuanli11, @jianwen_xie, @zhuowent for their contributions, collaboration, and support throughout this project.
#NeurIPS2026 #ComputerVision #ImageGeneration #GenerativeAI #MachineLearning
MLPC Group retweeted
Excited to share that our paper: OverLay++: Dense-Overlap Layout-to-Image Generation Dataset has been accepted to #NeurIPS2026 in the Evaluations & Datasets Track!
Generating images with multiple objects is challenging, especially when scenes involve dense layouts, overlapping objects, and complex spatial relationships. Even when an image looks visually appealing, getting every object, attribute, and position right remains a difficult problem. With OverLay++, we address this limitation by introducing a large-scale dataset designed to provide richer supervision for layout-to-image generation.
What makes OverLay++ different?
- Scale and complexity: Nearly 500K images, averaging 6.6 objects per image, covering diverse scene configurations.
- Richer annotations: 1.67× higher annotation density than existing datasets, complemented by detailed object-level captions.
- Improved generation quality: Training on OverLay++ improves object fidelity, attribute alignment, and spatial alignment, particularly in dense and overlapping scenes.
- Faster convergence: Models trained on OverLay++ converge faster across different levels of scene complexity.
📄 Paper: lnkd.in/gPfgpNxy
🌐 Project website: lnkd.in/g4p9ehZE
🤗 Dataset: lnkd.in/gqJFUABd
💻 Code: lnkd.in/gnFncnbu
📍 Meet us at NeurIPS 2026 in Sydney! We’ll be presenting our work during Poster Session 6 at PM AEDT in Hall 1–4. If you’re attending, please stop by. We’d love to discuss the work and hear your thoughts!
A huge thank you to my collaborators @shroglc, @dsrivastavv, @HaiyangXu3110, Bingnan Li, @zx1239856, @EthanArmand2, @chuanli11, @jianwen_xie, @zhuowent for their contributions, collaboration, and support throughout this project.
#NeurIPS2026 #ComputerVision #ImageGeneration #GenerativeAI #MachineLearning
Excited to share this awesome new work from our lab! 🚀 Huge congratulations to the team!
I've long believed hierarchical representations are one of the most promising paths forward for vision problem. QuadTok is our first step: a tokenizer for adaptive image representation and generation.
Paper: arxiv.org/abs/2610.10497
Code: github.com/myc634/QuadTok
Details below.
ALT QuadTok overview. A dog image is reconstructed with 64 to 192 tokens, with orange overlays marking finer tokens. Below are generated images and a supplied quadtree used to guide the spatial layout of a generated bird image.
Long videos should remember what appeared, not when. 🎬
Introducing RECAP-Forcing, a training-free memory for long video generation. It stores each new subject, object and scene once, at first sight, and keeps it, so minute-long videos stay consistent and keep moving. 🧵
4/🧵
And it's a drop-in:
✅ No training, no new parameters
✅ Gains on Self-Forcing, Infinite-Forcing, LongLive and Helios (1.3B–14B)
✅ ~1.6× constant overhead, whatever the video length
✅ Up to 5-minute videos
n/🧵
📄 Paper: arxiv.org/abs/2608.26671
💻 Code: github.com/xXuHaiyang/RECAP-…
🌐 Project page: xxuhaiyang.github.io/RECAP-F…
🚀 Excited to share our new work - LIFT: Layout-In-Future Video Generation under Large Viewpoint Change via On-Policy Self-Distillation!
🌐 Project page: jsxzs.github.io/LIFT/
📄 Paper: huggingface.co/papers/2609.3…
LIFT enables controllable image-to-video generation with:
📷 Large viewpoint change
📦 Future-view layout control
🔄 On-policy self-distillation (OPSD)
Instead of requiring dense per-frame layouts, LIFT only needs last-frame object layouts to control what appears and where in the future view.
MLPC Group retweeted
🚀 Excited to share our #WACV2026 paper for 3D spatial reasoning:
CVP: Central-Peripheral Vision-Inspired Multimodal Model for Spatial Reasoning
Inspired by human vision, we introduce CVP, which combines:
👁️Target-affinity tokens (central vision) to focus on relevant objects
🌍Allocentric grids (peripheral vision) to capture global scene context
This simple idea significantly improves 3D spatial reasoning, achieving SOTA performance across multiple benchmarks.
📄Paper: arxiv.org/pdf/2512.08135
🌐Page: zeyuan-chen.com/cvp/
#spatialReasoning #MultimodalModel #VLM @LambdaAPI @UCSD @mlpcucsd @wacv_official
MLPC Group retweeted
📄 Excited to present our ICCV paper "YOLO-Count: Differentiable Object Counting for Text-to-Image Generation!"
🎯 We show a new regression objective that boosts both counting accuracy and controllability in T2I models.
📍 Poster #1545, Wed 2:45PM, Exhibit Hall I
#ICCV2025
Check out our #ICCV2025 paper Lay-Your-Scene!
We introduce a controllable and lightweight diffusion transformer architecture for natural scene layout generation and propose a comprehensive benchmark for evaluating layout quality.
Website: mlpc-ucsd.github.io/Lay-Your…
Paper: arxiv.org/abs/2505.04718v1
📍Poster Session: October 22, 2:45 PM, Exhibit Hall I
🚀 Introducing OverLayBench – A Benchmark for Layout-to-Image Generation with Dense Overlaps! [NeurIPS 2025]
📜 Paper: arxiv.org/abs/2509.19282
🌐 Project: mlpc-ucsd.github.io/OverLayB…
💻 Code: github.com/mlpc-ucsd/OverLay…
🔍 The Problem:
Current Layout-to-Image (L2I) models collapse under heavy bounding-box overlaps: objects blend together, distort, or misalign. Existing benchmarks are biased toward simple cases, leaving this critical challenge underexplored.
🤔 Why It Matters:
Dense overlaps are everywhere in the real world—from crowded streets 🚦 to busy dining tables 🍽️. Evaluating and improving L2I under such conditions is key for robust controllable generation.
💡 Our Solution:
✅ OverLayScore – a new metric capturing both spatial IoU and semantic similarity to quantify overlap complexity.
✅ OverLayBench – a benchmark with balanced difficulty splits (Simple / Regular / Complex), high-quality captions, instance-level grounding, and relationship annotations.
✅ CreatiLayout-AM – a new baseline using amodal mask supervision, significantly improving generation quality under occlusion (+15% O-mIoU).
Check out more results in the thread below. 👇 #NeurIPS2025
[7/n]
For full details, please check out:
📜 Paper: arxiv.org/abs/2509.19282
🌐 Project: mlpc-ucsd.github.io/OverLayB…
💻 Code: github.com/mlpc-ucsd/OverLay…
🤗 Dataset coming soon!
Happy to discuss & collab — feel free to reach out!
[n/n]
🎉 Congratulations to our students — Bingnan Li, Chen-Yu Wang, Haiyang Xu, Xiang Zhang, Ethan Armand, Divyansh Srivastava, Xiaojun Shan, Zeyuan Chen — on the successful release of OverLayBench! 📷 We also thank our collaborator from Lambda, Jianwen Xie, and our advisor Prof. Zhuowen Tu for their contributions to this journey.