Mark (@_M_Weber) profile picture

Researcher at FAR, Amazon. Prev. PhD @ TU Munich, ByteDance Seed, Google; Perception, Tokenization, VLMs, Humanoids

Joined November 2020
Exciting news! We're happy to announce our challenge / workshop at this year @ICCVConference focusing on Spatiotemporal Action Grounding in Videos. Here are the details: 🔷 Watch the video below for a demo. 🔷 The eval server is open until 09/19! 🔷 Links incl. code below. #ICCV
2
2
1
13
1,777
➡️ For workshop specifics, including the evaluation package, baseline code, and more, visit: motchallenge.net/workshops/b… ➡️ Eval server (sign-up now!): codabench.org/competitions/9… #ICCV2025
1
320
🎯 Challenge Launch Announcement We are pleased to announce the launch of the MOT25 Challenge, to be held in conjunction with ICCV 2025. 🔗 Workshop website: motchallenge.net/workshops/b… 🧪 The MOT25 Challenge is now live on Codabench: codabench.org/competitions/9…
1
1
1
166
A beautiful article by D. Graham Burnett newyorker.com/culture/the-we… ´The A.I. is huge. A tsunami. But it’s not me. It can’t touch my me-ness. It doesn’t know what it is to be human, to be me.’ ‘Historians have long extolled the “power of the archive.” Little did we know that the engineers would come along and plug it in. And it turns out that a huge amount of what we seek from a human person can be simulated through this Frankensteinian reanimation of our collective dead letters.’ Thanks for sharing @mustafasuleyman
1
4
1
19
4,908
Heading off to ICLR to present our work on image generation and latent spaces! If you're interested in tokenization or generation drop by at our poster. Also, if you'd like to chat about any of these topics, feel free to ping me! #ICLR2025
🧵1/9 Happy to share our paper "MaskBit: Embedding-free Image Generation via Bit Tokens" got published in TMLR with featured (aka spotlight) & reproducibility certifications! I'm especially excited about the disentangled visual concepts in our shared latent space. Details below!
1
13
672
Our exclusive with @JDVance ahead of @MunSecConf: -On Ukraine, he says there will be a good peace deal that will guarantee the country’s long-term sovereignty - and Putin will face sanctions and military measures if he doesn’t play ball. -On Europe, he will tell mainstream leaders in Munich that they’ve become Soviet-style enemies of free speech and democracy who ignore voters and fail to stop mass migration. Some Germans will be particularly shocked when he calls for ending the firewall against the far-Right @AfD and embracing the populist vote. With @alexbward via @WSJ wsj.com/world/europe/vance-w…
54
155
36
355
154,916
Everything you love about generative models — now powered by real physics! Announcing the Genesis project — after a 24-month large-scale research collaboration involving over 20 research labs — a generative physics engine able to generate 4D dynamical worlds powered by a physics simulation platform designed for general-purpose robotics and physical AI applications. Genesis's physics engine is developed in pure Python, while being 10-80x faster than existing GPU-accelerated stacks like Isaac Gym and MJX. It delivers a simulation speed ~430,000 faster than in real-time, and takes only 26 seconds to train a robotic locomotion policy transferrable to the real world on a single RTX4090 (see tutorial: genesis-world.readthedocs.io…). The Genesis physics engine and simulation platform is fully open source at github.com/Genesis-Embodied-…. We'll gradually roll out access to our generative framework in the near future. Genesis implements a unified simulation framework all from scratch, integrating a wide spectrum of state-of-the-art physics solvers, allowing simulation of the whole physical world in a virtual realm with the highest realism. We aim to build a universal data engine that leverages an upper-level generative framework to autonomously create physical worlds, together with various modes of data, including environments, camera motions, robotic task proposals, reward functions, robot policies, character motions, fully interactive 3D scenes, open-world articulated assets, and more, aiming towards fully automated data generation for robotics, physical AI and other applications. Open Source Code: github.com/Genesis-Embodied-… Project webpage: genesis-embodied-ai.github.i… Documentation: genesis-world.readthedocs.io… 1/n
553
2,944
1,607
15,774
3,824,699
9/9 If you like this research and are hiring, I will be on the job market next summer! Big thanks to my collaborators: @yucornetto1 @xueqingdeng77 @lcchen_jay @tumcvg @LijunYu0
2
281
5/9 Our analysis reveals fascinating properties of bit tokens: Most channels appear to capture different visual concepts, making the representation more interpretable! Flipping individual bits leads to systematic changes in attributes like texture, color, and style. 🎨
1
144
7/9 MaskBit achieves state-of-the-art performance with up to 1.52 FID on ImageNet 256×256, using just 305M parameters. That's better than prior diffusion and autoregressive models! 🔥
1
131
6/9 For our generation model MaskBit, the key innovation is: We are the first to utilise the SAME (bit) token representation for both tokenizer and generator, unlike prior methods that require separate embedding tables! No need to (re-)learn codebooks anymore!
1
126
3/9 We carefully revisit the VQGAN design and provide a complete, reproducible recipe for building a modern tokenizer. Our VQGAN+ improves reconstruction FID from 7.94 to 1.66 in the low vocabulary, low resolution setting - a huge 6.28 gain! 📈 See chapter 2 for details.
1
159
4/9 Our tokenizer learns a semantically structured latent space! Checkout what happens when we flip bits in each channel. Having a consist visual interpretable latent space could be a gamechanger for control!
1
1
163
2/9 We introduce three main contributions: 1️⃣ A systematic study modernizing VQGAN (VQGAN+). 2⃣ A shared latent space generator and tokenizer based on bit tokens between with disentagled visual concepts. 3⃣ A new embedding-free image generation approach based solely bit tokens.
1
203
🧵1/9 Happy to share our paper "MaskBit: Embedding-free Image Generation via Bit Tokens" got published in TMLR with featured (aka spotlight) & reproducibility certifications! I'm especially excited about the disentangled visual concepts in our shared latent space. Details below!
1
4
1
61
4,225
I'm happy to present our work "An Image is Worth 32 Tokens for Reconstruction and Generation" at #NeurIPS2024. Would you like to learn more about efficient image tokenization and generation? Then join us on Friday 11am-2pm and save the poster #1604 to your schedule!
5
9
2
71
4,922
I'm happy to share our latest work "MaskBit: Embedding-free Image Generation via Bit Tokens" which tackles class-conditional image generation with sota results! ➡️Project page: weber-mark.github.io/project… ➡️preprint: arxiv.org/pdf/2409.16211
7
16
2
105
9,601
If you're attending #ECCV2024 and would like to chat about generative models or tokenization, shoot me a message! ➡️ weber-mark.github.io/project… This has been my internship work with @LijunYu0 @yucornetto1 @xueqingdeng77 @tumcvg Xiaohui Shen and Liang-Chieh Chen.
1
319
If you want to learn even more about efficient image tokenization, also checkout our work "An Image is Worth 32 Tokens for Reconstruction and Generation"! ➡️ Project page: yucornetto.github.io/project…
2
225