I like to shrink dem models 🤏 ML/AI Perf @dropbox Prev. Co-Founder & Principal Scientist @mobius_labs (acquired by @dropbox) PhD @inria
Berlin, Germany
Joined November 2023
- Tweets656
- Following123
- Followers733
- Likes1.2K
mobicham retweeted
Today, we’re announcing Ternary Bonsai 2 27B.
Based on Qwen3.8 27B, Bonsai 2 27B is 9x smaller than its full-precision counterpart while retaining 98.2% of its aggregate benchmark performance.
Two months after the first Bonsai 27B release, the biggest change is quality. The footprint remains 5.9 GB, but the gap to full precision has narrowed materially, with particularly strong gains in agentic coding, multimodal reasoning, and long-horizon tool use.
Ternary Bonsai 2 27B is available today under Apache 2.0.
mobicham retweeted
Not all work happens in writing. Teams that work with photos, videos, and audio need AI that can search within multimedia content too.
Hear more from Dropbox machine learning engineer Appu Shaji—along with machine learning engineer Hicham Badri—on Working Smarter.
mobicham retweeted
If you’re at @EurIPSConf this week and looking for your next ML role, ping us. Few of us from @Dropbox including @mobicham @appughar will be there Happy to chat about roles in multimodal, search, and video-AI.
Looking for people with experience in : Building and scaling multimodal ML systems, Retrieval, ranking, and creative search problems, Turning prototypes into polished products
🌍 Remote (Germany or Denmark)
Apply here: jobs.dropbox.com/listing/738… (Please use the link, not DMs!)
mobicham retweeted
@Dropbox is acquiring our IP, and the Mobius core tech team is joining them. A huge milestone for us!
dropbox.tech/machine-learnin…
mobicham retweeted
New blogpost: Use NVRTC to explore MMA instruction variants, including FP16 accumulate and INT4.
gau-nernst.github.io/nvrtc-m…
Thanks @marksaroufim for bringing NVRTC to PyTorch, and @mobicham for (indirectly) giving me the idea.
mobicham retweeted
Excited to release tilelang v0.1.0, another pythonic dsl for writing AI kernels with optional layout/pipeline annotations, and optional thread-level programming interface. If these features sound useful, please check it out and give a try :) github.com/tile-ai/tilelang
mobicham retweeted
We’ve improved DeepSeek R1 distilled models using logits distillation—delivering +4-14% gains on GSM8K while only spending $3-18 per training run! 🚀
Now available on Hugging Face-e—run them efficiently with HQQ! 🤗
🔗More details at: mobiusml.github.io/r1_redist…
mobicham retweeted
We are releasing more re-distilled R1 models. Now including a Qwen-7B with a GSM8K score of 83.4%, +4.55% higher than that of the original distilled model.
7B: huggingface.co/mobiuslabsgmb…
1.5B: huggingface.co/mobiuslabsgmb…
mobicham retweeted
Releasing a new version of Gemlite github.com/mobiusml/gemlite significantly improved performance on datacenter GPUS (A100/H100) delivering up to 7–8x faster prefill and 3–6x faster decoding compared to PyTorch's tinygemm.
mobicham retweeted
Really happy to contribute to the batched version of faster-whisper that is 4x faster and more accurate 🚀🚀🚀 github.com/SYSTRAN/faster-wh…
mobicham retweeted
Speed up FLUX CPU offloading by 200%. On 4070Ti SUPER (16GB)
baseline (.enable_sequential_cpu_offload()): 3.72 s/it
+ pin memory: 2.09 s/it (+78%)
+ CUDA stream (explicit synchronization): 1.32 s/it (+182%)
+ record_stream: 1.23 s/it (+202%)
gist.github.com/gau-nernst/9…
mobicham retweeted
Check out our own @mobicham's lecture on building low-bit Triton kernels & the impressive speedups we’re achieving with them, presented at @GPU_MODE channel. Also includes plenty of tips & tricks for performance tuning in Triton & GPU programming.
youtube.com/watch?v=7c3c3bCG…
mobicham retweeted
We’re thrilled to announce that our team at Mobius Labs has won Runner-Up in the SoccerNet Game State Reconstruction Challenge, in collaboration with @IRI_robotics 🏆⚽
mobicham retweeted
Very happy to see HQQ integrated into TorchAO. Having a great time collaborating with the TorchAO team.
Here’s a topline metric for Llama 3, where we contributed:
"97% speedup for Llama 3 8B using autoquant with int4 weight only quantization and hqq'
We’re happy to officially launch torchao, a PyTorch native library that makes models faster and smaller by leveraging low bit dtypes, quantization & sparsity. Our techniques are written in easy-to-read PyTorch code spanning both inference & training: hubs.la/Q02RnjwJ0
mobicham retweeted
It was amazing to meet so many incredibly talented CUDA hackers in real life. My brain still needs to recalibrate to what is possible here in the Bay Area 🇺🇸💙.
CUDA MODE hackathon today!
Here's @karpathy on the 🏖️ origin story of llm.c, and what it hints at for the fast, simple, llm-compiled future of custom software.
mobicham retweeted
Thanks! TVM is fantastic. We are also working on a TVM based lang similar to Triton, which supports block-level programming and providing a thread-level interface. You can check out some example use cases, like FlashAtten and Dequantize GEMM, at github.com/microsoft/BitBLAS…
6/ I also tested BitBlas which is a library that supports mixed precision matrix multiplication. It's based on TVM (Tensor Virtual Machine) which is a deep learning compiler that compiles tensor expressions into efficient code that can be run on different backends. The performance is truly impressive !
kudos to @Lei_Wang_1999 and team @Microsoft for this amazing project !
BitBlas repo : github.com/microsoft/BitBLAS
and here is a simple BitBlas Linear layer to test it : gist.github.com/MekkCyber/a7…
mobicham retweeted
NVIDIA doesn't want you to know this one trick: train ML models with INT8 Tensor Cores🤯
Up to 70% end2end speedup on 1x 4090 and 40% speedup on 1x A100 with 4 lines of code. No noticeable accuracy loss.
More details here: github.com/pytorch/ao/tree/v…
github.com/pytorch/ao/pull/7…
mobicham retweeted
HQQ is now integrated into pytorchao 🤘
github.com/pytorch/ao/releas…
