🇪🇺 automating accounting. Prev. research @Aleph__Alpha
Berlin
Joined December 2016
- Tweets413
- Following304
- Followers321
- Likes1.3K
Johannes Messner retweeted
Laut dem Bundesrechnungshof fließen im Jahr 2040 36,7 Prozent der Steuern in die Rentenkasse. Durch das Rentenreformpaket könnten es demnach sogar fast 46 Prozent werden. trib.al/SE3f8JK
Johannes Messner retweeted
Strangely enough, one of founders' unique sources of strength is that they've experienced weakness. Hired CEOs tend to take the power of the companies they run for granted. But the founders remember when the company was so weak that it could only survive by delighting users.
Franz Kafka would have loved Gemini API I mean Google AI Studio I mean Vertex AI I mean Gemini Enterprise Agent Platform I mean Gemini Enterprise Agent Platform Inference I mean
Johannes Messner retweeted
prime-rl can now train 1T parameters MoE blazingly fast, under 5 minutes per step, or 1k steps in ~3 days
To achieve this we shipped in our latest prime-rl 0.6.0:
* inference: wide-ep, fp8 inference, llm-d router, mooncake, kv cache cpu offloading
* training: fsdp2, deep-ep expert parallelism, dsa cp, fp8 training, router replay
* agentic rollout: we rewrote the core of our rollout orchestrator for better scalability
support for glm5, kimi, nemotron, ...,
prime-rl is open source but also end to end optimized to run on our dedicated RL infra and compute layer
The last thing I worked on at Aleph Alpha!
With @SohirMaskey Constantin Eichenberg @douglasahorr
tl;dr:
- Quantisation-aware training works really well
- If you have a fixed memory budget you should probably go many parameters - few bits
- k-means quant. is better than uniform
Would you rather use 1 million × 16-bit weights, 4 million × 4-bit weights, or even 16 million × 1-bit weights?
In joint work between Aleph Alpha Research and Graphcore, we asked this question of LLMs — the answer encouraged us to embrace the wonder ✨ of 1-bit weights, which can outperform 4-bit and 16-bit weights on a fixed weight memory budget.
In our work
- ⚖️ A scaling laws evaluation prompts us to consider very low-bit formats
- 📈 Scaled-up tests show the power of memory-matched models with 1-bit weights
- ⚡ Kernel benchmarking demonstrates their feasibility for autoregressive inference
Read all about it in our blog and paper (link below! ⬇️)
Massive thanks to our collaborators at Aleph Alpha Research!
Authors: @SohirMaskey, Constantin Eichenberg, @atomicflndr and @douglasahorr
Johannes Messner retweeted
Today we’re releasing Trinity Large, a 400B MoE LLM with 13B active parameters, trained over 17T tokens
The base model is on par with GLM-4.5 Base, while being significantly faster at inference because it’s sparser and hybrid
The architecture we picked is one of my favorites: 3:1 local/global with SWA, NoPE on the global layers and RoPE on the local layers, gated attention, depth-scaled sandwich norm, and smooth training with Muon.
Our dataset is also high quality, curated by @datologyai .
We trained it on 2,000 B300s for a month on @PrimeIntellect infrastructure.
This is a preview release with an instruct model only — we’re ramping up RL on it.
When @latkins approached us a couple of months ago to train this model together, I thought he was crazy — but then he hired @stochasticchasm, and here we are.
We're excited to introduce @arcee_ai's Trinity Large model.
An open 400B parameter Mixture of Experts model, delivering frontier-level performance with only 13B active parameters.
Trained in collaboration between Arcee, Datology and Prime Intellect.
Johannes Messner retweeted
Curious how to accelerate inference of some of the recent byte level models like HAT/HNet/BLT?
Check out this vllm fork developed by my friends and colleagues, Pablo and Lukas!
To my knowledge first demonstration of inference speedups from dynamic chunking in byte models!
First high-performance inference for hierarchical byte models.
@LukasBluebaum and I developed batched inference for tokenizer-free HAT (Hierarchical Autoregressive Transformers) models, developed by @Aleph__Alpha Research. In some settings, we outcompete the baseline Llama.🧵
Johannes Messner retweeted
First high-performance inference for hierarchical byte models.
@LukasBluebaum and I developed batched inference for tokenizer-free HAT (Hierarchical Autoregressive Transformers) models, developed by @Aleph__Alpha Research. In some settings, we outcompete the baseline Llama.🧵
Johannes Messner retweeted
Our work on tokenizer free LLMs: Hierarchical Autoregressive Transformers (HAT)!
We recently dropped HAT models on HF, pretrained from scratch! huggingface.co/collections/A…
You can try them with both HF Inference AND our vllm fork: github.com/Aleph-Alpha/vllm
🧵
(1/6)
Johannes Messner retweeted
And imo what's the coolest is that we made it ready for production grade inference with our own vllm fork (more details on this soon!): github.com/Aleph-Alpha/vllm
So now you can now enjoy all the vllm features like continuous batching, paged attention etc also for HAT!
(5/6)
Seeing this pushback a lot - and it‘s fair!
However, these models don’t have a fixed vocabulary, i.e. there are infinitely many words the model can operate over instead of a finite set of tokens.
The real issue, as I see it, is that the word splitting rule is not learned, unlike BLT and HNet.
This is WIP.