Founder & CEO @Tracel_AI Building Burn, an open-source Deep Learning Framework in Rust 🦀
Québec, Canada
Joined January 2012
- Tweets561
- Following734
- Followers568
- Likes2.1K
Not true anymore, not since RL post training. Models are trained to maximize rewards, which can be anything
Ben Affleck says AI is simply matrix multiplication, linear algebra and floating point calculations, and however convincingly it sounds like a person, it has nothing to do with thought, humanity or feeling
“Nor do I believe, and I think it’s important to draw the distinction between what AI does, which is simply predict the next token or the next word in a sentence, then add that word on, run it back to the machine, do a lot of computation and pick the next word."
"That is a very different thing than sentience or thought. And I think we need to be really clear about that because it can be convincingly, in an illusory way, sound like thought or like a person."
"But let’s be real clear. This is matrix multiplication. This is linear algebra. These are floating point calculations. It has nothing to do with thought, humanity, or feeling.”
Btw batch decode is compute bound, and keeping the KV cache on the same compute unit is valuable. The benefit of a different chip for decode is only latency, but it comes with a higher per token cost.
AI inference is shifting from single GPU clusters to Heterogeneous Disaggregated Inference. Prefill (compute-bound) and Decode (memory-bandwidth-bound) have completely opposite hardware needs. Instead of forcing H100s/B200s to do both (poorly), the industry is splitting the pipeline across specialized chips.
There will be two kind of AI products. The closed AI that is fully vertical, from the application layer to the chips.
Then there will be open source AI that will provide more flexibility of choice to customers. They will be able to chose the software and hardware infrastructure, and the applications running on top.
Data brokers are now buying up SaaS data sets for anywhere from $100K to $2M
SaaS are struggling now and some extra money to fill the gap by declining revenues is nice
That data is then re-sold to AI training companies and AI frontier labs for the exact purpose of integrating whatever that SaaS did into the AI frontier lab's app
Pretty obvious to see where this is going
Vim was about automating typing to reduce cognitive load and focus more on the problem. It was worth it and I'm happy I learned it as well
Commoditise your complement: if AI models are free, it's easier to sell the hardware to run them, and you get better margins
Graph capture works on wgpu now, and Qwen3 0.6B decodes 25% faster with it on my Strix Halo. You capture one decode step once, and you just replay it.
We just added compute environments to CubeCL, and it removes the JIT cold start. You load one, and it starts as fast as if it was AOT
True, I think we could support some NPUs through LLVM, but a lot of them just accept ONNX graphs, which is terrible
Nathaniel Simard retweeted
Burn v0.22.0 is out
> This release removes backend generics from models/tensors (device selects runtime), adds LoRA/QLoRA, multi-optimizers, graph capture, and faster rebuilds.
ALT Image: Burn v0.22.0 is out > This release removes backend generics from models/tensors (device selects runtime), adds LoRA/QLoRA, multi-optimizers, graph capture, and faster rebuilds.
Nathaniel Simard retweeted
Burn 0.22 just shipped. The deep learning framework written in Rust got significantly faster to build, train, and deploy. 🦀
The headline change: backend generics are gone from user-facing code.
Before:
pub struct Model<B: Backend> { linear: Linear<B> }
After:
pub struct Model { linear: Linear }
The backend is now selected through the device, not wired through every type. This breaks the compilation dependency chain that previously propagated through the entire application. Real measured rebuild times:
→ CNN project: 28.42s → 4.57s (6.2x faster)
→ Transformer project: 14.73s → 1.00s (14.7x faster)
That's not a microbenchmark, those are real model-edit rebuilds on release builds.
Training performance on CUDA also improved significantly:
→ CNN: 80% higher throughput, 49% less peak VRAM (956 MiB → 486 MiB)
→ Transformer: 3.6% faster, 18% less VRAM
The VRAM reduction comes from adaptive memory pools that tune allocation sizes based on workload statistics, no manual configuration needed.
What else is in 0.22:
LoRA and QLoRA fine-tuning : attach low-rank adapters to frozen base weights, with per-parameter-group optimizer config. QLoRA keeps base weights quantized to reduce storage.
ONNX export : export a Burn model to ONNX with one call. Early stage, limited operator coverage, but the direction is there.
Remote compute via Iroh : run inference and training on another machine's GPU. Peer-to-peer, encrypted, authenticated. Repeated graph sequences are sent once and replayed by ID.
CubeCL Environment : ship pre-warmed compilation and autotuning caches with your application. Works across native, WASM, and no_std targets.
Graph capture and replay on CUDA and HIP : reuse kernel launch sequences without repeating launch preparation.
NDArray and LibTorch backends are deprecated. Candle is removed. Requires Rust 1.95+.
The team also hints at a native AI engine (LLM inference server + chat app + OpenAI-compatible API, all in Rust) shipping soon on top of Burn. Worth watching.
🔗 tracel.ai/blog/release-0.22.…
#RustLang #MachineLearning #DeepLearning #OpenSource #GPU
