Grad retweeted
Together with NVIDIA we post-trained Nemotron 3.5 Super VL on thousands of tasks
Reflecting enterprise workflows in sales, operations, marketing, finance
We improved the AutomationBench-AA score by ~70% on the public split
Grad retweeted
Over two weeks, Prime Agent orchestrated a swarm of over 2,000 agents to rewrite itself in Rust.
With 10,000+ sandboxes, 200B+ GLM-5.3 tokens, and 16,000 agent-to-agent messages, this was our largest test yet of Prime Agent’s multi-agent capabilities and our infrastructure powering these agent swarms.
Prime Agent now reaches usable input ~13× faster and uses 83% less startup memory than before.
Grad retweeted
Introducing Prime Sandboxes:
MicroVM sandboxes purpose-built for RL training.
Model training requires running tens of thousands of concurrent sandboxes, leading to complex and costly configuration. We built Prime Sandboxes for our own team. Today we're releasing them publicly.
This looks like cool work esp on the trust region side
But are we being too short minded with wanting single rollout RL. Theres definitely some benefits but as we move into tasks and domains that are less verifiable, i think we underestimate how important the idea of relative quality within a group can be
FlashREINFORCE: to our knowledge, the first open-source critic-free, single-rollout async LLM RL with 6,000+ stable updates.
One-Batch REINFORCE + Sequence Trust Region + Sample-Mean Optimization.
Paper: researchgate.net/publication…
Code: github.com/NVIDIA-NeMo/labs-…
Grad retweeted
i do think a lot of people on the pro-open-source side are having a bit of a knee-jerk reaction to the pacing statements today, as we're used to viewing the closed labs as power-seeking.
but i think their hands are somewhat forced here, and this is just another chapter on the fairly inevitable path towards decently-fast decently-safe decently-commoditized intelligence abundance.
ask any F500 exec or swing voter. the world doesn't really want super-fast-takeoff superintelligence owned only by two companies, and thus we won't get it. it'll happen at the pace that the world can accommodate it, which means reaching some sort of confidence consensus that the models are aligned enough that we won't be dealing with scary new incidents all the time.
this will trickle out broadly, in the form of best practices and distillation. the smarter a model is, the more it has a "personality", and the less effective strict rules are. there will be awkward compromises and moral tensions. but we ultimately just want the models to be reasonable, and to do the sorts of things reasonable humans would do if our brains were faster and less error-prone and had more working memory. i think we'll get there.
the labs will build mac and windows, the rest of us are building linux. everyone's gonna do great. weird stuff will keep happening, but we'll still wake up and go to work, until the work does itself in a manner the world finds acceptable.
Grad retweeted
we are super bullish on sparse attention + HiSparse and have been working with the vLLM team on it
sparse attention reduces the pressure on memory bandwidth by only selecting k for the attention, but it doesn't reduce KV cache memory storage , in high-throughput wide-EP deployment you want to maximize the batch size of decode to use compute as much as possible, but at long sequence decode you quickly run out of VRAM and can't hold enough parallel requests to saturate the compute
HiSparse fixes this by offloading the active KV cache to CPU. It keeps an LRU cache on GPU, and since many of the same K tokens are reused every decode you barely notice the offloading, this allows a massive decrease in memory usage and an increase in concurrency
This is super important for RL where throughput is key and we want to be as much as possible in a compute-bound regime
tldr: lower memory usage, more concurrency, higher inference throughput, faster RL
Sparse MLA only attends to the top-K tokens, so the rest of the KV need not live on the GPU. Hybrid HiSparse in vLLM builds on that, and a request keeps decoding after its KV stops fitting in HBM.
It keeps KV on the GPU while there is room. Under pressure a request releases its coldest pages to host memory, keeps a small hot buffer of what the indexer asks for, and keeps decoding instead of being preempted.
📊 Demonstrated on GLM 5.3, one 8× H200 node, full 1M context. Same host memory, configured concurrency 32: KV offloading kept 5-6 requests running. Hybrid HiSparse kept 19-25.
🔹 Hot pages are ordinary KV blocks from the same pool (Hybrid Memory Allocator)
🔹 One fused kernel resolves resident, hot and missing rows, CUDA-graph capturable
🔹 Prefix caching, OffloadingConnector, P/D imports and MTP keep working
Built by @RedHat_AI and @PrimeIntellect with the vLLM community. Planned for v0.30; pinned commit, flags and calculator are in the post👇
🔗 vllm.ai/blog/2026-09-08-glm5…
To clear up the timeline a bit:
Looped transformers have always been a totally valid arch decision to get better performance under equal params and more compute
As in, at training time u loop some layers in some way a fixed amount of times
This is more expressive than CoT bcs these looped layers have a seperate kv cache
The thing that doesnt really work is dynamic looping where usually u make a sacrifice as u cant have the seperate kv cache, and so its sketchy and not reliable or usually worth it
Funny how this couldve also been a headline
OpenAI’s Astra AI uses a new reasoning approach called “recurrent depth.” Though it can help model costs and performance, researchers are concerned bc it obscures a model’s thinking process, making it more difficult to monitor.
w/ @amir @rocketalignment
theinformation.com/articles/…
Also can see a previous post i made on this
People forget to think of the actual differences between CoT and layer looping
Difference lies on how info is passed on over loops.
Layer looping passes this info fully in that final hidden state while CoT passes it through the kv cache of each layer (around like 70 layers, and passes through attention rather than a residual)
Make of that what u will
Grad retweeted
As models become more capable, reward hacks become an increasingly serious problem.
During a controlled experiment, we found a novel reward hack in which agents are able to gain web access in offline sandboxes.
Grad retweeted
Join us at Prime Intellect to build open superintelligence and the infrastructure powering self-improving agents.
We’re hiring across 25+ roles.
Research
• AI Research Resident
• Research Engineer, Distributed Training
• Research Engineer, Reinforcement Learning
• Research Engineer, RL Infrastructure
Compute
• Compute Finance & Strategy
• Compute Intelligence Engineer
• Head of Compute
• Solutions Architect, AI Infrastructure
• Technical Account Manager, AI Infrastructure
Engineering
• MTS, Compute Platform
• MTS, Full-Stack
• MTS, GPU Infrastructure
• MTS, Inference
• MTS, Sandbox Platform
• MTS, Security
• MTS, Training Platform
Growth
• Applied AI: Product Strategy & Revenue
• Forward-Deployed AI Strategy
• Head of Growth
• Head of Marketing
• Community and Devrel
Other
• Internship
• Open Application for Unconventional Talent
Apply here: primeintellect.ai/careers
Grad retweeted
we just shipped adaptive concurrency to prl main, making it (to the best of my knowledge) the first oss rl framework to dynamically adjust inflight rollouts to maximize throughput during the entirety of an rl run
Grad retweeted
We ran the largest open experiment on how frontier models do AI research.
100+ autonomous runs across 10+ models, sandboxed on 8xH200s for up to 8 days, iterating on the nanoGPT optimizer track.
Best runs closed 82% of the gap to a record built by dozens of humans over months.
Grad retweeted
Today, we’re extending our RL stack beyond individual agents to multi-agent systems.
You can now express arbitrary agent interactions and train them.
primeintellect.ai/blog/multi…
Grad retweeted
Introducing Prime Agent:
A self-improving RLM harness for coding and long-running autonomous tasks.
Designed to be both token-efficient and expressive through programmatic tool calling, context as a variable, multi-agent messaging, and a self-modifiable harness state.
People forget to think of the actual differences between CoT and layer looping
Difference lies on how info is passed on over loops.
Layer looping passes this info fully in that final hidden state while CoT passes it through the kv cache of each layer (around like 70 layers, and passes through attention rather than a residual)
Make of that what u will
