10/ 🔎 How does an agent arrive at its next move?
Our replay UI brings observations, reasoning, actions, rewards, and memory into one view.
Step through a trajectory to inspect decisions and understand where an agent succeeds or gets stuck.
ALT GlyphBench replay UI showing a Miniatari BankHeist trajectory with the glyph observation, task, reasoning, chosen action, and environment feedback.
11/ Led by @creus_roger, in collaboration with @Cote_Marc, @GlenBerseth and @pcastr at @Mila_Quebec & @UMontreal.
📄 Paper: arxiv.org/abs/2609.34214
💻 Code: github.com/VmaxAI/glyphbench
🌐 Blog: vmax.ai/team/glyphbench
ALT Paper title, six authors and their Mila, Université de Montréal, and Vmax affiliations, followed by the abstract.
Vmax retweeted
We built cybersecurity monitors for Kimi K3 and GLM 5.3 — more accurate, 50x faster, & 50x cheaper than an optimized LLM judge.
External red-teaming from @farairesearch found that our monitor greatly reduces universal jailbreaks. 🧵
Vmax retweeted
Come join us next week at our CoLM workshop:
Context Beyond the Window
We have some great speakers coming to speak on the opportunities, limitations and safety-considerations of current LLMs.
We will also be cohosting a social alongside Reflection!
luma.com/eocj5cm5
Vmax retweeted
Want to train your SWE agents 2× faster 🏎️💨?
We introduce KV-streams, which, rather than flushing the KV cache on agentic compaction, we preserve it and keep generating. This avoids the re-prefill cost on both the inference engine and the trainer.
Unlike other works that assume a fixed compaction strategy, we show that KV-streams can work with any compaction method in agentic tasks.
Using what we learn from experiments in text-based games. We scale KV-streams to SWE tasks, where we match the performance of re-prefill compaction and full context in half the time 🚀
arxiv.org/pdf/2609.35750
🧵
Nethack is a really underrated benchmark
astra playing NetHack:
- score: 48,978
- BALROG progress: 49.4%
- max dungeon depth: 17
- max experience level: 14
important note: this is a cherry-picked episode (best of 10), and Astra has been able to modify its own game-playing harness
no privileged information or unfair game modifications, but it uses tools such as pathfinding to previously visited landmarks and code-written trackers for previously observed entities. these offload a fair amount of low-level reasoning, while Astra remains responsible for choosing what to do at every step
imo, the ability of LLMs to autonomously create tools that amortize low-level control costs (and then interpret and use their outputs in-context) is remarkable, and one of the reasons i'm excited about LLM agents. like, we could run the same tool computations and... concatenate the outputs into a flat vector for an MLP... but it would still have to learn what those outputs mean and how to use them. with LLMs we get free hierarchies and natural separation of responsibilities...
anyway, i'm now running episodes with the default settings from BALROG for a more fair comparison
the auto-research programme continues. will report back!
full run below ⤵️
Vmax retweeted
Interesting details from @deepseek_ai on the benefits of scaling environment/task synthesis.
If you want to scale open-ended environment generation for your own use case, sign up to the @VmaxAI waitlist - link below
Vmax retweeted
Greatest product placement ever. Placing the Vmax name in the latest Spiderman movie. 👏
Congrats @VmaxAI
Vmax retweeted
Legendary discussions at the office. @rronak_ discussed applying and scaling up SDPO for continual learning in the real world to prevent context degradation and more
@kevingu gave a talk on learning from production traces for knowledge-work tasks that have no direct verifier: how to generate reward signal without ground truth and keep organizational context current as the work changes
@matthewjsargent explained how to stabilize asymmetric self play and new research fields in open ended learning
Thanks to all those who came out and asked great questions we’ll be hosting more of these soon!
Most agents are static once deployed. Trained once, frozen, and brittle outside their training distribution.
We’re bringing together researchers and builders working on systems that keep learning, decompose problems into reusable pieces, and improve their own reasoning over time. Talks are from @kevingu CTO of Thirdlayer, @rronak_ CEO of Trajectory, @matthewjsargent CEO of Vmax followed by open Q&A.
Join us! luma.com/l666hxiw
You should try Silico. Not only for mech interp, but for AI Research in general.
Goodfire have put a lot work into building a great interface for long-horizon agent-driven research.
