Solving the hardest problems I can get my hands on 🚀 Co-Founder & CTO @withmantle Formerly: Pivotal, VMware, Xtreme Labs
Toronto, Ontario
Joined March 2010
- Tweets7.1K
- Following578
- Followers1.1K
- Likes5.6K
Me: claude, make the tests faster.
Claude: done.
Me: why do they seem slower?
Claude: my comparison script had a bug: it read "152ms" as 152 seconds. 👀
Just when you think you can put down your benchmark harness…
I did say we'd be releasing our models periodically...introducing Gemini 4 Carbon, our next frontier model.
Following Argon's release just two weeks ago, Carbon shows the pace at which we’re pushing the frontier, it ranks #1 on the recently updated v5.0 of @ArtificialAnlys Intelligence Index at 59. It also leads across key cybersecurity evaluations scoring 88.2% on SEC-bench Pro for finding vulnerabilities, 54.8% on ExploitGym for exploitation, and 71.0% on CWE-bench for patching.
We’re seeing significant improvements in long-horizon tasks even compared to Argon, and we’ve strengthened frontier safeguards.
As we did with Argon, we’re going to roll it out gradually + responsibly. Some trusted cyber defenders will get access starting this week. We'll be expanding it to more partners through our Fairwind Program, ahead of wider availability soon, so hang tight!
I added Cloudflare’s Claric to our AWE benchmark, and it finished near the bottom, which is what I expected and not really the point.
Falls short:
- four sections (called by year, rows, highlights and sourcing channels) came in at 0%
- table-heavy financial statements sat at 69%
The spread is the interesting part. A fast single-pass model that nails some sections and misses others could work as a first pass, or as a gate that routes the hard sections to a reasoning model. See the full results at research.withmantle.com/awe
Looks like @GoogleCloud has quietly beta released SSH access to Cloud Run containers. Much appreciated for those quick debugging sessions. 👏 docs.cloud.google.com/run/do…
yes plz 🤩
A single command to install @omarchy on your iPad
Coming soon
Video is sped up, total install took 3.8 minutes
@OmarchyMac @OmarchyUpdates @dhh @OmarchyLinux @tcballard
Amazing framework to share. Hoping to have the hardware at some point to test it out.
🚀 Introducing DeepInfer, our in-house inference framework, now fully open source!
Powering DeepSeek-V4.1-Flash at scale with high throughput and low latency. We’re sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable.
🔗 github.com/deepseek-ai/DeepI…
1/4
Dwayne Forde retweeted
Claude, meet your cap table 🤝
blog.withmantle.com/mantle-e…
If you can do this in Toronto (we have a lot of edge cases), you can do this anywhere 👏🫡
We’re bringing the magic of Waymo to Toronto! 🇨🇦
In the coming days, you’ll begin seeing Waymo’s all-electric vehicles driving through Toronto’s streets as we lay the groundwork for our ride-hailing service.
Our goal is to begin serving riders in 2028.
Sign up for future updates: waymo.com/waymo-in-canada
The part I liked most in the @claudeai speedup write up isn't the 3x, it's the loop: deterministic benchmarks, ship behind a flag, ratchet the benchmark down, repeat. claude.dev/blog/how-we-made-…
Really curious to try this out on some local hardware. github.com/Niko1221/Strata
Gemini 4 Argon is new on the board and comes in sixth, 1.2 points behind the leader @claudeai Opus 5.5. It is a clear step up from @geminiapp 3.1 Pro Preview, which scored 92.2% but not enough for a podium finish.
On AWE-bench, @WithMantle's research benchmark for extracting names, dates and numbers from wealth and equity documents. At the top of the board, the frontier models are bunched so tightly that the gap between them matters less than what they cost and how fast they run.
If you've compared frontier models on financial reasoning, what made you choose one over another? Would love to hear thoughts. You can see the full results on the AWE-bench website. research.withmantle.com/awe