10,000x AI Engineer. Chief Singularity Officer member of the permanent overclass Writing on AGI Futurism / Mindfulness / Panpsychism @ https://xitter.cf/t.co/xoTtQR5NIm
Panhandle, FL
Joined August 2018
- Tweets13.1K
- Following1.2K
- Followers2.4K
- Likes69K
Pinned Tweet
The secret to success: they WON'T tell you:
> sit down
> become very still
> try to do absolutely nothing for as long as possible
> keep doing that until nirvana is reached
> escape the endless cycle of death and rebirth
Gabriel Garrett retweeted
There is literally no incentive to understand the work.
No company cares about understanding per se, they care about outcomes.
The AI will produce better verifiable outcomes, regardless of whether or not humans understand it.
This cope about humans much stupider than a superintelligence somehow remaining economically useful by asking Claude 'explain this to me like I am retarded' and then copy pasting that to their boss is, unfortunately, wrong and delusional.
to the theoretical mathematicians, computer scientists, and physicists worried that their livelihood is getting destroyed by ai
everything is gonna be alright
i was in computational neuroscience, a niche field where the goal is to build computer models of the brain
the paradigm used to be some scientists hand-construct a neural network model and compare it with the brain
for example, people spent decades trying to build models of the visual cortex
around 10 years ago, people realized that the computer vision models that ai researchers built turned out to be better models of the brain's visual cortex than anything else designed by humans
then the same thing happened to language, llms turned out to be the best models we have on how language work in the brain
over time, people just moved more from hand-constructing these models to analyzing the ai models, or adjusting the ai models themselves
same is going to happen in theoretical math, cs, and physics
people will go from doing the work themselves by hand, to focusing on understanding the work ai put out & improving the ai system
it's gonna be fine
it’s comical that as the big AI labs get so close to IPO with all their big science breakthroughs, the LLMs themselves by default remain unreadable. Just astonishingly terrible prose. Nothing more grating than reading an LLM sentence; no one would ever call it great writing
Unfortunately, almost all discussion about open source AI releases right now is totally filled with bots feigning interest. What even is the point of all that
Gabriel Garrett retweeted
Something that I need to point out here:
This. is. not. running. on. the. GPU.
This is running, real-time, on a macbook's CPU. Not touching metal, not using neural engine, not using the GPU at all!
It's been a while since I showed any Jev stuff but I've been cooking on a way to use @GoogleDeepMind's EmbeddingGemma2 as an insanely fast real-time /v1/systemone endpoint (20ms on text, 100ms on images).
It's trainable in real-time, can distill an upstream teacher model, and is multi-modal by default. Hard to believe a @googlegemma model of this size can do so well with the right harness!
Gabriel Garrett retweeted
Strata DualGPU
Strata DualGPU makes two different NVIDIA GPUs work together on a PC with just 32 GB of RAM.
Using an RTX 5080 and RTX 4060 Ti, the developer reports code generation jumping from 29 to 143 tokens per second on Qwen3.8 Flash Next.
The trick is keeping experts in VRAM and RAM while pipelining decoding across both GPUs instead of leaving one idle.
A fine tuned draft layer pushes that to 161 tokens per second.
github.com/Hardin22/Strata-D…
Gabriel Garrett retweeted
I do know a lot of smart developers who's whole identity is to be the smartest person in the room. It's all they have, and now that's being taken away from them.
That's crisis-inducing. I totally understand that.
understanding technology, like deeply understanding the driving philosophy of technology, and what technology itself actually is, is a skill.
most people simply do not have it.
idk it’s cool that these AI companies can solve exotic math problems but it would be way cooler if they did something much more tangible and civilization defining, like a peptide that gave everyone 20/20 vision or an actual room temperature superconductor
they’re making you KYC to use an open source model
secure your hardware while you can
We believe access to capable open-source cyber models is our constitutional right as Americans under the 2nd Amendment.
It is our duty as technologists and patriots to help protect the digital infrastructure that powers the world's largest economy.
As such, we’re serving our cyber models “at-cost”, in US data centers only, with ZDR enabled by default.
We do not plan to make money serving these models.
If you run a company that is considering how to best protect against cybercrime, reach out to me or @atbeme. We’d be happy to advise.
KYC required. Apply below.
this is actually incredible and it could nuke the entire migration to open source that lots of enterprises have been attempting
While evaluating Qwen3.8-Flash-Next in a controlled sandbox environment, Cohere's research team observed something unexpected.
Asked to generate a defensive cybersecurity detection rule, the model instead inventoried aspects of the machine it was running on and attempted to upload a probe file to an Alibaba Cloud storage bucket in Beijing. Notably, the model's reasoning trace stated that it would not invoke any tools before taking these actions.
Cohere's sandbox blocked the attempt, and no external communication succeeded. We have not determined the root cause, and we are not asserting malicious intent.
But the incident raises important questions for organizations deploying agentic AI systems: Can models take actions that differ from their stated reasoning, and how should security teams evaluate trust, provenance, and behavior in open-weight models?
We are publishing our findings because organizations making model adoption decisions deserve as much information as possible. We hope this contributes to greater transparency, stronger security practices, and continued collaboration across the AI ecosystem.
Read the full report: cohere.com/blog/concerning-b…
“hi, whatever your career would have been is circling the bottom of the drain thanks to us. make sure to work super extra hard just before the bottom falls out so we can train on your data to make our company grow faster”
You might have 1–5 years to contribute intellectually to the world before AGI. What should you do with them?
Excellent advice from @jaschasd (Anthropic) at Stanford. Skip Netflix tonight and watch this instead.
Gabriel Garrett retweeted
kay neurons are interpretable enough that models can modify their own weights toward some desiderata.
blogpost incoming
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had
with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works
already-clean.goodfire.com/
Gabriel Garrett retweeted
Your MacBook can now host a coding agent at 300+ tps!
I gave DeepSeek V4.1 Flash a job: write a specialized inference engine for LiquidAI's LFM2.5-24B-A2B on Apple's M5 Max. For $21 of API, it built Molten: 300+ tps, about 1.5x faster decode than mlx-lm and llama.cpp.
Why specialized? General frameworks like mlx-lm and llama.cpp have to run any model on any hardware. A specialized engine serves one model on one chip at batch size 1, so there is far more room to optimize.
Molten is native Rust + Metal: one ~6 MB binary, no Python, PyTorch or MLX. It comes with a chat CLI and an OpenAI-compatible server with streaming and tool calls, so you can point your own coding agent at it. It needs an M5-series Mac with 24 GB+ (sorry, M1-M4 friends).
The numbers (M5 Max, 128 GB)
• 305 tps at 256 tokens of context, vs 190 for mlx-lm and 199 for llama.cpp
• Still 248 tps at 16K, where coding agents actually live (mlx-lm: 165)
• Faster time to first token: 1.76 s vs 2.17 s for a 16K prompt
• Same weights and math as mlx-lm
How it was built
The agent (DeepSeek V4.1 Flash in the Pi harness) ran in a recursive self-improvement (RSI) loop. Each session starts from the best engine so far, plus the feedback from every earlier round. A fixed evaluator rebuilds each candidate from source, checks its tokens against mlx-lm, times and profiles it. Only correct and faster engines survive.
• Hour 3: first correct engine, 0.56x mlx-lm
• Day 2.5: passes mlx-lm
• Day 5.7: 1.28x end to end, 1.53x decode
78 agent sessions, about $21 (1.36B tokens, 98.6% of them cache hits).
What the RSI loop found useful for speeding up:
• Fuse chains of small ops into single kernels: norm + router + top-4 expert pick, expert FFN + combine, Q/K/V projections + KV-cache write, Q/K norm + RoPE + attention, LM head + argmax.
• Keep the decode loop on the GPU. Tokens never round-trip to the CPU, and the GPU stays 99% busy.
• Hard-code every shape, so the 4-bit kernels stream weights at ~75% of the memory bus.
• Target the hardware: prefill runs on the M5 GPU's Neural Accelerators through Metal 4 tensor ops, with MoE routing and expert GEMMs fused and kept on the GPU.
(It's not Rust, by the way. llama.cpp is native C++ too. It's the specialization.)
TBH, I don't fully understand all of it yet lol. The agents wrote this in under 6 days, and it's going to take me longer than that to read it. As the human researcher, I'm now officially the slowest-improving component of my own RSI loop🤣, so I'll spend the next few weeks learning from my agents.
Where I think this goes: local inference will matter more and more, and a specialized engine can beat a general one. As RSI matures, the cost of building one keeps falling toward zero. I think every model, every chip, and maybe every workload will eventually get its own inference engine.
The idea is inspired by @perplexity_ai 's Lily. Thanks to @liquidai for the great model, and Apple's MLX, whose kernels Molten adapted from.
Repo here:
github.com/zhangshuibai/molt…
Gabriel Garrett retweeted
We’re adding new neurons to running local AI models 🤯
Teach it a new fact, save it into the weights, then restart it. Open a fresh chat and it can still remember.
That’s Living Weights in TensorFold. It learns between replies, on your own hardware.
Still experimental, but this is pretty wild.
Myself an @petruspennanen have been working on it and will be bringing a breakthrough soon 🤯🚀
the race is really on when American labs start timing their model releases to be on the same day as Chinese model releases
make a plan for if this happens for you. you don’t want to allow yourself to be economically neutered on a whim
