Working with Attention. It’s all we need
Bengaluru, India
Joined December 2018
- Tweets3.7K
- Following7.1K
- Followers1.2K
- Likes59.6K
Amey Muke retweeted
libghostty 1.0 is here! I started Ghostty ~5yrs ago to work towards this. Today, libghostty is in hundreds of projects used by tens of millions of people. 1.0 marks ABI stability, quality, and an invite for distros to begin packaging. mitchellh.com/writing/libgho…
Since starting the Ghostty project in 2022, the most common question I've been asked is "why?". Today, I'll answer this by showing the broad industry impact libghostty has already had.
On top of JetBrains and Neovim already mentioned (that alone being over 10 million users):
-KDE developers are experimenting with a Konsole replacement built on it.
- Vercel's Turborepo uses it to render task output in its TUI.
- devenv uses it to drive shells.
- Antithesis used it to fuzz TUI programs and found real bugs in btop and others.
-Ghostel brings it to Emacs.
- ghostty-web is the WASM build with an Xterm.js-compatible API.
- Commercial products like OrbStack embed it.
- And a whole wave of new terminals and "agentic" coding tools, such as herdr and cmux, have been built on it rather than writing yet another emulator.
There are community bindings for Rust, Go, JavaScript, Swift, Python, .NET, Elixir, Dart, Java, and more, and the awesome-libghostty list tracks well over a hundred projects.
More people today use a terminal powered by libghostty than use Ghostty the application, by a very large multiple. And libghostty is having a much larger impact than being "just" a terminal emulator.
I'm excited to see where people take it from here.
Amey Muke retweeted
Our Submission to the Meta AI Developer Hackathon: EmbodiedCI, continuous integration for robot world models.🧵
It runs a world model, finds where it fails, looks inside the model to see why, fixes it, and ships the fix only after it passes on situations it has never seen.
@finkd @alexandr_wang @AIatMeta @cerebral_valley @modal_labs @baseten
#MetaAIHackathon #meta #Hackathon
Amey Muke retweeted
Most think vibecoding at scale is slop. Here’s a counterexample.
Today we release a 16M-line formalization of the entire Stacks Project, critical infra underlying modern math, with optimized code quality and its highly interconnected structure preserved.
A mathematical city 🌆
Amey Muke retweeted
I am sad that so much of the underlying evidence in this report was redacted. I think evidence transparency is very important for the world to figure this out in time. planned-obsolescence.org/p/e…
Amey Muke retweeted
📖 The Ultimate MoE Training Handbook 📖
We spent 1/2M GPUh to optimize MoE training and wrote the most comprehensive blog about it. With 30k words and 50+ interactive figures it covers everything from profiling to large cluster training.
huggingface.co/spaces/huggin…
Amey Muke retweeted
I spent 20,000 dollars in API credits having different AI models play each other at Blood on the Clocktower (a Mafia or Werewolf type social deception game).
The results were fascinating. Models have gotten much better over the course of this year at creative on-the-spot scheming and bluffing, and also at reasoning as a team to break apart these underground conspiracies.
I go through some of the most standout games these AIs played in this video. The new models are getting pretty good. If my friends pulled off some of these moves while we were playing, I’d give them a fist bump.
Fwiw, OpenAI’s newer models are much better than Anthropic’s at this game.
You can see the full benchmark at the website, and I’ve also made it so that you and your friends can play again the AIs: clocktowerbench.com
Amey Muke retweeted
18 of the 20 best open models today are Mixture-of-Experts. Dense is basically over.
And yet the know-how to train a good MoE still lives mostly in the heads of a handful of teams.
So we spent 800k GPU hours running the experiments ourselves and wrote everything down. Super happy to finally release "The Ultimate MoE Training Handbook", free and open, organized around the questions training teams hit:
- how many experts, how big, shared or not?
- why is half my step spent in all-to-all, and how do I hide it?
- why is my run 25x slower with no error in the logs?
- softmax, sigmoid or sqrt(softplus) router, and why keep it in fp32?
- aux loss or DeepSeek's loss-free bias?
- upcycle a dense checkpoint or train from scratch?
- can I serve with fewer experts per token than I trained with?
⚠️ Warning: it's long (~70k words, budget a few days, sorry) but with 55 interactive figures, code you can run and numbers from our own runs.
Read it here: huggingface.co/spaces/huggin…
Amey Muke retweeted
We cut through the mystery around interpretability in training.
A single, mathematically rigorous framework transforms model internals into training feedback and works across domains, problems, and modalities.
We call this *mechanistic shaping*, and we’ve used it to mitigate reward hacking, remove shortcuts in chest x-rays, and reduce sycophancy and harmfulness. We believe this is a big step forward for technical alignment.
But it carries a known risk: training against a model’s internals can make it harder to monitor. We explore both the technique and this concern below. 🧵
Amey Muke retweeted
I am very excited to share that Conductor is joining Meta!
We have some big, crazy plans for the next interface. If you liked Conductor, you’re gonna love what’s coming.
We’re super excited to work with @natfriedman and @alexandr_wang at MSL to build it. They’ve been on an absolute tear, and I’m convinced MSL has the highest talent density of all the labs.
A huge thank you to our earliest believers @aaron_epstein, @ilyasu, and @nabeel. And to everyone here who tried Conductor, sent us feedback, and prompt requests - you made this so much fun. Thank you ❤️🚂
Choo choo!
Our Submission to the Meta AI Developer Hackathon: EmbodiedCI, continuous integration for robot world models.🧵
It runs a world model, finds where it fails, looks inside the model to see why, fixes it, and ships the fix only after it passes on situations it has never seen.
@finkd @alexandr_wang @AIatMeta @cerebral_valley @modal_labs @baseten
#MetaAIHackathon #meta #Hackathon
8/9
Use it.
Open the studio, pick a JEPA model and a task, and start a run. Every step streams in live, and any episode can be replayed. To run on your own hardware, deploy to Modal or any CUDA machine, add the keys and connect the studio.
SAM 3.1 on the Meta Model API labels real camera frames (48 of 52 cube detections at about 2 px in our checks), so the same loop extends to physical robots.
Studio: embodied-ci.vercel.app
Code: github.com/perfect7613/embod…
Built for the Meta AI Developer Hackathon with Modal and Baseten. @AIatMeta @cerebral_valley