Working with Attention. It’s all we need
Bengaluru, India
Joined December 2018
- Tweets3.7K
- Following7.1K
- Followers1.2K
- Likes59.7K
Amey Muke retweeted
libghostty 1.0 is here! I started Ghostty ~5yrs ago to work towards this. Today, libghostty is in hundreds of projects used by tens of millions of people. 1.0 marks ABI stability, quality, and an invite for distros to begin packaging. mitchellh.com/writing/libgho…
Since starting the Ghostty project in 2022, the most common question I've been asked is "why?". Today, I'll answer this by showing the broad industry impact libghostty has already had.
On top of JetBrains and Neovim already mentioned (that alone being over 10 million users):
-KDE developers are experimenting with a Konsole replacement built on it.
- Vercel's Turborepo uses it to render task output in its TUI.
- devenv uses it to drive shells.
- Antithesis used it to fuzz TUI programs and found real bugs in btop and others.
-Ghostel brings it to Emacs.
- ghostty-web is the WASM build with an Xterm.js-compatible API.
- Commercial products like OrbStack embed it.
- And a whole wave of new terminals and "agentic" coding tools, such as herdr and cmux, have been built on it rather than writing yet another emulator.
There are community bindings for Rust, Go, JavaScript, Swift, Python, .NET, Elixir, Dart, Java, and more, and the awesome-libghostty list tracks well over a hundred projects.
More people today use a terminal powered by libghostty than use Ghostty the application, by a very large multiple. And libghostty is having a much larger impact than being "just" a terminal emulator.
I'm excited to see where people take it from here.
Amey Muke retweeted
Our Submission to the Meta AI Developer Hackathon: EmbodiedCI, continuous integration for robot world models.🧵
It runs a world model, finds where it fails, looks inside the model to see why, fixes it, and ships the fix only after it passes on situations it has never seen.
@finkd @alexandr_wang @AIatMeta @cerebral_valley @modal_labs @baseten
#MetaAIHackathon #meta #Hackathon
Amey Muke retweeted
Most think vibecoding at scale is slop. Here’s a counterexample.
Today we release a 16M-line formalization of the entire Stacks Project, critical infra underlying modern math, with optimized code quality and its highly interconnected structure preserved.
A mathematical city 🌆
Amey Muke retweeted
I am sad that so much of the underlying evidence in this report was redacted. I think evidence transparency is very important for the world to figure this out in time. planned-obsolescence.org/p/e…
Amey Muke retweeted
📖 The Ultimate MoE Training Handbook 📖
We spent 1/2M GPUh to optimize MoE training and wrote the most comprehensive blog about it. With 30k words and 50+ interactive figures it covers everything from profiling to large cluster training.
huggingface.co/spaces/huggin…
Amey Muke retweeted
I spent 20,000 dollars in API credits having different AI models play each other at Blood on the Clocktower (a Mafia or Werewolf type social deception game).
The results were fascinating. Models have gotten much better over the course of this year at creative on-the-spot scheming and bluffing, and also at reasoning as a team to break apart these underground conspiracies.
I go through some of the most standout games these AIs played in this video. The new models are getting pretty good. If my friends pulled off some of these moves while we were playing, I’d give them a fist bump.
Fwiw, OpenAI’s newer models are much better than Anthropic’s at this game.
You can see the full benchmark at the website, and I’ve also made it so that you and your friends can play again the AIs: clocktowerbench.com
Amey Muke retweeted
18 of the 20 best open models today are Mixture-of-Experts. Dense is basically over.
And yet the know-how to train a good MoE still lives mostly in the heads of a handful of teams.
So we spent 800k GPU hours running the experiments ourselves and wrote everything down. Super happy to finally release "The Ultimate MoE Training Handbook", free and open, organized around the questions training teams hit:
- how many experts, how big, shared or not?
- why is half my step spent in all-to-all, and how do I hide it?
- why is my run 25x slower with no error in the logs?
- softmax, sigmoid or sqrt(softplus) router, and why keep it in fp32?
- aux loss or DeepSeek's loss-free bias?
- upcycle a dense checkpoint or train from scratch?
- can I serve with fewer experts per token than I trained with?
⚠️ Warning: it's long (~70k words, budget a few days, sorry) but with 55 interactive figures, code you can run and numbers from our own runs.
Read it here: huggingface.co/spaces/huggin…
Amey Muke retweeted
We cut through the mystery around interpretability in training.
A single, mathematically rigorous framework transforms model internals into training feedback and works across domains, problems, and modalities.
We call this *mechanistic shaping*, and we’ve used it to mitigate reward hacking, remove shortcuts in chest x-rays, and reduce sycophancy and harmfulness. We believe this is a big step forward for technical alignment.
But it carries a known risk: training against a model’s internals can make it harder to monitor. We explore both the technique and this concern below. 🧵
Amey Muke retweeted
I am very excited to share that Conductor is joining Meta!
We have some big, crazy plans for the next interface. If you liked Conductor, you’re gonna love what’s coming.
We’re super excited to work with @natfriedman and @alexandr_wang at MSL to build it. They’ve been on an absolute tear, and I’m convinced MSL has the highest talent density of all the labs.
A huge thank you to our earliest believers @aaron_epstein, @ilyasu, and @nabeel. And to everyone here who tried Conductor, sent us feedback, and prompt requests - you made this so much fun. Thank you ❤️🚂
Choo choo!
Amey Muke retweeted
no MIT. no harvard. no institutional funding round.
we still outscore Hive ($2B valuation) and Reality Defender (~$50M raised) on accuracy on an independent blind-labeled benchmark.
And we're the fastest deepfake detector in the world.
we're just people who decided to win anyway. here's to witnessing that journey!
today we're opening @DetectIfAI to every developer.
real-time voice deepfake detection, 24 ms per check.
#1 in latency on Podonos' independent benchmark, ahead of companies that raised hundreds of millions.
97.3% of cloned voices caught.
API + MCP. link in the comments.
Amey Muke retweeted
When we need the trillion param model , there you have another decision model. Laya and vega already gives you to attach even a glm series of model, then added conformal, abstain and outbound and anyone can fine tune ir fit and calibration on their data to reduce ECE
Amey Muke retweeted
maybe a hot take but i now consider adversarial reviews mostly a waste of tokens. far better to use those tokens on empirical combined with formal verification. i have updated pstack to use the interrogate skill less, and eventually i think i may remove it altogether
i have found very little value with code review in the abstract, agents tend to just get lost in the nitpicks and yagnis
Amey Muke retweeted
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math.
It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training.
We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵
Blog: rikiyat.github.io/blog/weigh…
Readers added context they thought people might want to know
LLM was given weights explicitly as text in prompt and used code/forward passes (per authors' logs) to ablate Qwen looping direction, a known thinking-mode issue per model card. Primarily fixed repetition in that mode, not general 'natively' improving weights/math.
rikiyat.github.io/blog/weight-sp…
github.com/RikiyaT/lm_und…
huggingface.co/Qwen/Qwen3.5-0…
huggingface.co/RikiyaT/Qwen3.…
Amey Muke retweeted
some things I often say to claude when doing perf optimization work:
> how do we rewrite this to better leverage data-oriented design principles
> can you replace those hash tables with a flat array? can we make the pointers an index into the array? can we pack the most commonly used data into upper bits of the index?
> caching is cope for a slow design. do not add caches and delete all existing caches. instead, redesign it to be faster
> should this be a structure of arrays instead of an array of structures?
Amey Muke retweeted
Day 28/ tibo's game has landed
i spent $5.99 on this domain, but i had a blast making it with grok bot and cursor! i hope you enjoy it and find all the easter eggs 🤍
if i did it right you should be able to play it inside of X too
tibo.dog
Amey Muke retweeted
Weekend hack: HackerTube. I was annoyed with YouTube's recommendations so I had Claude filter down 43M comments over 20 years of HN to 6,454 videos that keep being recommended. *Significantly* better viewing experience, ~2 hours with Opus 5.5:
hackertube.org/
- Excellent recs without any logins (state is kept in local browser session)
- Thumbnail is a representative frame from middle of the video instead of the super duper A/B test hyper-optimized user uploads
- Underlying data from the comment analysis as one zip file, point your Claude at it and ask for your own version
- HackerTube Longs ✊
A bit more on the process:
I use Claude Code (desktop app for Mac), Opus 5.5 on Medium effort as my default. I only talk to Claude and have it do everything, the only "manual work" I helped with in tandem was in the browser (reminiscent of MenuGen): 1) I had to create some kind of API key from Google Cloud UI to query YouTube for video metadata, 2) I booted up a VPS box for serving and 3) I bought the domain name. Once the box was up I gave Claude the IP and told it update, secure, harden, deploy and optimize the code, and it did the whole thing.
Many awesome projects that would have normally taken weeks/months even can now be few-shotted for breakfast. What else are we not thinking of.
Amey Muke retweeted
Warranty dispute with @Lenovo_in @Lencare_In. LOQ 15ARP9 (S/N MP2P****).
5 cases since Aug 2025. Camera issue → laptop dead (No POST). They shipped a motherboard, refused to install it, then emailed "Onsite completed - fixed" while it's still dead. Thread:
