fun demo! this runs on the ontologizer work we’ve been doing. sparse autoencoders can’t make this chart. they give you a bag of 16k latents per person. an ontologizer gives every attendee a discrete code word, which is why you get a typology instead of a feature dump.

happy to announce @flockcap has signed its first contract in AI hardware. can't say who yet, but we're really excited about the product. more soon!

we made sparse autoencoders compete with tiny softmax classifiers and the classifiers won on bits 🐐 a pure discrete code (no coefficients!) beats a top-k SAE at matched bits on SONAR, and an 800-bit hard code recovers 97.6% of gpt-2's loss paper out now! (link in replies)