Cancel the social contract.
Joined June 2023
- Tweets785
- Following547
- Followers47
- Likes296
Hash Bam's retweeted
my spidey senses are telling me the labs are, as we speak, using unguardrailed versions of their unreleased models to systematically reverse-engineer ALL of the internet’s software, apps, and games, then feeding the reconstructed source code straight into their training pipelines while simultaneously building guardrails to ladder-pull the rest of us
Hash Bam's retweeted
We believe voice is the most natural way to interact with intelligence, so we created Dessa, the most authentic full-duplex speech model ever built. Talking with it feels like talking with a person…full of emotion, nuance, pauses, and a natural cadence.
block.xyz/dessa
Hash Bam's retweeted
Paul Biya has been president of Cameroon since 1982. At the end of the term, he’ll be 99. Nearly half the country has known no other president.
Despite serving his eighth term, he appears to spend much of his time in Switzerland. wapo.st/4i1ce9V
Hash Bam's retweeted
Researchers proved every LLM is secretly obsessed with Japan.
And they finally figured out why.
They published a paper investigating hidden cultural biases in large language models.
they tested 31,000 questions across 24 different languages on models like gemini, chatgpt, claude, and llama.
they stripped out all geographic hints and just asked the models to pick a place and answer.
the finding was universal.. across a wide range of models and languages, there is a pervasive "obsession" with japanese culture.
even when you prompt them in a language with zero ties to japan, they default to japanese norms, values, and aesthetics at a disproportionately high rate compared to its actual global demographic weight.
the craziest part? they tracked down exactly when the ai develops this obsession.. and it is not from reading the raw internet.
during pre-training, the models actually have a fairly balanced, global view of the world. the obsession gets injected during the safety phase.
when tech companies fine-tune these models to be "helpful, harmless, and polite," the ai starts mapping those corporate safety guidelines directly onto japanese cultural concepts. in the pursuit of creating the perfect, inoffensive, compliant assistant, they warped how the models view the world.
the quantization process (compressing models to run faster locally) actually amplifies this determinism. the smaller and faster your local model is, the more stereotypically it behaves, causing japanese cultural tropes to spike hard.
we spend all this time worrying about existential ai safety, but the real alignment problem is that our most advanced reasoning engines are hyper-fixated on japanese internet culture.
every model upgrade is reinforcing an invisible regional bias that dictates how they generate stories, facts, and code. the cultural commonsense of the internet is not universal..
it’s just localized to whatever the heaviest reddit and anime wiki users were obsessed with.
Hash Bam's retweeted
Introducing Tag to Rescue
Secure the future of open weight models. Tag @thepirateface in any Hugging Face model discussion on X.
Pirate Face automatically downloads the model, verifies every file, and lists it as a torrent.
View them here ↓
Hash Bam's retweeted
So many people have DM’d me about this trailer and I’ll be honest, watching it made me physically throw up.
I was COO from before FTX even had a name, it was my baby, and I was the last one out of the Bahamas after Sam got extradited. I lost everything I had on it and it took me a long time to get over, so I have zero interest in defending the people who caused this.
The media loves the story that FTX blew up because we were all partying in the Bahamas, but the place I worked looked nothing like that.
We had under 200 people in the core company at its peak, not counting subsidiaries, when other exchanges our size had thousands, so a lot of us were doing 16 to 18 hour days for over three years and barely had time to sleep, and whenever we walked in, day or night, Sam was at his desk.
I worked with Caroline and never once saw her get aggressive like she does in this trailer. She’s quiet and mild, and often lost in her own thoughts.
Most of us didn’t even know she and Sam were dating until we read about it after the bankruptcy, and I never saw him so much as put an arm around anyone.
FTX’s collapse ruined so many people’s lives, including ours, and Netflix decided to make it a love story.
Shame on you @netflix.
Hash Bam's retweeted
The real world abounds with recursively self-improving systems, but one in particular deserves attention: Science, modeled as a system (perhaps even as an agent, with goals and resources). If you want to really understand AI RSI, science should be your reference point.
Science is an intelligent system, and it is obviously recursively self-improving:
1. Scientific discoveries unlock new technology that helps build better experimental tools. This is a top driver of progress in nearly all fields.
2. They unlock new conceptual advances (ideas, theories) that help solve more problems.
3. They increase society's economic output, leading to more resources flowing into science.
4. They unlock better faster tooling (e.g. more compute via better chip & networking technology).
As a result, many measures of scientific *input* grow exponentially:
1. Headcount (doubles every ~15 years)
2. Global R&D spending (doubles a bit faster, every ~13 years)
3. Papers and patents (technically this is a measure of headcount)
4. Compute dedicated to science (doubles every ~2 years)
But is scientific progress exponential? Historically, the rate of scientific impact over time has remained roughly constant since the start of the industrial revolution (i.e. scientific progress is *linear*). 1850-1900 was about as dramatic as 1900-1950 or 1950-2000.
1850–1900: Evolution, germ theory & antiseptic surgery, thermodynamics, electromagnetic field equations, the periodic table, pharmaceuticals, electricity, telegraph and telephone, internal combustion engine, skyscrapers, mechanized agriculture...
1900–1950: special and general relativity, quantum mechanics, nuclear fission & atomic energy, antibiotics, genetic theory, electronic computers, information theory, synthetic polymers and plastics, the transistor, aviation...
1950–2000: DNA, genetic engineering, integrated circuits, microprocessors & personal computing, the Internet, crewed spaceflight, moon landing, satellite communications & GPS, standard model of particle physics...
In real terms, like life expectancy, which has increased in a remarkably linear fashion of roughly 3 months per year since 1840, progress is a straight line. This is especially apparent for fields where impact is easy to measure, like biology, medicine, and agriculture.
I first wrote about this phenomenon and its causes in 2012, and a steady stream of research has confirmed it in the years since. Examples include the 2018 paper by Nielsen and Collison, "Science Is Getting Less Bang for Its Buck," and the 2020 economic paper, "Are Ideas Getting Harder to Find?" (In fact, I believe the Nielsen paper stemmed from a conversation I had with him about this exact idea six months earlier)
In short, the primary cause is that research solves the highest-impact, easiest problems first, and every subsequent problem is either harder or lower-impact. Exponentially so. The paper that presented information theory wasn't very hard to write (single author!) but you'd have a hard time ever writing a CS paper that beats it in impact.
This is why science as a system requires exponential resources (input) to produce linear impact (output). It gets exponentially harder over time.
Worth thinking about if you're pondering RSI for AI. I fully believe AI RSI is already happening and will accelerate in the future. But I do not believe this leads to an "intelligence explosion" -- that would fly in the face of everything I know about intelligence and everything I know about recursively self-improving systems.
Hash Bam's retweeted
De 38.400 km/h à zéro en quelques minutes: voici la rentrée sur Terre d'Artemis II après son tour de Lune, vue par une GoPro collée à un des hublots.
Ambiances visuelle et sonore garanties.
📸NASA/JSC/Simeon Schmauß
Hash Bam's retweeted
🦔Someone is buying Japanese used books by the ton. Not novels or comics. Philosophy, history, medical law, Edo-period cultural texts. Multiple buyer accounts all ship to one logistics center in Okayama Prefecture that won't answer questions. Export records show over 50 tons, roughly 100,000 volumes, shipped to the US since last year. No buyer has been identified.
Separately, court documents confirmed Anthropic's "Project Panama" bought millions of books in the US, cut the spines off, scanned them, and shredded the originals. An internal memo said the goal was to "destructively scan all the books in the world." Similar operations have been reported across Europe.
My Take
Anthropic's US book operation came out in court filings earlier this year. Project Panama. Millions of books, cut the spines off, scanned, shredded. Their VP chose the codename so nobody outside the company would find out. That story broke and apparently the same operation just moved to Japan. Anonymous buyer accounts, a warehouse in Okayama that won't take questions, and 100,000 volumes of specialized academic texts that nobody buys to resell.
I don't know how you build a trillion-dollar industry on material you had to acquire in secret through middlemen because you knew the public would object. Authors spent years on those books and now they're fed through a scanner and thrown away so a chatbot can sound smarter. If the data was free to use, they wouldn't need codenames and anonymous warehouses. The lawsuits over who owns this material have barely begun and I think the copyright exposure across the whole AI industry is enormous. Anthropic's internal memo said the goal was every book in the world, which describes the foundation of the product.
Hedgie🤗
Hash Bam's retweeted
Why KV cache stores K and V vectors but never Q?
(a popular technical LLM interview question)
LLMs are autoregressive so each token is predicted from every token before it, one at a time.
This autoregressive nature has a direct consequence inside the model.
A forward pass over <n> tokens produces <n> hidden states, but only the last one is projected to logits and is required to generate the next token.
So to understand why KV cache just stores K and V vector, we must back track to see how exactly is the last hidden state produced.
Let's walk through this with a 10-token prompt.
1) Prefill:
All 10 tokens go through the model in one forward pass, in parallel (with causal masking), since the whole prompt is already known.
At every layer, each of the 10 positions produces a query, a key and a value vector, and attention at each position runs against all positions up to it.
This pass is compute-heavy, and it's why the first token takes noticeably longer than the ones after it. TTFT is mostly prefill.
2) The first output token:
To generate the 11th token, only the 10th token's hidden state is needed. So this is projected from the hidden-dim to vocab-dim to generate logits over vocab.
These logits then go through softmax and sampling to generate token 11.
3) Back-track the hidden state:
The last hidden state is the last row of the feedforward block's output. The feedforward block is position-wise (it's applied to each row independently) so that row comes from the last row of the attention output before it.
So now we need to see how the last row of attention is computed.
4) Attention matrix:
QKᵀ for a 10-token prompt will give a 10 × 10 matrix.
Row <i> will have the dot product of query <i> with every key.
Row 10 is therefore Q₁₀·K₁, Q₁₀·K₂, all the way to Q₁₀·K₁₀.
Notice that only Q₁₀ appears in it. Q₁ through Q₉ only belong to their corresponding rows 1-9, and those rows' hidden states we already discarded because they were never needed.
The last row of attention goes through softmax and multiplies the full stack of value vectors, V₁ through V₁₀, to give the last row of the attention output.
So the last hidden state depends on exactly three things: Q₁₀, every key, and every value.
5) Generating token 12:
Token 11 is appended, and this time, we need row 11's hidden state to generate token 12.
Mathematically, attention operation turns out to be Q₁₁ against K₁ through K₁₁, then multiplied by V₁ through V₁₁.
K₁ through K₁₁ and V₁ through V₁₁ are bit-for-bit what prefill + first token produced since under causal masking, a token's key and value depend on that token and the ones before it, never on anything after, so appending token 11 cannot change anything at position 3.
6) The cache state:
Overall, this implies that you just need to retain the keys and values at each decoding step, and compute only the new position's Q, K and V.
Each decode step requires one query vector, which is never used again, so they are never cached across the decoding process.
The visual below explains the entire process.
That said, KV cache is only one of four separate caching layers in an LLM stack.
The other three are prefix caching on the server, prompt caching billed by a provider, and a semantic cache that skips the model entirely.
I wrote a full breakdown of all four caches in LLM serving that you should know as an AI engineer, with code for each.
Read it below.
Hash Bam's retweeted
LLM Inference Engineering - Problem and Solution
Problem: LLMs are slow
Solution: KV Cache - Avoid recomputing previous tokens.
Problem: KV Cache consumes huge memory
Solution: PagedAttention - Manage KV memory efficiently.
Problem: GPU is underutilized
Solution: Continuous Batching - Dynamically add/remove requests.
Problem: Token generation is sequential
Solution: Speculative Decoding - Generate and verify multiple tokens together.
Problem: Attention requires large memory usage and expensive memory movement
Solution: FlashAttention - Reduce memory usage and memory movement during attention.
Problem: KV Cache is still large
Solution: MQA / GQA - Share K/V across multiple query heads.
Problem: Many optimizations need to work together
Solution: vLLM - Build an efficient inference engine.
Problem: Applications repeatedly process the same prefixes
Solution: RadixAttention / Prefix Caching - Reuse previously computed work.
Problem: Models are too large
Solution: Quantization - Reduce model memory and computation.
Problem: Large models are expensive
Solution: Distillation / SLMs / MoE - Make inference itself cheaper.
Keep Learning, Keep Sharing, and Keep Growing.
Hash Bam's retweeted
1/ I'm a cardiologist. I've practiced for twenty-five years, through a lot of "breakthroughs" that turned out to be press releases. So understand the weight of what I'm about to say: I have never seen a single week in medicine like the one we just lived through.
In the span of a few days, four separate scientific breakthroughs landed. The stock market treated them as four unrelated stories and sent a handful of biotech companies soaring. But that's the shallow read. Look closer and they are not four stories at all. They are four faces of the same story — the biggest shift in medicine since the discovery of antibiotics.
Medicine is becoming programmable. Individualized. Written for one human being instead of the average of millions.
Let me walk you through exactly what happened, in plain language, and show you where this is actually headed. Because the future arrived quietly this week, and most people scrolled right past it.
Hash Bam's retweeted
Humans are basically hardware running a local AI.
Runs on about 20W of power, needs constant refueling.
Learns from data in real time, then compresses weights and fine-tunes while sleeping.
I feel like future AI will probably end up working the exact same way.
Hash Bam's retweeted
For my first post, I’m sharing a letter @NVIDIA signed on why open models matter.
AI will transform every industry, power every company, and be built by every country.
Open models strengthen safety and cybersecurity, accelerate innovation and diffusion, and enable sovereignty.
The world needs both frontier closed models and frontier open models.
images.nvidia.com/pdf/Open-W…
Hash Bam's retweeted
Insane, Yeah he really did it, he made a RAM at home in his backyard shed.
While big tech cries about RAM shortages Man builds functional DRAM from scratch using homemade sputtering and lithography tools.
20-bit memory cell array, 12pF capacitance.
Turned it into a legit Class 100 cleanroom and fabricated memory cells himself. 5x4 memory cell array fabricated,This is the first RAM ever made at home.
Drug lab vibes, semiconductor god mode.
Hash Bam's retweeted
A French engineer who lives quietly in Paris has spent 30 years writing software that the entire internet now runs on without knowing his name.
He wrote the code that streams every YouTube video, every Netflix show, every TikTok clip. He wrote the code that runs the virtual servers underneath AWS, Google Cloud, and Microsoft Azure. He calculated more digits of pi than anyone in history. He has no Twitter. He has no marketing. He just keeps shipping.
His name is Fabrice Bellard.
Here is the story, because almost nobody outside the systems programming world knows what one man has built.
Fabrice was born in 1972 in Grenoble, France. He studied at École Polytechnique, the top French engineering school. He never went to Silicon Valley. He never built a startup empire. He just wrote code.
In 2000 he started a project called FFmpeg, an open-source multimedia framework for encoding, decoding, and streaming video. He was 28. The project did one thing nobody else had done well. It handled every video and audio format that existed, in one library, on every operating system. He led it himself for years.
Today FFmpeg is the invisible engine of the internet. YouTube uses it. Netflix uses it. VLC uses it. Chrome and Firefox use parts of it. Every Android phone, every iPhone, every smart TV, every video editing tool you have ever touched runs FFmpeg somewhere underneath. If you have watched a video on a screen in the last 20 years, Fabrice's code processed it.
He was not done.
In 2003 he started QEMU, a machine emulator and virtualizer. He wrote it solo until version 0.7.1 in 2005. QEMU lets you run any operating system on any other operating system. It became the foundation of modern virtualization. KVM, the Linux kernel hypervisor, runs on top of QEMU. Every major cloud provider, AWS, Google Cloud, Microsoft Azure, IBM Cloud, runs virtual machines on infrastructure built around it. The Quick Emulator is the most cited piece of cloud infrastructure code on Earth.
He kept going.
In 2001 he won the International Obfuscated C Code Contest with a small C compiler that grew into TCC, the Tiny C Compiler. TCC can compile and boot a Linux kernel from source in under 15 seconds. In 2004 he calculated the most digits of pi ever computed at the time, using a personal desktop computer and an algorithm he derived himself called Bellard's formula. In 2011 he wrote a complete PC emulator in pure JavaScript that runs Linux in your browser, a project called JSLinux that engineers still cannot believe is real.
In 2019 he released QuickJS, a small but complete JavaScript engine that fits where V8 cannot. In 2021 he released NNCP, a neural network based lossless data compressor that immediately took the lead on the Large Text Compression Benchmark.
Then he turned his attention to large language models. He built TextSynth Server, a web server with a REST API for running LLMs locally. He released ts_zip and ts_sms, compression utilities that use language models to compress text and short messages at ratios traditional algorithms cannot reach. He released TSAC, a very low bitrate audio compression system. In December 2025 he released Micro QuickJS, a new JavaScript engine for microcontrollers, separate from QuickJS, designed for environments with almost no memory.
Fabrice co-founded a telecom company called Amarisoft in 2012, where he serves as CTO. Amarisoft builds 4G and 5G base station software used by carriers and labs around the world. He has been running it for over a decade while continuing to ship personal projects from his own home page at bellard dot org
He has no Twitter. He has no Instagram. He gives almost no interviews. His personal website is a flat list of projects with no styling, no fonts, no marketing copy. Just titles and links.
A quiet French engineer who never moved to Silicon Valley wrote the code that quietly runs the internet.
He is still shipping.
Hash Bam's retweeted
Cette année, j'ai commencé à investir à la BRVM (Bourse Régionale des Valeurs Mobilières).
Donc j'ai cherché comment analyser les entreprises cotées. Est-ce que cette entreprise est rentable ? Est-ce que sa marge progresse ? Est-ce que son P/E est raisonnable par rapport au secteur ?
J'ai vite compris que les données existent : la BRVM publie les rapports de chaque entreprise. Mais pour avoir une vraie vue, il faut télécharger les PDF un par un, fouiller les tableaux, recalculer les ratios soi-même, et tout ça pour une seule entreprise. Recommencer pour la suivante.
J'ai essayé de trouver un outil qui fait ça proprement. Rien. Pas pour la BRVM.
Alors avec mon grand frère, expert des marchés et investisseur à la BRVM depuis bien plus longtemps que moi, on a commencé à construire un outil pour ça.
On est partis de son fichier Excel dans lequel il suivait ses investissements, vers un outil que tout le monde peut utiliser.
Ça s'appelle Vision Boursière. Tu rentres le nom d'une des 47 entreprises cotées, tu vois l'évolution de ses KPIs sur 10 ans, ses ratios, ses marges, son historique de dividendes. Tu peux comparer deux entreprises côte à côte, suivre ton portefeuille, mettre des alertes de prix.
On vient de lancer. On cherche des gens qui investissent à la BRVM ou qui veulent se lancer pour tester et nous dire ce qu'ils en pensent.
7 jours gratuits, pas de paiement requis 👇
visionboursiere.com
Si c'est ton cas, commente ou envoie-moi un message directement.
Hash Bam's retweeted
stick to theoretical computer science and algorithm design. go in depth on computer architecture, chip design, low level C & assembly, intuitively understand the world of 1s and 0s, know the different sections of memory, you hsould be able to visualize the program in memory; go in depth on GPU architecture, marinate your brain in it, you must be able to swim through a gpu in your imagination. strip it naked to the first principles.