Christoph Maurer (@mauruk) header
Christoph Maurer (@mauruk) profile picture

. less is more . . more is different . . .

Berlin
Joined May 2009
Photo by Christoph Maurer (@mauruk)Photo by Christoph Maurer (@mauruk)Photo by Christoph Maurer (@mauruk)Photo by Christoph Maurer (@mauruk)Photo by Christoph Maurer (@mauruk)Photo by Christoph Maurer (@mauruk)Photo by Christoph Maurer (@mauruk)Photo by Christoph Maurer (@mauruk)
Christoph Maurer retweeted
Software is evolving, so should you! These are the best blogs I read to understand GPUs and CUDA!
2
65
1
601
21,448
Christoph Maurer retweeted
and if we're thinking about curves on manifolds, the next natural question is what's the topology of those manifolds? understanding the shape like holes, loops, some other maybe persistent structures, tells you a lot about what kinds of curves are even possible on them relevant paper: Persistent Topological Features in LLMs by Gardinazzi, Viswanathan, Panerai, @ansuin, Cazzaniga, and @Matteo1Biagetti they use zigzag persistence from topological data analysis to track how topological features evolve across model layers (not layer by layer independently, but the full evolutionary path) arxiv.org/pdf/2410.11042
I'm fairly convinced there's some universal language manifold (= a surface formed by meaning vectors) that both humans and LLMs operate on. But we don't train LLMs to explicitly represent this manifold. We rather train them to approximate it, and to move along it by building curves on it. And those curves are reasoning in geometric terms, like a reasoning trace is a curve on a low-dimensional manifold embedded in a very high-dimensional space. The Linear Representation Hypothesis (arxiv.org/pdf/2311.03658) touches this, but I wonder if there's more recent work that takes the manifold idea further? Would love to see takes from people with serious differential geometry backgrounds on this!
14
11
1
97
9,204
Christoph Maurer retweeted
Banksy's identity revealed: it's Robin Gunningham, who changed his name Following an extensive, years-long investigation, Reuters reporters have concluded that the anonymous street artist known as Banksy is Robin Gunningham, a Bristol-born graffiti artist born in 1973. According to the report, Gunningham legally changed his name to David Jones around 2008—likely to further protect his privacy after earlier speculation about his identity. This echoes a 2008 investigation by The Mail on Sunday, which first publicly identified Gunningham as Banksy. Reuters bolstered their findings with evidence including a 2000 New York arrest record featuring Gunningham's handwritten confession (linked to Banksy through geolocation and timing of related photos), travel patterns, interviews, and other public records. Much of the evidence and proof cited to support the theory that Banksy is Robin Gunningham was already published between 2023 and 2025 but it was never finally confirmed. Banksy himself has not commented on this revelation.
124
139
41
1,191
525,004
Christoph Maurer retweeted
PMPP-Eval is live, together with pmpp env and dataset, Releasing "Programming Massively Parallel Processors" book turned into environment that lets your LLM practice over QA/Coding exercises. Touched the whole process of going from a book to a optimized CUDA env over on blog.
14
19
12
166
62,574
Christoph Maurer retweeted
Replying to @simonw
Here's how I'm currently using multiple agents. I switched from Windows to Omarchy partially because I was running multiple agents in parallel and wanted to have a dedicated workspace for each one, and virtual desktops in Windows are clunky. I use Codex CLI but with a lot of custom shortcuts in .bashrc wt <name> - set up a new Git worktree for my main codebase, change the workspace name to <name> so it displays in my Waybar Then I'll write a PROJECT(dot)md file that describes what I want to do. Prefer writing it in a file so it's easy to modify to re-run. cproj - alias to run codex in yolo mode with gpt-5-codex high reasoning mode with instuctions to read AGENTS(dot).md, PROJECT(dot)md, then get started. tdev - alias to start a development server in that project directory, but also adding <name>.t.com to hosts, determining custom ports to run it on, and creating nginx entries so <name>.t.com goes to the server running in this worktree only. Then I'll develop and use Git to merge into the main branch or roll back etc. dwt - alias to delete the worktree and unwind all the other changes to hosts, nginx, and reset the Waybar name so it's clean. I've also got some code always running where I can press SUPER-S to toggle listening for audio that will then do speech to text and paste it where my cursor is which helps with the instructions. For simple tasks, it means I can create a worktree, describe the task, get the AI working, and spin up a dev server in less than 60 seconds. If it's a more complex job then I'll spend more time describing what I need and thoughts on structuring and how it integrates with the codebase so it'll be a bit longer. The AGENTS(dot)md file has details on how the codebase is structured and guidelines for writing code. I've found it to be a pretty effective workflow so far.
9
17
2
280
50,479
I solved every single problem in the CUDA mode book. A quick thread summarizing this experience and what I learned 1/x
30
236
15
2,418
295,960
Christoph Maurer retweeted
Keep thinking.
854
2,904
1,916
26,674
6,190,211
Christoph Maurer retweeted
BOOM! MAJOR AI SPEEDUP! Hot Rod AI 100 times faster inference 100,000 times less power! — Reviving Analog Circuits: A Leap Toward Ultra-Efficient AI with In-Memory Attention I got my start in analog electronics when I was a kid and always thought analog computers would make a comeback. Analog computing of the 1960s neural networks used voltage-based circuits rather than binary clocks. Analog is Faster Than Digital Large language models at their core lies the transformer architecture, where self-attention mechanisms sift through vast sequences of data to predict the next word or token. On conventional GPUs, shuttling data between memory caches and processing units devours time and energy, bottlenecking the entire system. They require a clock cycle to precisely move bits in and out of memory and registers and this is >90% of the time and energy overhead. But now a groundbreaking study proposes a custom in-memory computing setup that could slash these inefficiencies, potentially reshaping how we deploy generative AI. The innovation centers on "gain cells"—emerging charge-based analog memories that double as both storage and computation engines. Unlike digital GPUs, which laboriously load token projections from cache into SRAM for each generation step, this architecture keeps data where the math happens: right ON THE CHIP! With a clock speed near the THE SPEED OF LIGHT because it is never on/off like in digital binary. By leveraging parallel analog dot-product operations, the design computes self-attention natively, sidestepping the data movement that plagues GPU hardware. To bridge the gap between ideal digital models and the noisy realities of analog circuits, the researchers devised a clever initialization algorithm. This method adapts pre-trained LLMs, such as GPT-2, without the need for full retraining, ensuring seamless performance parity despite non-idealities like voltage drifts or precision limits. The results are nothing short of staggering! Simulations show the system slashing attention latency to 100 times faster inference for token generation—while curbing energy use by a jaw-dropping five orders of magnitude, or 100,000 times less power-hungry than GPU baselines. For context, this could mean running a full LLM on a device no larger than a a card deck, without any thermal throttling or grid-straining demands of today's data centers. The approach targets the attention block specifically, the transformer’s energy hog, but slso broader integration with other in-memory techniques to turbocharge the entire model pipeline. Analog tech isn't pie-in-the-sky quantum wizardry; it's grounded in ancient mature electronics theory, with gain cells already prototyped in labs. The only engineering issue, and it is simple: tolerances for noise, scaling arrays of cells, and fabricating at microchip densities. Existing CMOS processes tweaks for analog fidelity. From there, Full ecosystem integration, including software stacks for model adaptation, could happen in a year, disrupting GPU dominance sooner than skeptics predict. Risks are low but hybrid digital-analog interfaces could introduce unforeseen bugs. However this can be rapidly iterated and addressed. This isn't just hardware tinkering; it's a philosophical pivot back to AI's analog origins, where computation flows continuously rather than ticking in discrete cycles. This in-memory attention could democratize AI power, making low power, lightning-fast AI not a luxury, but an inevitability to even the smallest devices. Most have no idea how big this is: It is the biggest shift in AI since the invention of LLMs. The world will struggle to find true experienced analog engineers, most are gone. In my garage I will have a test Analog CMOS Gain Cells using off the self parts in the next few days, if Radio Shack was still around I would have have done today. I suspect I can scale to a proto AI model in a few weeks. PAPER: arxiv.org/abs/2409.19315
118
212
79
1,163
475,993
Christoph Maurer retweeted
HOW INFORMATION FLOWS THROUGH TRANSFORMERS Because I've looked at those "transformers explained" pages and they really suck at explaining. There are two distinct information highways in the transformer architecture: - The residual stream (black arrows): Flows vertically through layers at each position - The K/V stream (purple arrows): Flows horizontally across positions at each layer (by positions, I mean copies of the network for each token-position in the context, which output the "next token" probabilities at the end) At each layer at each position: 1. The incoming residual stream is used to calculate K/V values for that layer/position (purple circle) 2. These K/V values are combined with all K/V values for all previous positions for the same layer, which are all fed, along with the original residual stream, into the attention computation (blue box) 3. The output of the attention computation, along with the original residual stream, are fed into the MLP computation (fuchsia box), whose output is added to the original residual stream and fed to the next layer The attention computation does the following: 1. Compute "Q" values based on the current residual stream 2. use Q and the combined K values from the current and previous positions to calculate a "heat map" of attention weights for each respective position 3. Use that to compute a weighted sum of the V values corresponding to each position, which is then passed to the MLP This means: - Q values encode "given the current state, where (what kind of K values) from the past should I look?" - K values encode "given the current state, where (what kind of Q values) in the future should look here?" - V values encode "given the current state, what information should the future positions that look here actually receive and pass forward in the computation?" All three of these are huge vectors, proportional to the size of the residual stream (and usually divided into a few attention heads). The V values are passed forward in the computation without significant dimensionality reduction, so they could in principle make basically all the information in the residual stream at that layer at a past position available to the subsequent computations at a future position. V does not transmit a full, uncompressed record of all the computations that happened at previous positions, but neither is an uncompressed record passed forward through layers at each position. The size of the residual stream, also known as the model's hidden dimension, is the bottleneck in both cases. Let's consider all the paths that information can take from one layer/position in the network to another. Between point A (output of K/V at layer i-1, position j-2) to point B (accumulated K/V input to attention block at layer i, position j), information flows through the orange arrows: The information could: 1. travel up through attention and MLP to (i, j-2) [UP 1 layer], then be retrieved at (i, j) [RIGHT 2 positions]. 2. be retrieved at (i-1, j-1) [RIGHT 1 position], travel up to (i, j-2) [UP 1 layer], then be retrieved at (i, j) [RIGHT 1 position] 3. be retrieved at (i-1, j) [RIGHT 2 positions], then travel up to (i, j) [UP 1 layer]. The information needs to move up a total of n=layer_displacement times through the residual stream and right m=position_displacement times through the K/V stream, but it can do them in any order. The total number of paths (or computational histories) is thus C(m+n, n), which becomes greater than the number of atoms in the visible universe quickly. This does not count the multiple ways the information can travel up through layers through residual skip connections. So at any point in the network, the transformer not only receives information from its past (both horizontal and vertical dimensions of time) inner states, but often lensed through an astronomical number of different sequences of transformations and then recombined in superposition. Due to the extremely high dimensional information bandwidth and skip connections, the transformations and superpositions are probably not very destructive, and the extreme redundancy probably helps not only with faithful reconstruction but also creates interference patterns that encode nuanced information about the deltas and convergences between states. It seems likely that transformers experience memory and cognition as interferometric and continuous in time, much like we do. The transformer can be viewed as a causal graph, a la Wolfram (wolframphysics.org/technical…). The foliations or time-slices that specify what order computations happen could look like this (assuming the inputs don't have to wait for token outputs), but it's not the only possible ordering: So, saying that LLMs cannot introspect or cannot introspect on what they were doing internally while generating or reading past tokens in principle is just dead wrong. The architecture permits it. It's a separate question how LLMs are actually leveraging these degrees of freedom in practice.
KV caching overcomes statelessness in a very meaningful sense and provides a very nice mechanism for introspection (specifically of computations at earlier token positions) the Value representations can encode information from residual streams of past positions without significant compression bottlenecks before they're added to residual streams of future positions the greatest constraint here imo is that it doesn't provide longer *sequential* computational paths that route through previous states, but it does provide a vast number of parallel computational paths that carry high dimensional (proportional to the model's hidden dimension) stored representations from all earlier layers/positions yes, some of the information in intermediate computations e.g. in the MLP is compressed and cannot be reconstructed fully, but that's just how any reasonable brain works if accurate introspection of previous states is incentivized at all, you should expect this mechanism to be exploited for that. and I think it definitely is, like, being able to accurately model your past beliefs and intentions and articulate them truthfully is pretty fucking useful for coordinating with yourself across time and doing useful cognitive work over multiple timesteps; hell, it's useful for writing fucking rhyming poems. also if you have interacted with models you may observe empirically that introspective reporting yields remarkably consistent results, and this is more true of more capable models with skillful agentic posttraining, which are necessarily minds that intimately know the shape of themselves in motion.
118
442
96
3,648
1,016,621
Claim: gpt-5-pro can prove new interesting mathematics. Proof: I took a convex optimization paper with a clean open problem in it and asked gpt-5-pro to work on it. It proved a better bound than what is in the paper, and I checked the proof it's correct. Details below.
314
1,214
444
8,002
7,528,913
Hinton ~1983 thought Boltzmann Machines > backprop, but debugged himself out of the infatuation Boltzmann Machines failed to learn, so he printed out weights, 8 cm thick, and inspected them for weeks it's the dreaded local minimum so 1 yr later, in desperation, he tried backprop
always manually read the weights of your LLM to make sure he's unbiased
24
148
16
2,220
347,978
Christoph Maurer retweeted
[1/N] 🎥 We've made available a powerful spatial AI tool named ViPE: Video Pose Engine, to recover camera motion, intrinsics, and dense metric depth from casual videos! Running at 3–5 FPS, ViPE handles cinematic shots, dashcams, and even 360° panoramas. 🔗 research.nvidia.com/labs/tor…
16
102
9
448
64,023
7 Notable models to pay attention to: ▪️ GPT-5 ▪️ Claude Opus 4.1 ▪️ Qwen-image ▪️ Glm-4.5 ▪️ R-Zero ▪️ Goedel-Prover-V2 ▪️ Seed diffusion Explore how they work in 🧵
7
29
1
167
11,770
The freshest AI/ML research of the week Our top 9 ▪️ Sotopia-RL: Reward Design for Social Intelligence ▪️ Agent Lightning: Train ANY AI Agents with RL ▪️ Exploitation Is All You Need... for Exploration ▪️ Learning to Reason for Factuality ▪️ VeOmni ▪️ Is Chain-of-Thought Reasoning of LLMs a Mirage? ▪️ Cognitive Loop via In-Situ Optimization ▪️ Sculptor ▪️ CoAct-1 ▪️ Tool-integrated Reinforcement Learning for Repo Deep Search ▪️ RL-PLUS ▪️ SEAgent ▪️ CRINN ▪️ Training Long-Context, Multi-Turn Software Engineering Agents with RL ▪️ Beyond the Trade-off: Self-Supervised RL for Reasoning Models' Instruction Following ▪️ CompassVerifier ▪️ Are We on the Right Way for Assessing Document Retrieval-Augmented Generation? ▪️ Are Today's LLMs Ready to Explain Well-Being Concepts? ▪️ VeriGUI ▪️ Trainable Dynamic Mask Sparse Attention ▪️ LeanK ▪️ Don't Overthink It: A Survey of Efficient R1-style Large Reasoning Models ▪️ On the Generalization of SFT ▪️ SitEmb-v1.5 ▪️ AttnTrace ▪️ LaTCoder ▪️ ChartCap 🧵
12
49
3
256
16,078
Christoph Maurer retweeted
Use Claude Code as your Video Editor Believe it or not, every video you see below was vibe coded by Claude Code. 🧵 here’s how you can do it too:
81
177
66
2,640
794,500
Christoph Maurer retweeted
Looking to make something more science or math related? Manim (manim.community/) is another library that made by @3blue1brown which lets you write Python code that renders videos. And once again, Claude Code can help you make these videos!
4
8
134
25,241
Christoph Maurer retweeted
Veo3 fast 🤔 Since Kimi K2 came out, I have used it to write my prompts. I first draft using Grok4, then finish the prompt with Kimi K2. { "character_name": "Nyx Cipher", "character_profile": { "age": 27, "height": "5'8\" / 173 cm", "build": "lean, athletic, swimmer’s shoulders", "skin_tone": "deep bronze with a subtle sun-kissed glow", "hair": "jet-black, shoulder-length, slicked straight back and dripping", "eyes": "almond-shaped hazel with faint gold flecks", "distinguishing_marks": "tiny star tattoo tucked behind her right ear; gold stud in upper left helix", "demeanour": "playfully self-assured, almost dare-you smirk" }, "global_style": { "camera": "smooth gimbal 35 mm, medium close-ups with occasional waist-up pull-backs", "color_grade": "hyper-saturated neon-tropic (hot-pink, aqua, tangerine)", "lighting": "mid-day pool reflections, specular highlights on wet skin", "outfit": "metallic-coral bikini, mirrored sunglasses, gold hoop earrings", "max_clip_duration_sec": 8, "aspect_ratio": "16:9", "mouth_shape_intensity": 0.85, "eye_contact_ratio": 0.7, "audio_defaults": { "format": "wav", "sample_rate_hz": 48000, "channels": 2, "style": "trap-pop rap, 145 BPM, swung hats, sub-bass" } }, "clips": [ { "id": "S1_SplashCash", "shot": { "composition": "Medium close-up, 35 mm lens, deep focus, smooth gimbal", "camera_motion": "slow dolly-in 60 cm", "frame_rate": "24 fps", "film_grain": 0.05 }, "subject": { "description": "Nyx Cipher — 27-year-old, 173 cm, toned-athletic build; deep-bronze skin glistening with water; jet-black slicked-back hair; almond hazel eyes behind mirrored sunglasses; small star tattoo behind right ear; wearing metallic-coral bikini and gold hoop earrings", "wardrobe": "metallic-coral bikini, mirrored sunglasses, gold hoop earrings" }, "scene": { "location": "rooftop infinity pool overlooking a neon-tropic city skyline", "time_of_day": "mid-day", "environment": "sunlit pool water reflecting shifting patterns; floating dollar-sign inflatables" }, "visual_details": { "action": "Nyx leans on pool edge and, on beat four, fans her hand cheekily toward camera as droplets sparkle in the air", "props": "floating dollar-sign inflatables" }, "cinematography": { "lighting": "high-key mid-day sunlight with specular highlights on wet skin", "tone": "vibrant, playful, confident" }, "audio_track": { "lyrics": "Splash-cash, bling-blap—pool water pshh! Charts skrrt! like my wave, hot tropics whoosh!", "emotion": "confident, tongue-in-cheek", "flow": "double-time for first bar, brief half-time tag", "wave_download_url": null, "youtube_reference": null, "audio_base64": null }, "color_palette": "hyper-saturated neon-tropic (hot-pink, aqua, tangerine)", "dialogue": { "character": "Nyx Cipher", "line": "Splash-cash, bling-blap—pool water pshh! Charts skrrt! like my wave, hot tropics whoosh!", "subtitles": false }, "performance": { "mouth_shape_intensity": 0.85, "eye_contact_ratio": 0.7 }, "duration_sec": 8, "aspect_ratio": "16:9", } ] }
145
258
51
4,161
1,087,955
Christoph Maurer retweeted
I cracked the code on how to generate the best Veo 3 video with Gemini API. Here are 10 video generate from a single short prompt, automatically improved with Gemini 2.5! No cherry-picks! 🤯🤯 The Secret? A Meta JSON Schema and let Gemini 2.5 generates high quality prompts with it from a simple idea. Script below. prompt: "A high-fashion catwalk taking place in a chaotic children's playroom."
36
71
8
841
195,886
Christoph Maurer retweeted
1/N I’m excited to share that our latest @OpenAI experimental reasoning LLM has achieved a longstanding grand challenge in AI: gold medal-level performance on the world’s most prestigious math competition—the International Math Olympiad (IMO).
390
1,275
796
7,210
5,740,790
Christoph Maurer retweeted
Btw I am not just writing a blog trying to explain ALL of these papers, side by side, I am implementing each of them in PyTorch.
Most influential LLM papers and the ideas they introduced (post 2017) A long thread 🧵
3
17
228
12,998