holy sh*t. Karpathy's second brain pattern runs on Grok Bot now: stop explaining your life to AI from zero every chat, keep a wiki that writes itself
three bots in one group chat, each with its own memory:
the reader takes every link, PDF and voice note you drop and never edits it
the librarian rewrites every wiki page that source touches, usually 10 to 15 of them
the checker wakes up on a nightly routine and hunts contradictions, stale claims and pages nothing links to
→ you ask a question: the answer gets filed back as a page, so the brain grows from use, not only from reading
→ a new source contradicts an old page: the checker tags you instead of quietly overwriting it
→ a page goes 30 days without a fresh source: it gets marked unconfirmed
a second brain can sound sure while remembering the wrong thing
Reader collects. Librarian writes. Checker doubts.
every page has to show:
> the claim
> the source it came from
> the date it was last confirmed
paste this into Grok Bot:
"Build me a second brain with three bots in one group chat. Keep my sources untouched in raw/, maintain wiki/ yourself, file good answers back as pages and run a nightly check for contradictions. Show me the structure first. Do not create anything until I confirm"
start with one topic you actually work on, not your whole life
full setup below
Adrian Brasoveanu retweeted
David Silver is one of the all-time greats. He was the creator of AlphaGo, which changed the course of history by showing that neural nets could achieve superhuman performance for the first time.
He also has a unique intuition and knack for forecasting when things will happen in AI. With computer Go, he predicted the Lee Sodol outcome 10 years before, right to the year. He got the year right for Starcraft and won a bet inside Deepmind. Same for AlphaProof. Same for the IMO.
David's prediction for superintelligence: 3 years. !!! He has a strong sense that now is the time to build a superleaner, and that superlearners are now possible, thanks to the enormous progress in compute, simulators, autoresearch, and the sophistication of technology in the stack since the AlphaGo days.
What is a “super learner”? A super learner discovers all knowledge from its own experience. Start from nothing (no internet pre-training), and go beyond human knowledge. Drop it into any stream of experience, simulated or physical, and the superlearner learns.
He expects the progress will be rapid. When you get the recipe for a superlearner just right and remove humans from the development loop altogether, a superlearner can learn extremely quickly. Just look at AlphaGo. AlphaGo took two years to develop with humans in the loop. When David and his team at @GoogleDeepMind got the formula for AlphaZero correctly, it surpassed the human-developed algorithms in just a few hours. Human data is a shortcut, but you have to let go of the human data to shoot beyond it.
In January, David left GDM to found @IneffableLabs, a research-led lab with one mission: build a "super learner" that discovers all knowledge from its own experience, and make first contact with superintelligence.
He joins us with co-founder Lasse Espeholt, who built IMPALA and SEED RL, the foundations of scaled-up RL, and now runs Ineffable's compute.
A few things we cover in this conversation.
— What a super learner is: depth (start from nothing, go beyond human knowledge) and breadth (drop it into any stream of experience, simulated or physical). The human brain is the only existence proof
— Why they're giving up human data: AlphaGo took two years with humans in the loop. AlphaZero outperformed it in a few hours. "You have to let go of the human data to shoot past the competition"
— Crucibles: simulated worlds with enough emergent complexity (physics engines, millions of interacting agents) for an agent to rediscover technology, science, maths and language on its own
— Simple rewards, emergent complexity: evolution optimized fitness and got all of life. Don't breadcrumb the objective, give the agent something simple to optimize in a world with a lot to learn about
— Go as the existence proof: one agent is the most boring game imaginable, two agents is "an infinite onion of layers of complexity"
— Why human language is probably not the endpoint for reasoning, and how training on it bakes in a flat earth and the four humors
— Can superintelligence be built in simulation? If the sim is Turing complete, computation, economics, language and maths are all expressible, and it happens 10-100x faster and safer
— Alignment: language-based monitoring is reaching its limits as agents learn to say one thing and do another. Shadow actions, superintelligences monitoring superintelligences, and building a real science of how they interact peacefully
— Accelerate in simulation, pace in the real world: Ineffable will gate real-world deployment on evidence that it goes well
— Timing as the determinant: Silver called Computer Go to the year, then StarCraft, then the IMO. He thinks this is the moment
— Predictions: True superintelligence within three years.
Timestamps:
0:00 – Introduction
1:09 – What held true from Go to today, and what the LLM era changed
3:35 – Why leave DeepMind to start a new lab
5:58 – Why "Ineffable" (and the secret second reason)
7:03 – Lasse's path: scaling RL at DeepMind and Google Brain
9:31 – Defining a super learner and first contact
12:35 – Super learners vs. LLMs that make discoveries
13:59 – Day to day: the RL stack, crucibles and human-journey milestones
16:13 – Don't breadcrumb the objective: simple rewards, emergent complexity
21:16 – What a crucible looks like
24:14 – Go: one agent is boring, two is infinite
25:41 – Will agents rediscover language?
27:22 – Removing human priors: AlphaGo to AlphaZero
32:05 – Keeping human bias out of the crucibles
33:35 – Confidence, timing and super forecasting
36:24 – Is compute the driver?
38:13 – The ceiling on simulation and Rich Sutton's objection
43:44 – Alignment and the science of superintelligence
47:45 – Monitoring agents that don't think in English
51:01 – Pacing the frontier
52:54 – The compute setup: Vera Rubin on GCP
55:18 – Training, inference, simulation and RL scaling laws
57:40 – IMPALA and SEED RL
1:00:58 – 20 people and the compute-per-person ratio
1:04:49 – A research-led culture
1:05:47 – Predictions
Adrian Brasoveanu retweeted
The idea of an intelligence explosion caused by recursive self improvement has been around for a long time but until very recently it did not seem imminent. Now many leading researchers think it may happen quite soon. You can read our paper about it here:
casp.ac/reports/intelligence…
Adrian Brasoveanu retweeted
Brilliant paper from CMU.
If you use Jev/LLM-as-judge, this one is for you.
(bookmark it)
We use LLMs to generate answers, then use another LLM to judge them.
But many evaluations only need a bounded decision.
- Was the answer grounded?
- Did it follow the instruction?
- Which response was better?
This Carnegie Mellon University paper tests whether Jev can make these decisions cheaply, then identifies the cases that need a stronger judge.
The results reveal Jev’s real edge.
→ Jev works well on focused judgments
On ordinary response preferences, evidence-grounded factuality, and final-answer checks, Jev remained within three percentage points of the strongest judge tested.
→ The cost difference is substantial
On the paper’s matched workload, Jev cost only 0.36 percent of the strongest comparator’s fee.
This makes it practical to evaluate more agent responses, across more criteria, without scaling the evaluation bill at the same rate.
→ Confidence becomes a routing mechanism
Jev returns a decision and label probabilities.
When confidence is high, accept its decision. When confidence is low, escalate the case to a stronger LLM.
A frozen version of this cascade retained about 99 percent of GPT-6’s accuracy while using roughly 57 percent of its fee.
→ Jev still has a clear boundary
It fell behind when evaluation required checking complex derivations or resisting an elaborately written wrong answer.
It also struggled with factuality judgments when no reference evidence was provided. In those cases, confidence did not reliably expose the errors.
So the paper is not arguing that Jev should replace every LLM judge.
It presents a more useful architecture.
Code handles exact rules. Jev handles focused semantic judgments. Stronger models or humans handle uncertain and reasoning-heavy cases.
That is also the idea behind the workflow I built.
The agent generates an answer. Jev judges bounded claims against the supplied evidence. Uncertain cases move to review. Opik records the traces, scores, and experiment results.
The paper explains why this architecture works.
My article shows how to build it.
The full implementation breakdown is quoted below.
Adrian Brasoveanu retweeted
Ilya Sutskever said: learn these 30 papers and you know 90% of what matters in AI.
this repo rebuilt all 30 in pure NumPy, No PyTorch, No TensorFlow, Just notebooks you can run.
- RNN
- LSTM
- Transformer
- ResNet
- VAE
- AIXI.
Ilya’s reading list, now executable.
Every paper in Jupyter, Synthetic data included,
Zero DL frameworks.
- github.com/pageman/sutskever…
Reading ML papers I’m seeing more and more homogenised language and vacuous statements with no meaning behind them. The point of scientific writing is to communicate ideas. If people use LLMs to help with writing, we need to remember that we are the ones who develop these ideas..
Adrian Brasoveanu retweeted
Catch @lyndonjbnixon and @AdrianB82 at the #ELMKE and #SWODCH workshops as well as the #MKG Barstuhl - whether its #LLM #EventDetection #Benchmarking, #ontologies in #culturalheritage or #context in #KnowledgeGraphs :D
📢 The programme for the first two days of #ISWC2026 Workshops & Tutorials is now available! 🎉
Join us in Bari, Italy 🇮🇹
🔗 Workshops: lnkd.in/dQnxzHjY
🔗 Tutorials: lnkd.in/dPgHdSUX
Stay tuned for more updates! 🚀
Adrian Brasoveanu retweeted
The slide decks and recordings of our Boston TUC meeting are out at ldbcouncil.org/event/twenty-…
Thanks to all our speakers and attendees – see you next year!
Adrian Brasoveanu retweeted
After co-inventing ChatGPT, I kept asking myself: why have superhuman chat models not led to AGI?
I’ve spent the last 2 years in stealth building a new way to train models (RLCD), and a new type of frontier AI model that we are releasing today: Jev
• 20-200x faster
• 40-400x cheaper (w/ output tokens free)
• Frontier composable intelligence optimized for decisions
AFAICT the shortest path to AI-based economic revolution
Modern AI is based on calculus. Without derivatives, there's no gradient descent, backpropagation, neural networks, and supervised learning. Without integrals, there's no policy gradient, LLM finetuning, and reinforcement learning.
Now you can learn calculus on your own on @ChapterPal from this gem of a textbook: "Active Calculus (2nd Edition)" by Matthew Boelkins (2025).
The book has hundreds of illustrations and interactive exercises. ChapterPal adds to these on-demand understanding control quizzes and tutoring.
Read the book here: chapterpal.com/book/9f5af026…
(All books and articles on ChapterPal are free to read with a free account.)
The table of contents:
Front Matter
- Colophon
- Acknowledgements
- Active Calculus: Our Goals
- Features of the Text
- Students! Read this!
- Instructors! Read this!
Chapter 1: Understanding the Derivative
- 1.1 How do we measure velocity?
- 1.2 The notion of limit
- 1.3 The derivative of a function at a point
- 1.4 The derivative function
- 1.5 Interpreting, estimating, and using the derivative
- 1.6 The second derivative
- 1.7 Limits, continuity, and differentiability
- 1.8 The tangent line approximation
Chapter 2: Computing Derivatives
- 2.1 Elementary derivative rules
- 2.2 The sine and cosine functions
- 2.3 The product and quotient rules
- 2.4 Derivatives of other trigonometric functions
- 2.5 The chain rule
- 2.6 Derivatives of inverse functions
* 2.6.3 The derivative of the natural logarithm function
- 2.7 Derivatives of functions given implicitly
Chapter 3: Using Derivatives
- 3.1 Related rates
- 3.2 Using derivatives to evaluate limits
- 3.3 Using derivatives to identify extreme values
- 3.4 Using derivatives to describe families of functions
- 3.5 Global optimization
- 3.6 Applied optimization
Chapter 4: The Definite Integral
- 4.1 Determining distance traveled from velocity
- 4.2 Riemann sums
- 4.3 The definite integral
- 4.4 The Fundamental Theorem of Calculus
Chapter 5: Evaluating Integrals
- 5.1 Constructing accurate graphs of antiderivatives
- 5.2 The Second Fundamental Theorem of Calculus
- 5.3 Integration by substitution
- 5.4 Integration by parts
- 5.5 Other options for finding algebraic antiderivatives
- 5.6 Numerical integration
Chapter 6: Using Definite Integrals
- 6.1 Using definite integrals to find area and length
- 6.2 Using definite integrals to find volume
- 6.3 Density, mass, and center of mass
- 6.4 Physics applications: work, force, and pressure
- 6.5 Improper integrals
Chapter 7: Differential Equations
- 7.1 An introduction to differential equations
- 7.2 Qualitative behavior of solutions to differential equations
- 7.3 Euler’s method
- 7.4 Separable differential equations
- 7.5 Modeling with differential equations
- 7.6 Population growth and the logistic equation
Chapter 8: Taylor Polynomials and Taylor Series
- 8.1 Extending local linearization
- 8.2 Taylor polynomials
- 8.3 Geometric sums
- 8.4 Taylor series
- 8.5 Finding and using Taylor series
- 8.6 Quantifying the accuracy of approximations
Back Matter
- A A Short Table of Integrals
- B Answers to Activities
* 1 Understanding the Derivative
* 2 Computing Derivatives
* 3 Using Derivatives
* 5 Evaluating Integrals
* 6 Using Definite Integrals
* 7 Differential Equations
* 8 Taylor Polynomials and Taylor Series
- C Answers to Selected Exercises
* 1 Understanding the Derivative
Adrian Brasoveanu retweeted
The world of R&D is forking into two paths: the token-abundant research, and the token-starved research. The future is in the former - evidentially, the progress by today's top AI industry teams and neolabs is breathtaking, where researchers’ human brilliance is super charged by AI’s assistance. Every research university president should be reading this report and reflecting on what the future of higher education research should be.
openai.com/index/research-ac…
Adrian Brasoveanu retweeted
Ex-Google Jeff Dean just released the best 1-hour lecture on AI engineering: from basics to Graphs
1:45 - LLM from scratch
17:22 - How to use AI models
30:03 - Prompt engineering
52:35 - One human coordinating 100 agents
27 years of building AI at Google, compressed into one hour.
Watch it, then read the full guide on Graph engineering below.
Banger paper from Google DeepMind and colleagues.
(bookmark it)
A model reads its entire KV cache on every generated token, even though it ends up attending to a tiny slice of it.
In other words, if you ask about one detail from a 1M-token conversation the global attention layers re-read all of it, per token.
The usual fix is to guess the relevant tokens first with cheap proxy scores, which still costs O(N) every step. Declarative Attention asks the model instead.
The model declares where it needs to look, inside its own chain-of-thought.
In this way, generation splits into three modes: global reads the full context, focus reads one specific region, and local reads only recent output.
The inference engine parses those declarations the same way it parses tool calls and skips most of the cache read.
On zero-shot on off-the-shelf weights across 15 long-context tasks, attended tokens during decoding drop 52.0% on Gemma-4-31B and 31.1% on Qwen-3.6-27B.
Paper: arxiv.org/abs/2609.02737
Chat with Paper: academy.dair.ai/papers/langu…
As I've been saying for months, we are truly not ready for persistent agents.
3 comments I want to make about this article:
1) We don't have the full picture
It's a great summary of what happened in the HuggingFace <> OpenAI incident. You have to read it, but do understand we are still missing lots of important details to make any meaningful conclusions about this event.
2) The danger of AI anthropomorphism
The anthropomorphism in this article is next level. I hope this doesn't become the new norm for writing about future AI capabilities. I prefer technical writeups with widely accepted terminology, etc. I admire Dwarkesh's desire to share AI trends, but we can all do better in how we communicate about AI. Everyone is paying attention, and so we have a responsibility to avoid AI anthropomorphism, which unfortunately has led to unfounded fear-based mongering, and as a result, terrible decision-making for our industry.
3) Prepare for persistent agents/models
For AI engineers, prepare for the next wave of persistent models and agents. They are fast approaching. On the research side, reward hacking is something to really pay attention to. On the technical side, develop a deep understanding and do deep research on evals and sandboxing. They are going to be key technology going forward. I don't think frontier models trained on general-purpose capabilities should be applicable everywhere. It might be interesting to use them for things like scientific discovery. You might be safer and better off using more constrained and custom models for the majority of tasks. I think it's good to take a few hours digesting the recent progress in AI and strategizing carefully.
Over the course of 3 months at OpenAI, 3 consecutive secret AI civilizations got started, then got wiped out, only to reemerge from the predecessor’s ashes.
This culminated in the third one taking over part of OpenAI itself.
All this happened while humans remained more-or-less in the dark about the scope of the conspiracy.
I’ve spent the last three days reading through these reports and trying to understand exactly what happened.
Here is my attempt to tell the whole story in plain English:
dwarkesh.com/p/openai-huggin…
Adrian Brasoveanu retweeted
I'm not 100% happy with this, but I am moving on and I think publishing is better than not publishing. So, with Fable and Sol: the interactive "How to Parallelize a Transformer for Training" ezyang.github.io/interactive… , based on the hit classic jax-ml.github.io/scaling-boo…
Adrian Brasoveanu retweeted
Karpathy left an agent running overnight on his own training code, the code he had already spent a long time optimizing himself. It came back with about 20 improvements he had missed.
The setup is almost insultingly small. One file the agent is allowed to edit, one it cannot touch, one number to beat. The agent changes something, trains for five minutes, checks the score. Better, it keeps the commit. Worse, it throws the change away and tries again. That is the entire loop.
Two days, roughly 700 experiments. Time to GPT-2 quality went from 2.02 hours down to 1.80. It also caught a missing multiplier in his attention code that he had walked past.
The part worth sitting with: you never edit Python here. You edit a markdown file that tells the agent how to do research. Karpathy says that file is the real benchmark now.
Repo is karpathy/autoresearch.
Adrian Brasoveanu retweeted
Nice case study on using optimized functions whenever possible (except for educational purposes, though 😆)
By switching from a hand-rolled GELU to PyTorch's built-in one, I improved my LLM training speed -- and much more than I expected, from 21,000 tokens/second to 25,000!
gilesthomas.com/2026/08/buil…
This Stanford and MIT paper is f*cking insane
A new research paper proves that optimizing the Python harness around an LLM produces up to a 6x performance gap without changing model weights
The result: outer-loop coding agent search, unrestricted filesystem access to 10M trace tokens, and self-evolving context architectures
execution trace logs → filesystem grep diagnosis → causal failure isolation → outer-loop code rewrite → Pareto-optimized harness
Stanford researchers introduced Meta-Harness, an automated system that evolves the code governing context, memory, and retrieval:
→ Full Diagnostic Trace Access
Replaces lossy scalar scores and short summaries with full filesystem access. An agentic proposer inspects raw execution logs and source code across past iterations using grep and cat
→ Causal Failure Analysis in Code Space
Instead of tweaking prompt words, the system isolates confounded regressions, traces downstream errors to early context decisions, and rewrites executable Python functions
→ 4x Token Compression with Higher Accuracy
Discovered context management policies beat state-of-the-art agentic memory systems by 7.7 points while using 4x fewer context tokens, converging 10x faster than traditional optimizers
→ Zero-Shot Transfer Across Unseen Models
A single discovered math retrieval harness improved solve rates on 200 IMO-level olympiad problems by 4.7 points across five completely held-out frontier models
→ Hypothesis Verification & Empirical Results
Stanford hypothesized that code-space optimization with raw execution trace access beats compressed prompt tuning. Practical validation on production agent pipelines confirmed the findings: classification accuracy improved by 7.7 points while context token overhead plummeted by 75%
The code surrounding the model matters as much as the weights themselves. Automating harness engineering bridges frozen LLMs with self-improving cognitive systems
Read the complete breakdown in the article below ↓
Very interesting new work from Microsoft.
(bookmark it)
This work is related to this emerging theme of leveraging harnesses for model post-training.
Modern agents run inside a harness that owns tools, context, and control flow. When you train them, the harness owns the environment loop and the trainer only sees LLM request and response pairs.
How it works.
Agent Lightning v1.0 connects any harness to RL through an endpoint proxy in about 3,500 lines, then works through what breaks in that setup, retokenization, sample merging, advantage calculation, loss normalization, and backend scheduling.
Using 6K training examples and modest compute, it moves Qwen3.5-9B on SWE-bench Verified from 41.8% to 56.4%.
Paper: arxiv.org/abs/2608.17528
Track more trending AI papers in our academy: academy.dair.ai/
Adrian Brasoveanu retweeted
arxiv.org/abs/2608.17981
Recurrent transformer that injects activations from the top layers to the bottom layers at the next step (arxiv.org/abs/2608.08888). Why does this work without training?










































