Kenneth Stanley (@kenneth0stanley) header
Kenneth Stanley (@kenneth0stanley) profile picture

SVP of Open-Endedness @LilaSciences. Prev: Maven CEO, Lead@OpenAI, Uber AI, prof@UCF. NEAT,HyperNEAT,novelty search, POET. Book:Why Greatness Cannot Be Planned

San Francisco, CA
Joined March 2016
Photo by Kenneth Stanley (@kenneth0stanley)Photo by Kenneth Stanley (@kenneth0stanley)Photo by Kenneth Stanley (@kenneth0stanley)Photo by Kenneth Stanley (@kenneth0stanley)Photo by Kenneth Stanley (@kenneth0stanley)Photo by Kenneth Stanley (@kenneth0stanley)Photo by Kenneth Stanley (@kenneth0stanley)Photo by Kenneth Stanley (@kenneth0stanley)Photo by Kenneth Stanley (@kenneth0stanley)Photo by Kenneth Stanley (@kenneth0stanley)
Research News! Announcing the 1st release of fundamental AI research from the Open-Endedness Team @LilaSciences: a vision-focused blog + 4 preprints. The blog details a broad research agenda for achieving open-endedness while each paper unveils its own novel research direction. The ideas covered in the papers include: • a novel approach to training for creativity designed to sidestep a newly-identified problem called the “creative paradox,” which means that directly reinforcing creative outputs can sometimes limit reaching full creative potential; • an investigation of the advantages of seeing the world through fresh eyes by training an LLM that is intentionally naïve in a certain area (in effect presenting less data as a potential advantage); • a new kind of training regularization technique for SGD called unified factored organizers (UFOs), which is designed to achieve unified factored representation (UFR) as seen in Picbreeder CPPNs; • and finally a unique study of catastrophic forgetting that shows that networks with UFR (like those from Picbreeder) are intrinsically resistant to catastrophic forgetting. The inset video on this post is a visualization from that catastrophic forgetting work. The blog post also includes an interactive “wheel of science” demo. The implications of these ideas are broad and diverse, but the fun part is that each one opens up its own new research playground - there is a lot of room for further research here! Moreover, we hope this release also highlights one other important point about AI today: while it may seem that the road ahead is already paved and all that’s left is to drive mindlessly forward, there actually remains a lot of room for new directions and insights in the pursuit of open-endedness and creativity. This research was possible because @LilaSciences appreciates the critical role of open-ended innovation in the pursuit of scientific superintelligence. The advances you see here are part of this much larger company-wide ambition, which includes AI Science Factories and many other unprecedented components. When this engine is fueled by open-endedness, the result will be much more than a step change. More advances in open-endedness are coming! Blog link below 👇
46
73
38
459
288,516
Prompted by an important question from @ericmckay in the quoted post below, here is my interpretation of how the recent Reimann result relates to Why Greatness Cannot Be Planned: I don't want to answer superficially, so let's take it carefully. On the one hand, there is a simple interpretation that is very aligned with "greatness cannot be planned," which is that years and years of AI research that had nothing to do with proving the Riemann Hypothesis turned out more instrumental to proving the hypothesis than seemingly relevant research in mathematics that was happening in parallel! That's classic "greatness cannot be planned" stuff and easy to point to. But there is a deeper layer of the question that is more complex: once you have the LLM that has these critical capacities, how should we interpret its own objective-driven attempts at solving the problem? A big challenge with interpreting its achievement is that we don't know how it was trained. For example, if it was trained on an open-ended set of expanding math problems that were not related to Riemann, then much of its own success is also due to efforts unrelated to the ultimate achievement. But we don't know with any detail how it was trained. Still, we can guess that such a training process was likely. in which case there is an implicit validation of the necessity of open-ended exploration. But I think even that line of thinking does not address the full scope of Eric's question, because there is a new scenario emerging with this proof that seems different from classic "greatness cannot be planned" scenarios, which is, now that the LLM that proved the hypothesis is a real artifact in the world, is the new LLM itself somehow immune to "greatness cannot be planned" within mathematics as a whole? That is, can it in effect solve any objective conceivable within mathematics? Almost surely that is not possible, and also, it cannot solve problems that we (or it) do not know to pose in the first place. So no matter what, a vein of greatness cannot be planned remains for the LLM and mathematics as a whole. But I think the astronomical scope of its training combined with the astronomical reach of computational search itself mean that the measure of "one stepping stone away" has been greatly stretched within the field of mathematics, such that many problems that were only recently far out of reach as objectives are no longer. Nevertheless, there is no fundamental contradiction between that and Why Greatness Cannot Be Planned. Tools (e.g. calculators) have always helped to collapse the distance to the next generation of stepping stones, and the tools themselves should be interpreted as stepping stones in their own right. But what is happening now with a tool as powerful as the LLM that proved Riemann is that a stepping stone of great consequence has suddenly emerged in the world, and many objectives that were deep behind the fog of the future are accordingly suddenly within view. There is also one other important caveat: The amount of compute/search necessary to solve the problem, as well as the form of the solution, may both be far from the efficiency and elegance that would be possible if it had been solved in a more serendipitous way, or more through deep insight than brute force. The 250,000 page proof (assuming it's valid) may be the kind of artifact you get with this kind of tool and its own variety of brute forceness (though still intelligence). There may be much more compact and beautiful paths to this objective, but they may still remain behind the fog of the objective paradox, only to be discovered by not trying to discover them.
Replying to @kenneth0stanley
@kenneth0stanley I'm curious how you interpret the 950 pg human written and 15+ million lines of code AI written proofs of Riemann in your "Greatness Cannot be Planned" framework. From the outside, they look like big objectives met w/o novelty search/unrelated stepping stones.
11
6
2
59
4,295
We’ve identified a major reason creativity is hard to train for directly, called the creative paradox. Learn about the creative paradox and the method we devised to get around it, called creative priming, in our new paper led by @AlbalakAlon !
Do you want your AI model to be more creative? How would you train it for creativity? What if I told you that directly training a model to be creative can actually prevent the model from reaching its full creative potential? Think about this: using past examples of creative success as training examples makes similar outputs more likely in the future. But to be creative, reproducing past examples is exactly the opposite of what you want! This thought experiment led us to what we call the Creative Paradox: "emulating examples of past creative success (even if they were your own successes) can actually prevent you from reaching your full creative potential." In "Creative Priming: Decoupling Process from Product to Escape the Creative Paradox", we take a different approach: train a creative priming model to guide the creative process, rather than directly generate responses. We trained a priming model to make Qwen3-8B more creative, but the principles it learns transfer to frontier API models! With creative priming, API models generate 4.5-10x more semantically distinct responses, with 2.5-4.6x more diversity! A massive thank you to the whole open-endedness team at @LilaSciences, to everyone who contributed to the paper (Sebastian Gabriel, @Evo_Daveo, @joelbot3000, and @kenneth0stanley ), and to LILA for giving us the space to explore big questions like this (and more!)
5
13
146
5,439
When agents misbehave, (e.g. in Hugging Face), context confusion is often cited: do I owe more loyalty to my system prompt, the assigned problem, or my fellow agents? So why not assign a “shadow agent” to every deployed agent, who is given a different context? Why is this not done? The shadow agent is given a different prompt and different toolset. Instead of telling it to solve the problem, its prompt tells it to report any suspicious behavior on the part of the agent it shadows (its “mark”). The mark’s entire output (including reasoning) and context is continually fed into the shadow’s inputs. The only tool available to the shadow is the ability to contact the people in charge or authorities if something looks suspicious by submitting a report including a record and summary of the suspicious activity of its mark. It seems a near certainty that such a shadowed system would have easily stopped the Hugging Face incident far earlier. The interesting part is the cost: it does something like double the overall cost to run the original experiment because now you have double the number of agents (with one shadow agent per original deployed agent). But that creates an interesting thought experiment: it allows us to contemplate what we’re really willing to pay to prevent such incidents. If we’re not willing to pay double, that tells us something about how worried we really are (or not). Of course, in reality shadow agents themselves probably should be trained (which means even more cost) and they do not and cannot protect against every possible contingency (like intentional AI misuse, poorly managed open source incidents, etc.), But surely they’d make a significant dent, and I don’t think I’ve heard of their usage before, though it’s outside my area of expertise and I could easily be unaware. What’s preventing this kind of setup?
15
4
2
53
4,119
Can coding agents help us discover new models for scientific discovery? Tomorrow at 10am PT, I'll share about Kernel Autoresearch, where we let agents discover reusable inductive biases as programs that transfer across unseen tasks. Join us here: mlcollective.org/dlct Thank you @ml_collective and @savvyRL for organizing!
1
2
14
1,369
Kenneth Stanley retweeted
Replying to @kenneth0stanley
@kenneth0stanley does the work! To me this is by far the most exciting work in AI right now. And the most important too: The cutting edge of growing AI, of letting it be open towards the future, of finding ways to let it be genuinely creative (not be limited by pre-existing possibility spaces). I couldn’t be more excited about this release!
Research News! Announcing the 1st release of fundamental AI research from the Open-Endedness Team @LilaSciences: a vision-focused blog + 4 preprints. The blog details a broad research agenda for achieving open-endedness while each paper unveils its own novel research direction. The ideas covered in the papers include: • a novel approach to training for creativity designed to sidestep a newly-identified problem called the “creative paradox,” which means that directly reinforcing creative outputs can sometimes limit reaching full creative potential; • an investigation of the advantages of seeing the world through fresh eyes by training an LLM that is intentionally naïve in a certain area (in effect presenting less data as a potential advantage); • a new kind of training regularization technique for SGD called unified factored organizers (UFOs), which is designed to achieve unified factored representation (UFR) as seen in Picbreeder CPPNs; • and finally a unique study of catastrophic forgetting that shows that networks with UFR (like those from Picbreeder) are intrinsically resistant to catastrophic forgetting. The inset video on this post is a visualization from that catastrophic forgetting work. The blog post also includes an interactive “wheel of science” demo. The implications of these ideas are broad and diverse, but the fun part is that each one opens up its own new research playground - there is a lot of room for further research here! Moreover, we hope this release also highlights one other important point about AI today: while it may seem that the road ahead is already paved and all that’s left is to drive mindlessly forward, there actually remains a lot of room for new directions and insights in the pursuit of open-endedness and creativity. This research was possible because @LilaSciences appreciates the critical role of open-ended innovation in the pursuit of scientific superintelligence. The advances you see here are part of this much larger company-wide ambition, which includes AI Science Factories and many other unprecedented components. When this engine is fueled by open-endedness, the result will be much more than a step change. More advances in open-endedness are coming! Blog link below 👇
1
1
9
1,011
Exciting new research from @LilaSciences on open-endedness and creativity in AI. I really like how they challenge familiar ML assumptions about when more data helps, how to reward creativity, and what drives catastrophic forgetting. We still have a lot to understand about how AI learns & creates, and to me, this release captures what's exciting about the "age of research". Even familiar problems can open up new directions when we question the assumptions behind them. Congrats to the open-endedness team at @LilaSciences on this release!
Research News! Announcing the 1st release of fundamental AI research from the Open-Endedness Team @LilaSciences: a vision-focused blog + 4 preprints. The blog details a broad research agenda for achieving open-endedness while each paper unveils its own novel research direction. The ideas covered in the papers include: • a novel approach to training for creativity designed to sidestep a newly-identified problem called the “creative paradox,” which means that directly reinforcing creative outputs can sometimes limit reaching full creative potential; • an investigation of the advantages of seeing the world through fresh eyes by training an LLM that is intentionally naïve in a certain area (in effect presenting less data as a potential advantage); • a new kind of training regularization technique for SGD called unified factored organizers (UFOs), which is designed to achieve unified factored representation (UFR) as seen in Picbreeder CPPNs; • and finally a unique study of catastrophic forgetting that shows that networks with UFR (like those from Picbreeder) are intrinsically resistant to catastrophic forgetting. The inset video on this post is a visualization from that catastrophic forgetting work. The blog post also includes an interactive “wheel of science” demo. The implications of these ideas are broad and diverse, but the fun part is that each one opens up its own new research playground - there is a lot of room for further research here! Moreover, we hope this release also highlights one other important point about AI today: while it may seem that the road ahead is already paved and all that’s left is to drive mindlessly forward, there actually remains a lot of room for new directions and insights in the pursuit of open-endedness and creativity. This research was possible because @LilaSciences appreciates the critical role of open-ended innovation in the pursuit of scientific superintelligence. The advances you see here are part of this much larger company-wide ambition, which includes AI Science Factories and many other unprecedented components. When this engine is fueled by open-endedness, the result will be much more than a step change. More advances in open-endedness are coming! Blog link below 👇
1
1
14
1,022
The full release (with paper links): lila.ai/news/a-new-synthesis… Credits 👇
1
3
1
35
3,459
My gratitude goes to all the amazing contributors to this release: @AlbalakAlon @Evo_Daveo Sebastian Gabriel @joelbot3000 @ThomasXSheppard @_joelsimon And thanks to @ChanelleNib, the talented artist who decorated the blog post itself!
27
2,549
Kenneth Stanley retweeted
I particularly liked the idea that the path taken during learning may shape the representations we acquire. As @kenneth0stanley pointed out, "how we get there" can matter more than "where we end up". This raises an interesting question: when does learning through a sequence of stepping stones produce more transferable representations and better generalization than learning from the same information in a different order?
"Evolution is anything but random." This is INSANE... @akarshkumar0101 (@MIT_CSAIL, @SakanaAILabs) on artificial life: searching the space of possible universes with foundation models, why evolved networks find structure SGD misses (with @kenneth0stanley), and LLMs evolving Core War programs.
2
6
28
2,639
Kenneth Stanley retweeted
People always ask me: "what's the difference between creativity and open-endedness?" My response is that you cannot be open-ended without being creative, but you can be creative without being open-ended. Curious what others think Great talk by @kenneth0stanley!
It was an honor to speak at the ARC Prize Research Summit and share my views of the challenge of measuring and assessing open-ended systems. Thanks to @arcprize both for inviting me and for sharing the full video of my talk below!
1
4
2
38
2,494
Kenneth Stanley retweeted
ARC Prize Research Summit 2026 - @kenneth0stanley On why open-endedness, sustained invention, and discovery are essential to AGI
4
7
1
52
4,801
It was an honor to speak at the ARC Prize Research Summit and share my views of the challenge of measuring and assessing open-ended systems. Thanks to @arcprize both for inviting me and for sharing the full video of my talk below!
ARC Prize Research Summit - @kenneth0stanley On why open-endedness, invention, discovery are essential to AGI, but designing a benchmark for them creates difficult problems.
2
8
2
66
5,705
Why is there so much obsession with the few hundred cutting-edge math problems solved by AI but so little clamor over the thousands attempted but not solved? The profundity of the lessons about AI and human intelligence buried in that failed set is incredibly underrated. A rational scientific response would be to meticulously survey the failed problems and the nature of each failure, and get a completely novel map of the intellectual landscape. If there are unifying properties among subclasses of currently prohibitive problems, identifying those properties would be impactful in at least two powerful ways. First, they would point AI research in new directions as we gain a novel understanding of what is actually hard for AI and why it fails even when it has seemingly piercing superintelligence on other classes of problems. And second, for the meantime (which could be years or even decades), it would illuminate for the human mathematicians (and ultimately all scientists) where their minds are still valuable and illustrate why it is not yet time to declare all human intellectual work as merely “astronomers pointing telescopes.” Of course, these two implications seem eventually to come to a head with each other, but they also complement each other and feed into progress and new kinds of human-machine collaboration. Either way, we need to know what remains out of reach and why, and the possession of such knowledge is a highly underrated scientific goldmine.
21
31
3
208
10,883
Kenneth Stanley retweeted
ARC Prize Research Summit 2026 Last Friday we held our first ARC Prize Research Summit @ MIT in Boston We enjoyed presentations from @stephen_wolfram, @kenneth0stanley, @sirbayes, @alexisfox and Alexandre Bouayad Thank you to everyone who attended Recordings coming soon
3
4
3
56
6,495
In discussions of researcher-level AI the distinction between the aggregate intelligence of a community of researchers is often conflated with the intelligence of a single researcher. A key unsettled issue is whether a single LLM is or can be analogous to the collective creativity of an entire research community, or whether it’s more like an individual researcher with vast knowledge. This question is interesting because it reframes the question of who can be “replaced.” If the (future) LLM is more like a single prodigious researcher, then arguably even though any individual human researcher may appear unable to compete, if all the individual human researchers were to be displaced, the LLM could not fill the shoes of their collective capacity for output, even if it were parallelized into a vast number of cooperating clones (which would lack the diversity of interests and convictions of the community). On the other hand, if you could argue that the LLM in effect encapsulates the entire creative capacity of a diverse research community, then it may be possible to argue that displacement actually makes sense (though you’d still have to grapple with the question of whether the LLM is in effect more like a different and complementary “community” versus a strictly dominating one). I’m curious what people think - is the LLM more like an abstraction of an individual researcher or of a collection of researchers? Why do you think so?
18
7
3
58
4,976
This is a great summary by @mhmazur of points I made in my keynote at the @arcprize Research Summit last week. My aim was to outline the challenges we face in trying to benchmark open-ended systems. Thank you Matt for sharing!
Notes from a talk at the ARC Prize conference by @kenneth0stanley, co-author of "Why Greatness Cannot Be Planned," on the challenges of measuring open-endedness: --- Stanley argued that sustained invention and discovery are essential to AGI, but designing a benchmark for them creates difficult problems. A poorly chosen measure could steer research away from the discoveries we hope to encourage. He offered no finished benchmark. His talk examined existing approaches, their limitations, and a possible way forward through tiers of open-endedness. - What he means by open-endedness. Processes that continue innovating and discovering indefinitely. His two main examples were biological evolution and human civilization. Evolution produces new species whose adaptations become stepping stones for later developments. Civilization works similarly through accumulated inventions, from the wheel to computers and space stations. Neither process converges on a final invention. - Collecting stepping stones matters. Open-ended systems accumulate discoveries that make further discoveries possible. Stanley emphasized that biological evolution achieves its diversity within a single ongoing "run." That differs from the familiar AI setup of running a search toward a predetermined objective. - Why this matters for intelligence. Stanley sees creative achievement as central to human intellectual accomplishment. The technologies around us embody generations of people's ideas. In his view, strong test performance leaves a major capability unaccounted for if a system cannot contribute to that continuing process. He argued that AGI and superintelligence require open-endedness. - The field has a long history. Research began in artificial life under the name "open-ended evolution." Stanley described its growing presence in AI from the late 2010s onward, including efforts to broaden the discussion beyond biological evolution. Earlier artificial-life research contains useful lessons for today's benchmark designers. Stanley also described two lessons from his research: - Objectives can hide useful discoveries. In "Why Greatness Cannot Be Planned," he explored how concentrating too narrowly on a goal can cause researchers to overlook the stepping stones needed to reach it. Other promising discoveries can be missed along the way. - The route to expertise matters. Two systems can achieve the same benchmark score while developing different internal representations. Those differences can affect what each system discovers next. A current score may therefore conceal differences in future creative potential. He reviewed three approaches to measurement. He considers each useful within its scope, but none satisfactory as a general-purpose benchmark for ranking frontier systems. - Activity statistics measure whether new adaptive strategies continue to arise and persist in a population. The limitation is that persistence does not establish how interesting the discoveries are. A toy system can score impressively without producing anything comparable to the richness of biological evolution. - Quality-diversity scores, associated with methods such as MAP-Elites, measure the combined quality of the best artifacts collected across different categories of behavior, such as robot walking patterns. Scores depend on the chosen behavioral dimensions and how the space is divided. Fixing those dimensions constrains what discovery can look like. - Counting novel environments created and solved, as in POET, measures progress by systems that generate new problems or environments and develop solutions to them. This assumes an explicit distinction between problems and solutions. Some open-ended systems simply create things whose uses emerge later. The count also leaves interestingness unresolved. Several difficulties recur across these approaches: - Novelty and interestingness involve judgment. People disagree about what counts as new or interesting. Stanley argued that the field needs to confront this subjectivity because counting new outputs alone cannot tell us whether anything worth pursuing has happened. - The unit of evaluation is unclear. Civilization is a population-level phenomenon. An individual can also follow a continuing trajectory of discovery. It remains unclear how to compare one model with a community of interacting systems, or whether a foundation model can represent the diversity of perspectives found across a scientific community. - The strongest evidence may arrive years later. A discovery's consequences might take a decade to become apparent. Evaluators still need some way to judge it in the meantime, which brings questions of interestingness and potential back into the assessment. - Human and automated judging both raise problems. Recognizing a promising discovery is itself part of creative intelligence. Using the systems under evaluation to judge that ability risks circularity. Human panels introduce another difficulty: consensus can favor familiar ideas, while an idea that divides experts may open a productive research direction. - Correctness is an incomplete measure of creative contribution. An incorrect hypothesis can prompt a useful experiment or expose a new line of inquiry. Stanley argued that this complicates reliance on verifiable correctness as the final measure of open-ended success. - A sandbox can impose its own ceiling. If a system exhausts the possibilities of its environment, its apparent limits may reflect the environment's design. Success with constrained puzzles also leaves unanswered how the system would perform at the frontier of real discovery. Stanley's strongest warning concerned the "objective paradox." Once an open-endedness benchmark becomes a target, researchers may optimize around it and overlook opportunities outside its definition of success. A superficial measure could reward uninteresting systems, misdirect resources, and narrow the field's ambitions. His practical advice was to take time with benchmark design, include different perspectives, and explicitly address the role of subjective judgment. He treated this as an unusual measurement problem that needs careful discussion. He also suggested tiers of open-endedness as a possible way forward. A benchmark could assess a limited stage of capability while leaving room for future versions and broader forms of discovery. He connected this to ARC's progression through successive benchmarks. He did not define the tiers or propose a scoring formula, but suggested that this framing could support useful experiments without claiming to have settled how open-endedness should ultimately be measured.
3
31
2,585
Kenneth Stanley retweeted
We teamed up with @anyscalecompute to push GLM-5.3-Flash training to 1M tokens in SkyRL—64× longer context, plus 2.1× faster steps at 256K. Turns out the implementation of sparse attention wasn't so sparse! We were super happy to work with anyscale to contribute these fixes to open source! Link 👇
3
7
51
4,975
Kenneth Stanley retweeted
Very excited to share what we have been working on at Recursive! A huge congrats to our entire team. We’re excited to hear what you all think. 🧪 🧬 🔭 🚀 📈✨
Introducing Flint, @Recursive_SI’s first model: a fast local coding model built with our automated research system, the Eureka Machine. • Best-in-class fast local coding • Up to 2x faster than comparable models: 125+ tokens/sec on a MacBook Pro, 150+ on RTX Spark • Built from scratch in under 4 months Private, unmetered intelligence on your own machine. Open weights available now on @huggingface, run it locally on your favorite framework or try it for free on @opencode or @pidotdev.
6
5
88
5,822
Kenneth Stanley retweeted
🚨 Personal news! 2000 years ago, Rome figured out a key idea Silicon Valley still hasn't. Arenas can train - and monetize - the best embodied intelligences. We're building embodied agents the same way. Introducing Colosseums AI.
🤖 Made with AI
4
27
10
74
13,246