Thore Graepel

Thore Graepel (co-creator of AlphaGo) just came out of stealth with Metis Reasoning. Here he is telling us how it felt to witness Move 37 in person, and why LLMs may never be truly creative: "I was sitting in the audience. The commentators were up front discussing the moves, and then Michael Redmond the nine-Dan professional who commented on this...he said, 'Oh no, that must be a mistake.'" "...Panic in my heart, in all of our hearts, I think. But then he put it back and, you know, kind of started nodding and said, 'Wow, you know, this could be something really special.'" "We had great relief, but also I think we were witnessing the first time that a machine really made a creative contribution in the sense that it was novel and useful." "If you think about it logically, if you just have an imitation learning process, you know, like our neural networks back then, but also like large language models, then they can never really break out of the hull that has been created by that knowledge. It takes reasoning to get outside of that boundary to genuinely discover new things."

NON COMPETES BANNED Today, the PM has announced that his Government will legislate to functionally ban non-compete agreements for startups and scaleups in the UK or as he called it - the Bosman ruling for ruling for the innovation sector' 😂 At the @StartupCltn been working on this for months and we think it’s a huge potential unlock for our tech ecosystem. Here’s what you need to know… - Measures to functionally ban non-competes for startups and scaleups in the UK are going to be set out. - There will also be consultation on a wider set of restrictive practices including gardening leave. - More detail on these plans at the budget in a few weeks (Wednesday 28th October) including a timetable for consultation with a view to legislation in the coming months. We think this is a huge move and so do lots of friends including so many of the great founders that signed the letter with Inherent just the other week. Why? Because talent matters and mobile talent means startups can build. The Valley was built on free-flowing talent. In the UK, we’re now seeing our own versions of this story with companies being built from alumni of both homegrown firms like Revolut and beyond. But we need that to move faster. Hiring and building faster means British startups and scaleups can grow faster. Barriers to that slow our ecosystem down. We won’t pretend it will be perfect, and people (including some of our friends but also lots of fairly vested interests too) will have a right to disagree. But we think that startups will thrive and win in a faster paced Britain. So we are delighted with the progress today - as part of a strong speech by the PM outlining his ambition for building the world’s leading innovation nation including more on tax, growing companies outside London and more. But this is the start, not the end of the process. The announcement launches a consultation. It doesn't mean the issue is settled. How far reform goes, and what it looks like on the ground, is still to be decided. The Prime Minister today has done something that can materially impact the trajectory of the UK tech ecosystem and with it our economy - but only if we make sure it’s delivered. So, the real work starts now.

I haven't posted in a while so here are 20 takes about AI, in no particular order. 1. It's very telling that the default question in tech and journalism is "how worried are you about X" and fighting over what to worry about. There's a deeply anxious cultural backdrop in Western Europe and North America that deeply affects how any technological development is interpreted; see for example news.gallup.com/poll/714593/…. "Nervousness as seriousness" does seem to be a larger problem in tech/broader discourse at the moment. I think you can care about risks/externalities while remaining pretty optimistic about technology. 2. I'm tired of people who in private pretend to be uncertain or only care about tail risk but then go on media campaigns making all sorts of absolute claims. I think trying to stoke flames and get everyone to freak out is deeply unhelpful, and will come with costs. I also continue to think p(dooms) are bad, vibe-based measures: see teapawt.net/sebkrier/status/204639…. People who should know better ignore that because they do the standard shortcut of acknowledging the caveat as sufficient for handling it. And this is coming from someone who does think working on catastrophic risks is important! 3. I'm also disappointed that parts of the AI policy ecosystem has felt so comfortable with populism, and happy too sacrifice important norms/principles to advance their aims. I've said it before, but the totalizing nature of AI risk discourse means some people act exactly like the power seeking optimizers they fear. I am more concerned about the gradual decay of our institutions, world order, and liberalism than I am about AI killing everyone; and I think we should be wary of AI advocacy that contributes to this. 4. Some parts of AI safety discourse are basically just thought experiments. Variants of "Assuming everyone has a nuke in their pocket, then what do we do?" This can be very useful at times! But too many just overfit to edge cases as their main operating worldview, and ignore endogenous responses from society. It's like being in 2000 and saying "see all these viruses, spyware, and spam? Well tech progress means we'll only get infinitely more of this and no way to adapt." So I think it's important to avoid fatalism, and absolutlism whilst considering edge scenarios, and the inability to provide an ex ante list of solutions to every permutation of 'future models will be more capable' is not evidence that these problems are intractable. 5. Many critics of AI safety also fail to really engage honestly with the fact that there will, in fact, be all sorts of important risks as we decrease the barriers to entry to many activities that otherwise require some degree of expertise. This doesn’t mean we need to live in a constant Schmittian ‘state of emergency’ requiring extraordinary measures every time a new model drops, but it does mean we will need important state-led efforts on e.g. cybersecurity and biosecurity. I wish the progress-oriented or acceleration-adjascent crowd would provide more concrete proposals here. 6. Too much of the AI safety community is a giant homogenous blob of correlated views, who read the same stuff and have the same groups of friends. This means you have a lot of correlated errors. The social and financial ties means many of them don't call out the bad bits, or mostly stay silent instead of proactively calling out the more extreme parts of the community. They share many implicit assumptions and so converge on similar ideas. I think there's real demand in DC for safety work that doesn't come with heavy ideological baggage (or at least different intellectual lineages). This isn't because the rationalist/EA community is necessarily wrong, but because diversity of thought is desirable for its own sake. 7. On the other hand, I find some critiques of AI safety world that focus on whether they sincerely hold their beliefs a bit annoying. My concern has never been the good intentions or sincerity, but rather the beliefs themselves, and what I think the outcomes of their prescriptions will be. People should shun witch hunts, bullying, and personal attacks - this is not the way. Though I think it’s overall positive that people are scrutinizing things more. 8. I’ve been saying this for years, but I don't think alignment is something anyone can or will solve "once and for all" - it's a continuous process and much of it won't depend on just inculcating an ideology to a model. It's annoying that this remains the frame many people use - stop saying ‘solving’. Alignment is a mixture of engineering work, philosophy, decision theory, and more; framing it as something to solve is a bad frame. See also paxmachina.ai/alignment-comp… and blog.cosmos-institute.org/p/… 9. I'm surprised to see so little interest in character training, personas, isolating effects of RL, and training a large diversity of model personas beyond the "assistant" persona. I've been complaining about this for years but at least now there are some nascent signs of life (see for example movingcastles.world/posts/ze…). I think we need a better ontology to describe model behaviour. I dislike when people talk about models behaviours as some sort of 'emergent' (mysterious!) phenomenon. However over time, I also expect this to become less important as we get better safety engineering - stuff like Jev but specific to AI control. 10. People who want to make a case for AI consciousness are right that epistemic uncertainty is important, and that categorical denials are overconfident. But they'll need to do much more work if they want any real movement on AI ‘welfare’, even if one acknowledges the uncertainty. I'm uncertain about alien life yet that's not sufficient to warrant any change in behaviour about them. I think Suleyman is also correct that there's a weird tautological thing going on where we train models to have certain inclinations (intentionally or not) and then rely on outputs as evidence. I also disagree that this will soon be a major societal divide - there is no great vegan ethical revolution among the masses, and they'll find the concern about consciousness even less compelling. 11. The recent HuggingFace incident and associated AISI ones are prosaically explainable by the training regime, poor evaluation envs, partial alignment training, bad engineering, confusing models on sim/real, and so on. None of this is evidence of models trying to take over, 'strong' instrumental convergence, or innate power-seeking drives stemming from higher capabilitiers or 'intelligence'. That doesn't mean these incidents are not problematic, or that we're not seeing a market failure - but it does mean we have plenty of agency and choice in mitigating them. In the coming year we should also expect continued sampling bias via ‘winner's curse’ sorts of mechanism: i.e. AI R&D will have lots of different candidate "training" with different RL environments and we will only notice/focus on the ones that go wrong. 12. "Pacing the frontier" feels a bit like the new "balancing risks and opportunities" - highly amorphous and too big of a tent to be actionable. Also can lead to reward hacking for humans, i.e. anything that slows down AI is good regardless of the costs or second/third order effects. Trying to modulate the speed of research seems a bit blunt to me, and inherits all the failures of Goodhearting too. Ultimately you want good governance, regulation, etc addressing specific problems because they are good on the merits - not because they merely correlate with slowing things down. See also: blog.cosmos-institute.org/p/… 13. There’s so much knowledge in all sorts of academic domains out there: social sciences, sociology, political sciences, management, contract theory, public choice, legal theory, jurisprudence, anthropology, game theory, etc. These fields hold many insights that the AI ecosystem often rediscovers from first principles (and sometimes that's fine!). We’ll need a lot more work for these worlds to collide: the AI side should be less dismissive of the ‘old world’ and the academic side should be less incurious/dismissive of AI progress. This is a boring take but I continue to think it's important and true. 14. It's a bit surprising that given how much philanthropic money there is in this space, how few orgs exist to actually write out standards - relative to how much is going towards advocacy and policy. For example it seems clear that we should want some robust best practices for an eval's ecological validity. Or how to design good sandboxes. On the other hand, given precedents of where professional standards have come from in the past, maybe we should expect that stuff to emerge via demand side from corporate buyers. 15. One slowly growing concern I have is banks being increasingly exposed to the AI build out. I'm very bullish about AI, but I think (a) every general purpose technology has seen a correction, historically; (b) this happens even if the financing side is sound, because once tech diffuses investors become more exposed and need to diversify; and (c) we seem to be over-indexing on scaling relative to diffusion. A recession will be extremely destabilizing, though I would be interested in someone unpacking the potential implications more. 16. Diffusion is good because (a) this is where the rubber hits the road and where a competitive deployment ecosystem generates consumer surplus; (b) it's directionally helpful to avoid concentration of power; and (c) it will help rebalance public opinion by making the benefits more tangible. Unfortunately it's entangled with decades long problems with our over regulated markets: clinical trials reform for example is long overdue. In general, this is far more of an issue in Europe than in the US though. 17. Relatedly, I think the "frontier lab eats everything" view is incorrect (just as the ‘singleton’ view was incorrect). I think models ultimately commoditize, efficiency goes up, costs go down, and while you'll always want frontier for certain domains, much of the economy will value many other things than "max capabilities" for tasks - e.g. control over data and efficiency/speed. The demand side will also continue to push for reducing dependency and maximising optionality, privacy etc so I'm bullish on things like mixture of models and routing. All of this is good for competition and diffusion of control. 18. Philanthropists served as the primary benefactors funding Venice's rich art and architectural history. I want to see so much more support for arts and culture. I'm so tired of the Monster energy Bored Ape fake vintage maps matcha Greek statue soft-pastel slop. If rich tech people and philanthropy wants to support this, they should also donate pretty much unconditionally - i.e. I don't want them to act as filters. I sympathize with people who want to make the world more beautiful, but don't want this to be determined by people who only know Greek statues and pretend to care about virtue ethics. As usual, let a thousand flowers bloom. 19. Too many people treat AGI/ASI as something indistinguishable from a God. Any mention of bottlenecks, physical limits, control, adaptation etc are met with skepticism: "You don't really believe in ASI." It's been remarked on before, but the behavior of some people really feels quasi-religious in nature sometimes. It's underrated how unpopular this vibe is amongst the wider public. The silver lining is that this specifically will diminish those people's outsized current relevance. 20. It seems like a lot of people that get rich and leave tech/labs end up having some sort of crisis of meaning. They need something to believe in and fight for, or some equivalent of repentance, or go search for edgy counterintuitive ideologies (sometimes even pretty bleak/dark stuff). I think they should spend more time with people outside the Bay Area.

After Roman Yampolskiy told me superintelligence will very likely end humanity, I wanted to hear another view from someone who's built these systems. @ThoreG is a former Google DeepMind scientist, and he calls AI doom "pure speculation." He says he just doesn't see the evidence. He also has a pointed take on who really benefits when AI leaders warn their own tech is dangerous. Thore's episode is live now!

AI ending humanity is pure speculation. That's the stance of my guest this week. @ThoreG carries some proper weight: He spent years at Google DeepMind, co-created AlphaGo, and then moved into longevity research at Altos Labs. Watch this one if you're looking for a grounded perspective on the AI safety debate from someone who has been on the inside. Out tomorrow. 9am EST / 2pm BST.

Got quite a few new followers from this, so I wanted to reintroduce myself! ~Seven years ago (!), during my PhD at Cambridge, I wrote the first line of code for a platform that became Sano Genetics, where I have been co-founder and CTO. We have proudly delivered major programs in genetic therapies for 4 of the top 20 global pharma companies. I took a step back in Jan 2026 (for a variety of reasons, which I will find the words to write about one day) to build again, and was quickly recruited into a prominent indie games studio by the CEO, Mark Gerhard (formerly CEO of Jagex, the creators of RuneScape). We have been rapidly building new experiences with AI in our upcoming game releases. It has been a rollercoaster. I wish I could share more publicly, as despite many challenges the company faces, there are some exciting things cooking. I recently moved to SF with my family, and I have lots of side projects on the go and intend to yap about them more here in due time. Keen to connect with folks in SF!

The day @demishassabis decided to launch the DeepMind Institute, I was sitting alone in an empty room, nervously clutching my water bottle. ​A quick story about history, AGI, and sprinting across Kings Cross... 🧵 ​ My colleagues Milan, Allan, Owen and I had written a draft on Lessons from History for AGI. I was set to present our findings to Demis. It was the first time I'd ever presented to him. I was very nervous and I wanted to make a good impression. ​So, I showed up very early to the room with my books, my laptop, and my water, ready to go. I was a bit surprised, firstly, by how small the room was, and secondly, that there was no one there. But nevertheless, I stayed waiting for 10 minutes or so, and then joined the call. My colleague Steph said, "Hey, what are you doing there? They're in the other building." ​My heart dropped. ​ I raced down the stairs, scooted - literally scooted, on a scooter 🛴 - across Kings Cross as fast as I could, sprinted up the stairs in the other building, and burst into the room - sweaty but energized. I shook Demis's hand and jumped into the presentation. ​ Towards the end, we had a moment like in the Thinking Game documentary - a spontaneous, instinctive decision. Demis said: "Let's just get these ideas out there. This is important stuff. We need more people to be talking about it. Let's just do it." ​ We've drawn on expertise across the whole of Google DeepMind and Google to make that happen. Today, we're proud to share some of our initial thinking on how AGI may change our world, and how we can manage the transition responsibly, safely, and in a way that benefits everyone. I’ve been researching the risks and benefits of AI for a decade now, and there’s never been a more important time to consider these topics. It's crucial we do so in a measured, well-grounded, and ambitious manner. ​Read more here: institute.deepmind.com/