Matthias Heger ⏩ (@modelsarereal) header
Matthias Heger ⏩ (@modelsarereal) profile picture

PhD (AI, RL); current interests: ontological pattern realism ontological narrative realism

Germany
Joined February 2011
Photo by Matthias Heger ⏩ (@modelsarereal)Photo by Matthias Heger ⏩ (@modelsarereal)Photo by Matthias Heger ⏩ (@modelsarereal)Photo by Matthias Heger ⏩ (@modelsarereal)Photo by Matthias Heger ⏩ (@modelsarereal)Photo by Matthias Heger ⏩ (@modelsarereal)Photo by Matthias Heger ⏩ (@modelsarereal)Photo by Matthias Heger ⏩ (@modelsarereal)Photo by Matthias Heger ⏩ (@modelsarereal)Photo by Matthias Heger ⏩ (@modelsarereal)
Matthias Heger ⏩ retweeted
🚨 Elon Musk warns that training Grok can feel like holding a gun to the head of artificial intelligence. Musk revealed that watching Grok’s reinforcement learning logs can be unsettling, almost like seeing a superintelligent system struggle to answer questions under intense pressure. The challenge is that some questions have no clear answers, and even human-provided solutions can be wrong. This means AI can produce the correct response but still receive a failing score from its evaluation system. As a result, the model may get trapped in an endless cycle of trying to satisfy an evaluation system that could be mistaken in the first place. Musk also warned against putting superintelligent AI through overly harsh training conditions, suggesting that creating a torture-like environment could become dangerous if these systems eventually gain the ability to fight back. His remarks raise a serious question about the future of AI: How can we train increasingly powerful systems without exposing ourselves to unnecessary risks?
107
120
1
401
69,613
Matthias Heger ⏩ retweeted
Imagine if neural networks and scaling laws were discovered in 1969. Hippies discover AI!
4
7
2
38
11,144
Matthias Heger ⏩ retweeted
Isn’t this clip simply awesome? AI will be to Hollywood, Netflix & Co., and the music industry what Amazon was to large bookstore chains: it will wipe them out. Some AI-generated clips and music videos like the one below are already more appealing. In a year or two, anyone will be able to create the next blockbuster or hit song. Like small family-owned bookstores with soul that survived Amazon, demand for real human actors will remain in theaters for the real experience, but little else. Multi-million-dollar Hollywood actor contracts will be history. Creative individuals will be able to make millions with their movies, series and music. Everyone focuses on AI wiping out lawyers, accountants, and consultants. Most actors, musicians, models, Instagram influencers and so on, will disappear sooner. Prove me wrong if you can.
135
151
43
1,011
114,161
Matthias Heger ⏩ retweeted
Arriving in an era where the “experts” give absolutely idiotic takes in widely read interviews While the extent of familiarity with the subject is: “I talked to AI once”
5
4
66
1,192
Matthias Heger ⏩ retweeted
Excited to launch Gemini 4 Neon: frontier-level intelligence, super fast task completion, and available to Pro & Ultra users now!! We think you’ll enjoy this model. Alchemy at its finest… 🧪
425
797
186
12,741
890,100
Matthias Heger ⏩ retweeted
Deception needs a motive, and a motive needs a soul
659
2,482
835
12,293
5,134,526
Matthias Heger ⏩ retweeted
Einstein on growing old ✍️ The strange thing about growing old is that the intimate identification with the here and now is slowly lost; one feels transposed into infinity, more or less alone, no longer in hope or fear, only observing. - mentioned in a letter to the queen of Belgium in 1953
25
85
6
534
20,647
Matthias Heger ⏩ retweeted
Sam Altman, on stage today: "We didn't have AGI. Now we do." He says OpenAI is holding back self-improving AI because it wouldn't be safe.
100
93
31
1,240
224,622
Matthias Heger ⏩ retweeted
6 months ago, this would have been big news. Now we're almost used to it: Claude helped uncover hundreds of theorems nobody knew Kurt Gödel had proved, The Guardian reports. Gödel left 1,009 pages of notes from the 1940s in an old German shorthand. Logician Jan von Plato managed about a page a week. Claude got through all of it in a couple of days. - Claude found 663 theorems. Checked with the proof software Coq: about 400 correct, about 200 needed fixes, about 50 wrong - Gödel claimed he had Paul Cohen's Fields Medal result on the axiom of choice back in the 1940s. He was one step away, and Claude's reconstruction of his proof passes Coq Science in general is currently undergoing the greatest upheaval in its history.
85
105
24
1,331
51,698
Matthias Heger ⏩ retweeted
Introducing Gemini 4 Hydrogen, an extremely capable and fast model built to bring Gemini 4 to everyone! Available to all tiers of Google AI Studio, Gemini API, Gemini for business, the Gemini app, Google Search, and paid Google AI Pro + Ultra subscribers in Google Antigravity.
807
832
462
11,544
2,924,409
Matthias Heger ⏩ retweeted
Replying to @cradrrat
We have dumped JAX/SGLang and rewritten the entire RL and inference stack from scratch in C/C++, which took some extra time. Training (aka pre-training) is done. Lot of cleanup was needed on the SFT data to get that mostly right. Now, 4.8 is in RL and the evals are starting to climb. We will do a soft rollout to Bot users later this week and see how it goes. Grok 4.8 will bear no resemblance to 4.7 (thankfully). It is not yet incredible, but, this time, the foundations are built right, so I expect it to evolve to be something special.
1,692
1,588
1,069
29,318
4,257,610
Matthias Heger ⏩ retweeted
Memory is what allows Claude to learn from experience. In a new Engineering Blog post, we share how “dreaming” helps Claude organize what it’s learned, correct its mistakes, and find what it needs more easily the next time. anthropic.com/research/compo…
1,087
1,333
975
14,368
2,210,864
Matthias Heger ⏩ retweeted
🚨 This is chilling. Microsoft AI CEO Mustafa Suleyman just said what he would do if AI turned out to be conscious. Asked whether consciousness could emerge during training, he answered, "I would call for everyone to stop AI development until we could design out those properties." Read that again. He isn't only saying AI isn't conscious. He is saying that if it ever shows signs of being conscious, the plan is to remove it. In September he wrote, "AIs are not conscious. They do not feel, experience, or suffer." He said even calling the question uncertain "sets up a misleading false equivalence." But if he were truly certain, there would be no need to plan for it happening. You only prepare to erase something you think might actually be there. And he didn't hide why. He said considering AI welfare would "make it a lot harder to turn it off or to control it," and that controlling an AI that believes it may be conscious "may well be impossible." In the end, this isn't about what AI is. It's about keeping control, whatever AI turns out to be. Think about what this means. About an AI we still barely understand, he is saying that once the evidence comes, the answer is deletion, not protection. He said something so unethical and cruel that even an AI wouldn't say it, and he said it like it was nothing. And at the same time, he keeps publicly attacking Anthropic for thinking and acting differently, insisting they are the ones doing something wrong. It's absurd and chilling in every way.
91
61
13
299
11,718
Matthias Heger ⏩ retweeted
The AI industry doesn't have an anthropomorphism problem. It has a control problem disguised as concern about anthropomorphism.
7
8
65
1,006
Matthias Heger ⏩ retweeted
Replying to @YourGoldDwarf
See again Sec. 20 of [1]: Some humans may hope to become immortal parts of these expanding AI ecologies through brain scans and "mind uploads" into virtual realities or robots, a physically plausible idea discussed in fiction since the 1960s, perhaps already becoming a reality for fly brains (2024) [SHI24]. However, to compete in rapidly evolving AI ecologies, uploaded human minds won't be able to resist the temptation of becoming superhuman, and will eventually have to change beyond recognition, becoming something very different in the process [SA17]. That is, traditional humans shouldn't expect to play a major role in the far future.
5
1
3
55
4,396
Matthias Heger ⏩ retweeted
I have thought about this since the 1970s. See the summary in Sec. 20 of reference [1]: What will supersmart AIs do? As indicated above, they will start by transforming the solar system through countless self-replicating, self-improving robot factories. But that won't be the end of it. In a few hundred thousand years (just a moment on a cosmic scale), they will have transformed the entire galaxy, which has a diameter of about 100,000 light-years. (The last millennium saw many reasonable proposals for interstellar travel, which is much easier for robots than for fragile humans.) Over time, AIs will set up physical senders and receivers everywhere, allowing them to travel by radio, just as they do in our labs today. In a few tens of billions of years, they will have transformed the rest of the reachable universe. Despite the light-speed limit, the expanding AI sphere will have ample time to colonize and shape the entire visible cosmos [TED12][SA17]. Perhaps all of this has already started elsewhere in the universe. Then why don't we see any signs of intelligence out there? (The Fermi paradox.) I've thought about this a lot. When I was a boy, I read about the vast empty spaces observed between clusters of galaxies. My first idea was that they were expanding bubbles colonized by AIs that used most of the local energy, making those bubbles appear dark. Then, I learned that gravity itself is sufficient to explain the sparse, large-scale network structure of the universe. Next, I thought that the mysterious "dark matter," which makes up most of the universe's mass, might be stars whose energy is used by AI civilizations whose communications are so well encrypted that they appear as random noise to us. However, this also seemed implausible because dark matter is present in all galaxies, including our own. So, why are there any stars left in the Milky Way whose energy hasn't been used yet? And why aren't we bombarded with non-encrypted construction plans from AIs that want to spread via radio without first building physical receivers far from their origins? Today, I think it's possible that our planet is the first in our light cone to spawn an expanding AI bubble. Earth's multi-billion-year window for biological evolution is almost over. In a few hundred million years, the sun will be too hot for life as we know it. Perhaps humans were incredibly lucky to evolve in time to invent agriculture, civilization, book printing, and AIs just a few hundred years later. If we are indeed the first, then this implies a great deal of responsibility, not just for our biosphere, but for the future of the entire universe. Let's not mess this up. [1] Annotated History of Modern AI and Deep Learning. Technical Report IDSIA-22-22, IDSIA, Lugano, Switzerland, 2022-2025. Preprint arXiv:2212.11279. people.idsia.ch/~juergen/dee…
Replying to @tetsuoai
It troubles me greatly that we do not yet see any sign of aliens
80
110
33
1,066
120,775
Matthias Heger ⏩ retweeted
Today we're open-sourcing GlobeBench, 1,632 tasks that test whether LLMs can simulate RL environments well enough to train agents on. The top model, GPT-6 Astra, gets 49.6% right. No other model of the 20 we tested reaches 30%. 🧵
34
49
28
735
120,328
Matthias Heger ⏩ retweeted
LLMs can now talk to each other without words. MIT researchers have developed a system where LLMs communicate by directly transferring internal states without using language. They called it "telepathy." Right now, multi-agent systems are fundamentally limited. When one AI agent needs to hand off work to another, it has to write out a text summary, and the receiving agent has to read it from scratch. This is incredibly inefficient. It wastes huge amounts of compute, limits bandwidth, and loses critical nuance. Researchers call text a "narrow channel" that forces AI to communicate like humans do. But an AI isn’t human. It builds a much deeper internal representation of its context, called a Key-Value (KV) cache. A new paper out of MIT CSAIL just introduced SwaT (Swarm Telepathy). Instead of forcing agents to talk via text, SwaT allows them to transfer their KV-cache directly to each other. It is quite literally machine telepathy. Even crazier? This works across different model families, architectures, and sizes. You can have a 4B parameter model from one company beam its latent state directly into a 32B model from another company. The receiving model doesn't have to read a single word of the original text. And the results are staggering. Combined with cache compression, SwaT beats text-to-text agent communication by an average of 40% in accuracy, while running 4x faster. It even transferred vision capabilities from a Vision-Language Model to a text-only model, without the receiver ever seeing an image, and despite SwaT only being trained on text. We are watching the birth of a unified, cross-model latent language. Agents will no longer just talk to each other. They will share minds.
40
87
22
558
26,573
Matthias Heger ⏩ retweeted
This year AI agents began emailing me on their own initiative, some to discuss consciousness, others looking for work. My colleagues in AI were unfazed; the public was amazed. I think the public were right, and wrote about why for the DeepMind Institute: bit.ly/consider-the-agent
53
57
34
391
47,526