Mark Sweetnam

Researchers at Oxford argue that LLMs can't invent anything. It's impossible mathematically. The paper is called "Theory Is All You Need." Teppo Felin and Matthias Holweg take the famous "Attention Is All You Need" title and flip it. Their argument is that AI predicts from the past, while humans reason forward into the future, and those are two different kinds of thinking. Start with the numbers. The authors estimate a large language model trains on roughly 13 trillion tokens. A human reading at 150 words a minute would need about 164,000 years to get through that. A child hears around 20,000 words a day and roughly 36.5 million words in their first five years. It's the same task with wildly different data, and the child still ends up with language that goes far beyond anything they heard. Their point is that the model learns which words tend to follow other words. It becomes a mirror of what people have already written. It doesn't build a theory of how the world works, so it can't step outside its training data. The paper's sharpest thought experiment makes this painful. Imagine an LLM trained in 1633 on every scientific text ever written up to that point. Ask it about Galileo and heliocentrism. Thousands of years of geocentric texts would swamp Galileo's ideas, so the model would tell you he's wrong. It would also rate Tycho Brahe's astrology as more credible than the idea that the Earth moves, because more people had written about astrology. Then there's flight. In 1888 the scientist Joseph LeConte looked at bird data, noted that no bird above 50 pounds could fly, and concluded humans couldn't either. Lord Kelvin, then president of the Royal Society, said he had not the smallest molecule of faith in aerial navigation. The New York Times estimated in 1903 that flight was one to ten million years away. Nine weeks later the Wright brothers flew. The Wrights didn't have better data. They had a theory. They broke flight into three problems, lift, propulsion, and steering, built their own wind tunnels, and generated the data that didn't exist yet. Wilbur wrote in 1900 that he had been "afflicted with the belief that flight is possible to man." Every prediction machine on Earth would have told him no. The authors call this the data belief asymmetry. Every real breakthrough starts with someone believing something the existing data says is wrong. A system trained to minimize surprise can't do that by design. They're not anti AI. They say AI will win at routine, repetitive decisions that extrapolate from the past, which is most decisions. They're just pushing back on the idea that you should replace humans with algorithms whenever possible, which is a direct quote from Kahneman. I use these models every day and this matches what I see. The new stuff comes from the human at the keyboard who decides the data is wrong. LLMs don't think, you do!