haytam

In this interview, Dwarkesh asked Ilya about his "great research taste." How is it that whatever he researches leads to breakthroughs in AI? "What guides me is an aesthetic of how AI should be by thinking about how people are." It's a very top-down approach. We kind of know what intelligence looks like. There's a bunch of neurons in the brain, all energized and interconnected; it thus makes sense to try building a neural network. "It feels right." Then you want a local rule capable of changing connections between neurons. The brain seems to operate that way, so you want to emulate that. He started from a picture of how intelligence might be and worked backwards. Experiments and research are designed to align with that picture. That’s the difference between holding a truth and following a proxy. Something similar happened with Einstein. He never fully bought into quantum mechanics because of its probabilistic element. "God doesn't play dice." Same top-down approach. If Ilya didn’t hold the truth, he might’ve chased benchmark scores like much of the field does. That’s how you end up with models that ace tests and still fail at basic tasks. A baker who knows bread knows what changes when the wheat is grown in winter or when the dough is kneaded by hand. Give him different ingredients and he’ll still make good bread. When someone only has the recipe and no understanding, they can't produce when inputs change. What makes transferring the truth so difficult is that it can’t be fully put into words. Buffett can’t explain the thousands of calculations running through his head when he looks at a business. Federer couldn’t tell you why he swung the way he did. Truth has to be earned. To try to transfer it, we use proxies because proxies can be written down and you can measure against those. Issues this brings: - Each handoff loses some of the truth. - Proxies only work for someone who already holds some of the truth. After a few generations, no one knows what they’re looking for. Easiest example is education. It started by trying to teach people about the world and how to function within it. The grading system was introduced to try to capture whether students were learning. Over time, grading-focused administrations took over, and now nobody cares about learning. In your life, there are areas where you think by proxy and others by truth. Personally, when trying to hire good engineers, I look for people who have come out of IIT Delhi or MIT. I have to use some proxy because I'm not an engineer. When I look for writers, I can just look at their work; doesn't matter their age, education, or culture. Companies start with a founder who holds the truth. As they grow, they establish processes to scale this truth. Eventually, processes and proxies become the thing. That's when a business's culture spreads too thin, leading to what Bezos called Day 2. Almost all of society’s problems stem from people forgetting the truth and mistaking the proxy for it. (Video is clipped from @dwarkesh_sp podcast.)