Since my previous post got some attention, let me use the opportunity for something more positive: to explain why Antoine Maillard’s work on ellipsoid fitting matters so much to me personally. Cargèse, 2023. A conference I had organised with @zdeborova A blackboard, and d²/4. That is where I first learned about this work from Antoine. I still have these photos from the lecture! You can see the threshold on the board. What you cannot see is everything those ideas would help set in motion. Looking back, a surprising amount of what I now understand about neural networks, the relation between spectra and generalization, feature learning, emergence, power-law scaling, traces back to this line of work. Because its importance was never just the number 1/4. The important thing was the IDEA. The statistical-physics analysis of Antoine Maillard and Dmitriy Kunisky predicted the threshold and the geometry of the solutions. The work of Afonso Bandeira and Antoine Maillard made a crucial part of the programme rigorous, establishing the sharp threshold for approximate fitting through Gaussian equivalence. arxiv.org/abs/2310.01169 arxiv.org/abs/2310.05787 For me, the powerful lesson was not simply “here is the answer to this problem.” It was “here is a way to attack other problems.” And we did. The next step in our own story came in 2024, when I invited Simon Martin to EPFL to tell us about his work with Francis @BachFrancis . That visit helped bring these ideas together around a different question: what can we actually understand about learning in a two-layer neural network? Together with Antoine Maillard, Emanuele Troiani, Simon and Lenka Zdeborová, we asked what happens when one learns a two-layer neural network with quadratic activation, whose width grows proportionally to the input dimension, from quadratically many samples. And the key technical observation was precisely a connection to extensive-rank matrix denoising AND TO THE ELLIPSOID-FITTING PROBLEM. We derived the asymptotic Bayes-optimal learning curve and introduced GAMP-RIE, an algorithm combining approximate message passing with rotationally invariant matrix denoising to achieve that performance. This is the kind of thing we like here in Lausanne :-) Maillard, Troiani, Martin, Krzakala & Zdeborová: “Bayes-optimal learning of an extensive-width neural network from quadratically many samples” NeurIPS 2024 arxiv.org/abs/2408.03733 Then Yizhou Xu, Antoine, Lenka and I pushed this further, putting the statistical-physics predictions on a rigorous footing. We developed rigorous asymptotics and universality results for a broad class of structured matrix-sensing problems, including matrices whose rank grows proportionally to their dimension. Among the applications: establishing Bayes-optimal learning predictions for extensive-width quadratic neural networks. Xu, Maillard, Zdeborová & Krzakala: “Fundamental Limits of Matrix Sensing: Exact Asymptotics, Universality, and Applications” COLT 2025 arxiv.org/abs/2503.14121 And then came one of the most satisfying consequences of this whole programme. With Vittorio Erba, Emanuele Troiani and Lenka, we studied empirical risk minimization in overparameterized quadratic networks. L2 regularization on the network weights becomes NUCLEAR-NORM regularization in an equivalent convex matrix-sensing problem. The mapping itself was not new. What excited us was bringing statistical-physics ideas and AMP to bear on it to derive sharp predictions. Suddenly, we could connect global minima, generalization, memorization, weight spectra and the low-rank bias induced by weight decay. This was exciting!!! And d²/4 CAME BACK—as the interpolation threshold in the noise-dominated limit! More generally, the threshold depends on the target structure and the noise. Erba, Troiani, Zdeborová & Krzakala: “The Nuclear Route: Sharp Asymptotics of ERM in Overparameterized Quadratic Networks” NeurIPS 2025 arxiv.org/abs/2505.17958 But this was not the end of the story. In a complementary direction, Simon Martin, Giulio Biroli and Francis Bach have analyzed the actual gradient-flow dynamics of extensive-width quadratic networks. They developed a high-dimensional dynamical description and characterized the resulting spectra, generalization and recovery thresholds. Martin, Biroli & Bach: “High-Dimensional Analysis of Gradient Flow for Extensive-Width Quadratic Neural Networks” JMLR 2026 arxiv.org/abs/2601.10483 Meanwhile, for us, the matrix/spectral viewpoint opened another door. What if the target itself has a nontrivial spectrum? What if that spectrum follows a power law? How are different spectral modes learned as the amount of data increases? Can this explain neural scaling laws? Can it give us a concrete mechanism for the emergence of newly learned features? This led us to: Defilippis, Xu, Girardin, Troiani, Erba, Zdeborová, Loureiro & Krzakala: “Scaling Laws and Spectra of Shallow Neural Networks in the Feature Learning Regime” ICLR 2026 arxiv.org/abs/2509.24882 There, connections with matrix compressed sensing, LASSO and sparsity allowed us to derive phase diagrams for scaling exponents and connect generalization directly to the spectrum of the learned weights. The spectral viewpoint makes the question concrete: which signal components are detectable, which remain unlearned, and how does that balance change as we get more data? And remarkably, essentially the same viewpoint extends beyond quadratic neural networks. In our work on a simplified single-head attention model, we again characterize learning through the spectrum of a learned matrix. We obtain training and test errors, interpolation and recovery thresholds, and the full singular-value distribution of the learned query–key map, including low-rank structure and spectral outliers. For power-law targets, learning proceeds through sequential spectral recovery: progressively weaker signal directions become learnable as the amount of data increases. A concrete mechanism for emergence in this model—and scaling laws come out of the same analysis! Boncoraglio, Erba, Troiani, Xu, Krzakala & Zdeborová: “Single-Head Attention in High Dimensions: A Theory of Generalization, Weights Spectra, and Scaling Laws” ICML 2026 arxiv.org/abs/2509.24914 And this brings me back to why I care so much about the original story. A good scientific paper does not merely produce an answer to one isolated problem. Sometimes it gives a community a new connection, a new mapping, a new proof strategy. A PATH. For us, one path ran through: Ellipsoid fitting → Gaussian equivalence → extensive-rank matrix estimation → quadratic neural networks → feature learning → spectra → scaling laws → attention. Gaussian equivalence itself has a long, crazy history (don’t get me started on that one...) Of course, this is not a single linear chain. Many researchers and ideas contributed at every stage, and several of these connections long predate the papers I have mentioned. But having watched part of this story develop from very close by, I wanted to explain how Antoine’s work, with his collaborators, helped set this particular programme in motion—and how much we owe to it. Looking at that blackboard again, I do not just see a threshold. I see ideas that helped us do research we did not know how to do before. To me THAT is what “empowering researchers” actually looks like.

This is outrageous framing. Quite frankly, given the current tensions around AI and mathematics, this kind of announcement does not help. After the recent Buckmaster/OpenAI dispute over scientific credit and provenance, the last thing we need is more promotional PR language that blurs the history of a research problem. Meta presents its work by asking: “Can AI contribute when a problem is genuinely open and without an existing solution path?” Seriously?! For the ellipsoid-fitting example, this is misleading and insulting. There was an established research programme, substantial rigorous progress, and by the time of their announcement, publicly available proofs of the exact threshold. Let us be specific: - In 2023, Antoine Maillard and Dmitriy Kunisky gave a statistical-physics derivation of the conjectured transition at n/d² = 1/4, using the replica method. Their analysis also predicted the geometry of the solutions and suggested directions for rigorous proofs. This was not a proof, but it was a detailed and successful theoretical picture, not an absence of a path. arxiv.org/abs/2310.01169 - Afonso Bandeira and Antoine Maillard then rigorously established the sharp threshold for approximate fitting, using methods inspired by the statistical physics of disordered systems, using actually spin glass mathematical technics. Their preprint appeared in 2023 and was published in the Electronic Journal of Probability in 2025. Their Gaussian-equivalence framework provided a foundation for further work (including many of mine!). At this point, the "exact" fitting and removing the spectral restriction were the outstanding gaps, but these are technical problems, it was not about not discovering the threshold from scratch. arxiv.org/abs/2310.05787 - On 10 August 2026, Theodor Misiakiewicz and Garrett Wen posted a proof of the exact Gaussian threshold, explicitly closing those two gaps. arxiv.org/abs/2608.10184 - On 12 August, Sofia de la Cerda, Aaron Potechin, Madhur Tulsiani and Jeff Xu posted another proof, establishing both sides of the transition up to a vanishing multiplicative factor. arxiv.org/abs/2608.12415 - On 27 August, Frederic Koehler and Youngtak Sohn posted a broader universality theorem that includes the Gaussian threshold. arxiv.org/abs/2608.27372 These were not researchers merely “on the verge” of solving the problem. THE PROOFS WERE ALREADY PUBLIC. Meta's announcement comes in October, and Section 3 of their paper begins: “We use the groundbreaking approximate-fitting theorem of Bandeira and Maillard…” Dear META: Those are your words. This is not simply a reference in the bibliography: your feasibility proof explicitly starts from their theorem. How is that consistent with advertising this example through the suggestion that there was “no existing solution path”? Yes, your social post mentions independent solutions, and your paper and blog identify the August results and describe your work as independently developed. That does not answer the objection to the framing. A generic acknowledgement of parallel work does not make clear that the exact threshold already had publicly available proofs before this announcement. An independently developed AI-assisted proof, released after other proofs, should be presented as exactly that. Readers should not have to reconstruct the chronology to understand what is, and is not, new. A new proof can contain valuable ideas. But a new proof and a newly solved problem are not the same claim. Nor does a problem being open when a project begins mean it is still open when the marketing campaign launches. You say the aim is to “empower researchers”? Nothing more “empowering” than seeing years of collective research turned into the backdrop for a cheap marketing stunt. Correct the announcement! Give the scientific history the same prominence as the product. Give Antoine, Afonso, Theo and all the other researchers involved the public credit they deserve.