LLMs and neural networks catastrophically forget, yet knowledge is often still accessible by probing the hidden states at the right layer or by changing the prompts, even if LLMs cannot directly recall it. In our new work led by @vedantpalit1008 , we ask why this happens and study the mechanisms responsible for hiding this knowledge. We consider a super simple setting in which we are sure knowledge is hidden and not lost: recall initially drops during finetuning, then surprisingly increases without any replay. Shared structure is what is hiding behind hiding knowledge! Happy to have contributed to this work; our results nicely crystallize some intuition that has been around for decades (from the early connectionist days!). Looking forward to seeing how these insights can help us understand the learning mechanics behind such phenomena in actual LLMs!

Scaling laws have become a central part of how we think about scaling LLMs, but it’s not always easy to build intuition for why they work. I made a tutorial on: - scaling laws in practice: classic papers and recent developments - an intuitive associative memory toy model, where scaling can be predicted from simple intuitions - a hands-on environment for learning how to use scaling laws I gave this tutorial at MLSS Tübingen last week, and thought the material was in a good enough shape to share! x nicolaszucchet.github.io/Tut…