رقيا | Ruqiya

Can a model learn without training? I kept hearing about in-context learning, but didn't know what actually happens inside the model. Then I learned it isn't magic or new training: it's two attention heads working together to search and copy.

A follow-up to part one on the residual stream: once we can picture it, we need to interpret what it carries, which is why Sparse Autoencoders became my focus for understanding what goes on inside the model.

My entry point into mechanistic interpretability was simply trying to picture what the Residual Stream actually is and how it moves. Think of it as a quiet river flowing through 26 layers: each layer reads from it and adds its piece, never erasing what came before.

We added one vector inside Gemma 3, and it answered an English prompt in Arabic. How it works, in 50 seconds 👇

الحمد لله الذي تتمّ بنعمته الصالحات سلسلة نماذج RuqLM العربية الصغيرة، المدرّبة من الصفر على بيانات معظمها مأخوذ من أستاذها ومعلمها عَلّام ALLaM @HUMAIN بدأت هذه السلسلة بسؤال بسيط: هل يمكننا بناء نماذج عربية صغيرة جدًا ومتماسكة بأقل من مليار بارامتر؟ ruqlm.web.app/login.html

🔍 Looking for a Hindi-speaking volunteer for NLP & AI safety research. I'm conducting a pre-registered study on script anchoring in LLM safety alignment and need a native Hindi speaker to review ~200 prompt pairs (Devanagari ↔ ITRANS transliteration).