IBM Research, IISc, Trustworthy AI

Bangalore
Joined July 2023
Really excited to share this work! We look at a fundamental question in SAE training: How much data do you actually need to train a Sparse Autoencoder (SAE)? Give it a read and let us know what you think!
Exciting work led by @sambitghsh and @MadelynMathai on the diminishing returns of data in Sparse Autoencoders. Standard practice often uses millions of activation tokens, but it’s unclear whether that scale is necessary for different SAE goals. We investigate this systematically by training different SAE architectures on different LLM families with training budgets ranging from 5M to 500M activation tokens.
4
82
Sambit Ghosh retweeted
We are thrilled to announce the inaugural Agyeya Research Workshop, hosted in partnership with the @iiscbangalore! Our mission is to break down barriers for early-career researchers and cultivate India’s next generation of AI leaders. As part of the workshop, we are also launching the Agyeya AI Scholar program. This initiative aims to select and train a dedicated cohort of candidates through hands-on research and mentorship in AI Alignment, Interpretability, and Safety. 🗓️ Apply by June 30th and stay tuned for our upcoming mentor announcements! Find more details and submit your application here: lnkd.in/efGa44KT @ponguru Sriparna Saha, @rvbabuiisc, @NikitaKharya, @precogatiiith @val_iisc
15
36
5
425
330,998
📣Thrilled that "Rethinking Dataset Distillation: Hard Truths about Soft Labels" is a Best Paper Finalist @CVPR (Top 15 out of 16,000+ submissions)! 🏆 Congrats to the authors! 🥳🙌🏻 Join us today: 🎙️ Oral: 10:15 @ Mile High Ballroom 1A-2A 📍Poster 18 @ ExHall A
2
14
3
88
5,601
Sambit Ghosh retweeted
Do Vision Language Models (VLMs) really need deep visual processing? 🤖 Our paper accepted as Oral Presentation @CVPR TRUE-V Workshop suggests the current paradigm of multimodal LLM architectures might be wildly inefficient. Here is why we might be over-processing image tokens 🧵 (1/4) Paper: arxiv.org/abs/2604.09425 Code: github.com/sambitghsh/VLM-To… Congrats to @sambitghsh for leading this work! Great collaboration with @rvbabuiisc and @val_iisc!
4
1
5
666