PhD candidate at @GeorgiaTech | ML for bioengineering | @UCSanDiego '23
Joined April 2024
- Tweets58
- Following116
- Followers161
- Likes58
Excited to share that our review article, “Spectral Sparsity: A Unifying Framework for Scalable Model Interpretability Using Codes”, is out in IEEE BITS! We exploit spectral sparsity to scale feature attribution across language and biology models. ieeexplore.ieee.org/document… (1/n)
Explanation tools like Shapley values and higher-order interactions have elegant representations in the spectral domain. By framing feature recovery as a compressed sensing problem, coding tools let us bypass exponential costs to achieve sub-linear scaling. (4/n)
Huge thanks to our collaborators at UC Berkeley and my PI @amiraliagz for making this possible! (n/n)
⭐ Paper: ieeexplore.ieee.org/document…
Protein fitness landscapes are simpler under evolutionary distributions!
We show that much of the high-order epistasis observed in fitness landscapes is substantially diminished when analyzed under their natural evolutionary distributions. biorxiv.org/content/10.64898… (1/n)
Notably, the epistatic interactions highlighted by the eWHT were highly interpretable! Across diverse fitness landscapes, the eWHT prioritized interactions that were structurally localized and present at contact regions. (5/n)
Finally, to facilitate adoption by the community, we provide an open-source Python package implementation of the eWHT!
⭐ Paper: biorxiv.org/content/10.64898…
⭐ Package: github.com/amirgroup-codes/e…
P.S. special thanks to @kunalt12345 and my PI @amiraliagz!
Excited to share that our paper, ProGenMech, has been accepted to the Mechanistic Interpretability Workshop at ICML 2026!
Here, we bring circuit tracing to life in ProGen3, an autoregressive protein language model! (1/n)
Paper: arxiv.org/abs/2606.16044
Some personal thoughts: there’s still much to do! For one, ProGenMech abstracts away the MoE computation, which likely leaves performance on the table. We're also studying well-characterized proteins, and there's likely much to learn by studying lesser-known proteins. (4/n)
Finally, huge shout-out to the team @saeedi_daniel @amiraliagz! And a special shout-out to our undergrad William (who's not on here but you can find him on Linkedin at linkedin.com/in/william-dein…) who led a lot of our model development and engineering efforts! (n/n)
Darin Tsui retweeted
@saeedi_daniel will be presenting our cryoSENSE work at a poster session on Sunday at @CVPR. Stop by to learn about cryoSENSE!❄️🔬
At #CVPR2026? Come by ExHall F 508 on Sun, Jun 7, 11:45 AM–1:45 PM MDT to chat about cryoSENSE — using compressive sensing with sparse and diffusion priors to enable higher-throughput cryo-EM while preserving structural resolution.
#CryoEM #ComputationalImaging
The landscape of AI x bio is quickly changing. In this LessWrong piece, I give my perspective on leveraging AI as biology's digital microscope and how we can leverage mechanistic interpretability for scientific discovery
lesswrong.com/posts/BxziXpLC…
Sharing a thoughtful LessWrong piece by my colleague @darin_tsui on AI as biology’s digital microscope: how mechanistic interpretability can help turn biological AI models from black boxes into tools for discovery:
lesswrong.com/posts/BxziXpLC…
#LessWrong #MechInterp
Interesting find! We previously identified a similar phenomenon when applying speculative decoding in protein language models with @amiraliagz and solved this by adding evolutionary priors, resulting in significantly more plausibly functional sequences arxiv.org/abs/2509.21689
“Attention Drift: What Autoregressive Speculative Decoding Models Learn”
Speculative decoding makes LLM inference faster, but drafters break under small template changes and long context. But why?
This paper shows that as the drafter predicts more tokens, its attention drifts away from the prompt and onto its own recent guesses.
The key fix is to normalize the drafter’s hidden states so they stay stable across draft steps.
With post-norm and RMSNorm fusion, the paper gets up to 2x better acceptance under template perturbations and stronger long-context robustness.
Can we learn how protein language models (pLMs) work?👀
As a matter of fact, yes! I'm excited to shared that our paper, "Protein Circuit Tracing via Cross-layer Transcoders", has been accepted into ICML 2026! Check out our findings at protmech.github.io/ (1/n)
To democratize circuit tracing, we have open-sourced ProtoMech! Run ProtoMech through Colab and visualize your circuits on our website! We’re excited to see what the community discovers! (4/n)
⭐ Colab: colab.research.google.com/gi…
⭐ Video tutorial: youtube.com/watch?v=wF6xUQDx…
P.S. huge shout-out to the rest of the team @kunalt12345 @saeedi_daniel and my PI @amiraliagz for all their efforts! Special thanks to @saeedi_daniel for designing the website! (n/n)
⭐ Paper: arxiv.org/abs/2602.12026
#ICML #ICML2026 #MechanisticInterpretability #AI4Science
