Ben Levy

Really cool work! Initial thoughts: 1. The performance leap compared to Evo2, GPN-MSA etc. is quite remarkable, but we need to remember it's a supervised approach. The gene-level split sounds reasonable, but information leakage is notoriously difficult to eliminate in supervised variant effect prediction. It remains to be seen how well it generalizes (I hope to see more comprehensive evals in the peer-reviewed version). 2. I'm a bit skeptical about the LLM-generated reports reflecting either the true disease mechanism of the variants or the true reasoning behind the Evo2 probe predictions. 3. At the same time, I will not be surprised if geneticists end up liking and using these reports, even if they are not entirely faithful. At least they attempt to provide an explanation (unlike virtually all other variant effect prediction tools), so it's an important step in a neglected direction. 4. Moreover, I'd argue that a lot of biological and clinical reasoning is already in the realm of nice-sounding storytelling and half truths, so we also need to be honest about what human-level biological interpretation is when we judge whether AI is making things better or worse.