MLiNS Lab

⚡️🔬📣Excited to share our two new @NatureMedicine articles, we develop computational pathology foundation models, 1. UNI, a self-supervised computational pathology model trained on 100 million pathology images from 100k+ slides. 2. CONCH, a vision-language model for computational pathology trained on 1.17 million pathology image-text pairs. Access the articles @NatureMedicine UNI: nature.com/articles/s41591-0… CONCH: nature.com/articles/s41591-0… Access the code, models: UNI: github.com/mahmoodlab/UNI CONCH: github.com/mahmoodlab/CONCH Interesting aspects: - Both models are evaluated on a host of different clinically relevant tasks for WSI classification, ROI classification, segmentation, image retrieval, image-to-text retrieval, text-to-image retrieval, in 0-shot, few-shot and supervised settings. These adaptations encompass the utility of large public datasets and evaluations on independent test cohorts. - Both models exclude commonly used public computational pathology benchmarks from pre-training allowing for a much more holistic evaluation. Some limitations: Both UNI and CONCH represent early developments in foundation models for pathology. More data, and additional evaluation is needed to realize the full potential of these models. Nevertheless, we show the models capabilities on a variety of different benchmarks with several demonstrating state-of-the-art performance. Future work and insights: While these developments are exciting, they represent work we did about a year ago when the pre-prints were made available, since then we have been busy collecting significantly larger datasets and hope to make larger models available in the future. We have also used UNI and CONCH as the backbone for our Pathology specific chatbot, PathChat (arxiv.org/abs/2312.07814), which is further trained on hundreds of thousands of pathology specific Q-A instructions. We are also excited to see foundation models for several other areas of biomedicine including for single cell data (nature.com/articles/s41592-0…), radiology (nature.com/articles/s42256-0…) and the general trajectory towards general purpose AI for biomedicine. Congratulations to our superstar leaders @richardjchen @MYLu97 @DFKW_MD @TongDing99, Bowen Chen and everyone else who contributed to these studies @GuillaumeJaume @GreatAndrew90 @sharifa_sahai @Aparwani_dpath and others.