AI for scientific discovery. Head of Science, Biohub. Founder and scientific director of the ESM project.
Brooklyn, NY
Joined April 2009
- Tweets400
- Following899
- Followers16.4K
- Likes3.3K
Pinned Tweet
Today we're announcing ESMFold2, an open scientific engine to power prediction, design, and discovery across protein biology.
The new model delivers state of the art performance on protein interactions, especially antibodies, a critical modality for therapeutics.
We have designed and validated miniprotein binders and single chain antibodies across five therapeutic targets that are important in cancer and immunology. We are seeing very high success rates, and affinities at levels consistent with therapeutic activity.
We’re also releasing an atlas of 6.8 billion proteins, and 1.1 billion predicted structures.
ESMFold2 is built on a state of the art language model that has been trained on billions of protein sequences.
A world model of protein biology emerges through language modeling.
We’ve used the techniques of mechanistic interpretability developed to understand large language models to understand the concepts ESM uses to represent proteins.
The model’s representation space has a compositional organization of features across scales, levels of complexity, and abstraction, that reflects and mirrors the understanding of protein biology developed through a century of empirical science.
This understanding emerges without prior knowledge, just from language modeling of protein sequences.
Language models are becoming a powerful substrate to understand and program biology.
The design of protein interactions is one of the most fundamental problems in biophysics, and has critical implications for the discovery of new medicines. A simple gradient based search with the model was able to discover high-affinity protein binders.
I'm excited by the potential this has to accelerate basic science and the understanding of proteins. And especially for the new avenues it opens up for therapeutic design and medicine.
A universal predictive model of the cell would dramatically accelerate science, allowing biologists to perform experiments digitally. Today we’re announcing a major partnership with the DOE and NIH. The goal of this partnership is to generate the data that will be needed to build an accurate model of the cell with artificial intelligence.
Isomorphic, Google DeepMind, and Meta are joining us as founding partners in the Virtual Biology Initiative. This is the beginning of a large scale coordinated effort to map cellular biology to power the development of digital models of life. Other leading scientific institutions and consortia including the Allen Institute, Broad Institute, Gladstone Institutes, the Human Cell Atlas, the Human Protein Atlas, the Wellcome Sanger Institute, as well as NVIDIA, are working together as part of this international scientific project.
Creating a predictive model of the cell is one of the most important challenges for the next decade of science. The insights that come from this could unlock a far greater understanding of disease, and open new paths for cures.
We invite the worldwide scientific community and other funders to join us in this effort.
Alex Rives retweeted
PCA reveals beautiful geometry inside a protein language model. But does the model actually use it?
I explored how protein folds—like this beta-propeller fold—are represented in ESMC-6B using the interpretability tools built into Silico, @GoodfireAI's research platform 🧵
Together with UC Berkeley we are announcing the laser phase plate - a breakthrough in atomic resolution imaging. This is the brightest continuous wave laser in the world, 100 million times the intensity of the surface of the sun.
Phase contrast plays an important role in microscopy, but it was thought close to impossible for electron microscopy, where it would require interfering with an electron beam. Holger Mueller and Robert Glaeser proposed exactly this using a standing wave laser. It has taken over 15 years to make this a reality. Biohub partnered with UC Berkeley and Mueller to support this work and to engineer and build the technology.
Contrast has been the critical barrier to achieving atomic resolution imaging of the cell. In cryo-electron tomography, a cellular imaging technology that uses electron microscopy, the low contrast makes it impossible to resolve anything but the largest proteins within their cellular context. The laser phase plate removes that barrier.
With advances in AI this breakthrough in contrast will start to open up a new frontier in structural biology, that will allow us to see the molecular machines of the cell, and how they assemble into far more complex and dynamic systems, and understand how they work.
Great talking with RJ and Brandon on the Latent Space AI for Science podcast, about protein language models, scaling in biology, and the new ESMFold2.
🆕Biohub’s Protein World Model: ESMC-6B, ESMFold2, 6.8B proteins, 1.1B structures, antibody design, SAEs, & the bitter lesson for biology latent.space/p/esmfold2
@biohub Head of Science @alexrives explains why biology may scale like language modeling, how metagenomics unlocked the next ESM scaling curve, why protein LMs can learn structure/function from sequence alone, how sparse autoencoders reveal biology inside the model, why ESMFold2 can beat specialized systems on antibody-antigen prediction, and how Biohub’s $500M Virtual Biology Initiative aims to build predictive models of cells, disease, and eventually physiology.
Alex Rives retweeted
🆕Biohub’s Protein World Model: ESMC-6B, ESMFold2, 6.8B proteins, 1.1B structures, antibody design, SAEs, & the bitter lesson for biology latent.space/p/esmfold2
@biohub Head of Science @alexrives explains why biology may scale like language modeling, how metagenomics unlocked the next ESM scaling curve, why protein LMs can learn structure/function from sequence alone, how sparse autoencoders reveal biology inside the model, why ESMFold2 can beat specialized systems on antibody-antigen prediction, and how Biohub’s $500M Virtual Biology Initiative aims to build predictive models of cells, disease, and eventually physiology.
We’re also releasing the largest atlas of protein structure and function. With 6.8 billion proteins, and 1.1 billion predicted structures, it enables analysis and protein discovery at the scale of life.
At @Biohub, our goal is to build models that accelerate scientific discovery and progress toward the cure to disease. We’re releasing all of this under MIT license allowing commercial and non-commercial use.
Read more here: biohub.ai/esm/protein/
Alex Rives retweeted
The AI revolution in science is powered by open source software. Let's fund it.
Today we're launching the Open Source for Science Fund @os4science, a new multi-donor philanthropic fund by @RenPhilanthropy, seeded by @biohub and @wellcometrust, with support from @KavliFoundation and Research Software Alliance.
🧵os4science.org/news/open-sou…
1/4
Alex Rives retweeted
Congrats @alexrives and @biohub team! Truly spectacular vision.
Scaling laws are powering AI. It’s time to scale biology.
Today we’re launching the Virtual Biology Initiative to generate the data to unlock scaling laws in biology and build accurate predictive models of the cell.
Digital representations of proteins are already expanding our understanding of life at the molecular level, and accelerating the design of molecules and medicines. Accurate digital representations of the cell could reveal the mechanisms that are responsible for disease, and show how to reverse them.
The protein data bank, and worldwide repositories of protein sequence biodiversity were created through decades of work by the scientific community. The advances in artificial intelligence for proteins would not have been possible without them.
The cell is orders of magnitude more complex, and we will need to create the data in just a few years rather than decades.
This will require a coordinated global effort. We're partnering with Broad, Wellcome Sanger, Arc, Allen, Human Cell Atlas, Human Protein Atlas, NVIDIA, and Renaissance Philanthropy.
Biohub is contributing to this effort as both a funder and a builder. We are developing microscopy to observe millions of cells in living organisms, and cryo-ET to resolve the cell in atomic detail. We're building instruments that expand the range of modalities and parameters that can be simultaneously measured. We’re developing molecular, cellular, and tissue engineering to create models of disease and design interventions.
The data we generate will be available to the worldwide scientific community.
We’re also committing $100M over the next five years to support work beyond Biohub.
We invite other scientific teams and funders to join.
Link: biohub.org/news/virtual-biol…
Alex Rives retweeted
$500M+ towards open data in biology and an amazing set of partners committed to generating the data. It's exciting to think about what models built on this data will be capable of.
Scaling laws are powering AI. It’s time to scale biology.
Today we’re launching the Virtual Biology Initiative to generate the data to unlock scaling laws in biology and build accurate predictive models of the cell.
Digital representations of proteins are already expanding our understanding of life at the molecular level, and accelerating the design of molecules and medicines. Accurate digital representations of the cell could reveal the mechanisms that are responsible for disease, and show how to reverse them.
The protein data bank, and worldwide repositories of protein sequence biodiversity were created through decades of work by the scientific community. The advances in artificial intelligence for proteins would not have been possible without them.
The cell is orders of magnitude more complex, and we will need to create the data in just a few years rather than decades.
This will require a coordinated global effort. We're partnering with Broad, Wellcome Sanger, Arc, Allen, Human Cell Atlas, Human Protein Atlas, NVIDIA, and Renaissance Philanthropy.
Biohub is contributing to this effort as both a funder and a builder. We are developing microscopy to observe millions of cells in living organisms, and cryo-ET to resolve the cell in atomic detail. We're building instruments that expand the range of modalities and parameters that can be simultaneously measured. We’re developing molecular, cellular, and tissue engineering to create models of disease and design interventions.
The data we generate will be available to the worldwide scientific community.
We’re also committing $100M over the next five years to support work beyond Biohub.
We invite other scientific teams and funders to join.
Link: biohub.org/news/virtual-biol…
Alex Rives retweeted
We will look back and see this as the start of a wonderful accelerator.
And we @tahoe_ai are committed to building alongside these great orgs.
Scaling laws are powering AI. It’s time to scale biology.
Today we’re launching the Virtual Biology Initiative to generate the data to unlock scaling laws in biology and build accurate predictive models of the cell.
Digital representations of proteins are already expanding our understanding of life at the molecular level, and accelerating the design of molecules and medicines. Accurate digital representations of the cell could reveal the mechanisms that are responsible for disease, and show how to reverse them.
The protein data bank, and worldwide repositories of protein sequence biodiversity were created through decades of work by the scientific community. The advances in artificial intelligence for proteins would not have been possible without them.
The cell is orders of magnitude more complex, and we will need to create the data in just a few years rather than decades.
This will require a coordinated global effort. We're partnering with Broad, Wellcome Sanger, Arc, Allen, Human Cell Atlas, Human Protein Atlas, NVIDIA, and Renaissance Philanthropy.
Biohub is contributing to this effort as both a funder and a builder. We are developing microscopy to observe millions of cells in living organisms, and cryo-ET to resolve the cell in atomic detail. We're building instruments that expand the range of modalities and parameters that can be simultaneously measured. We’re developing molecular, cellular, and tissue engineering to create models of disease and design interventions.
The data we generate will be available to the worldwide scientific community.
We’re also committing $100M over the next five years to support work beyond Biohub.
We invite other scientific teams and funders to join.
Link: biohub.org/news/virtual-biol…
Alex Rives retweeted
Great to see Biohub's investment in the future of AI and cell biology with its 5-year Virtual Biology Initiative programme. Beautiful alignment with the roadmap for HCA 2.0, and lots of excitement about future synergies.
Today we're announcing the Virtual Biology Initiative—a major new commitment to build the data foundation biology needs to power the next generation of AI models. Next-gen imaging. Molecular engineering tools. Open data infrastructure for the entire scientific community.
Read the full story: bit.ly/3ONTkHO
Alex Rives retweeted
Congratulations to @biohub and their partners on the announcement of the Virtual Biology Initiative – a global effort to build predictive models of cells and understanding of health and disease. @RenPhilanthropy is delighted to participate and I hope that more funders and scientific teams will back this critical effort.
Scaling laws are powering AI. It’s time to scale biology.
Today we’re launching the Virtual Biology Initiative to generate the data to unlock scaling laws in biology and build accurate predictive models of the cell.
Digital representations of proteins are already expanding our understanding of life at the molecular level, and accelerating the design of molecules and medicines. Accurate digital representations of the cell could reveal the mechanisms that are responsible for disease, and show how to reverse them.
The protein data bank, and worldwide repositories of protein sequence biodiversity were created through decades of work by the scientific community. The advances in artificial intelligence for proteins would not have been possible without them.
The cell is orders of magnitude more complex, and we will need to create the data in just a few years rather than decades.
This will require a coordinated global effort. We're partnering with Broad, Wellcome Sanger, Arc, Allen, Human Cell Atlas, Human Protein Atlas, NVIDIA, and Renaissance Philanthropy.
Biohub is contributing to this effort as both a funder and a builder. We are developing microscopy to observe millions of cells in living organisms, and cryo-ET to resolve the cell in atomic detail. We're building instruments that expand the range of modalities and parameters that can be simultaneously measured. We’re developing molecular, cellular, and tissue engineering to create models of disease and design interventions.
The data we generate will be available to the worldwide scientific community.
We’re also committing $100M over the next five years to support work beyond Biohub.
We invite other scientific teams and funders to join.
Link: biohub.org/news/virtual-biol…
Alex Rives retweeted
Exactly what needs to be done. Biological data is the missing link. It may not be sexy or make for shiny announcements but building biological infrastructure is where the impact is. Huge props to CZI.
biohub.org/news/virtual-biol…
Alex Rives retweeted
🧬🤖 Thrilled to be part of this incredible initiative led by @alexrives and team!
Excited to contribute at @sangerinstitute
Can’t wait for the data we’ll generate and the models we’ll train 🚀
#AI #Genomics #SingleCell
Scaling laws are powering AI. It’s time to scale biology.
Today we’re launching the Virtual Biology Initiative to generate the data to unlock scaling laws in biology and build accurate predictive models of the cell.
Digital representations of proteins are already expanding our understanding of life at the molecular level, and accelerating the design of molecules and medicines. Accurate digital representations of the cell could reveal the mechanisms that are responsible for disease, and show how to reverse them.
The protein data bank, and worldwide repositories of protein sequence biodiversity were created through decades of work by the scientific community. The advances in artificial intelligence for proteins would not have been possible without them.
The cell is orders of magnitude more complex, and we will need to create the data in just a few years rather than decades.
This will require a coordinated global effort. We're partnering with Broad, Wellcome Sanger, Arc, Allen, Human Cell Atlas, Human Protein Atlas, NVIDIA, and Renaissance Philanthropy.
Biohub is contributing to this effort as both a funder and a builder. We are developing microscopy to observe millions of cells in living organisms, and cryo-ET to resolve the cell in atomic detail. We're building instruments that expand the range of modalities and parameters that can be simultaneously measured. We’re developing molecular, cellular, and tissue engineering to create models of disease and design interventions.
The data we generate will be available to the worldwide scientific community.
We’re also committing $100M over the next five years to support work beyond Biohub.
We invite other scientific teams and funders to join.
Link: biohub.org/news/virtual-biol…
Alex Rives retweeted
Really thrilled by the framing and scale of this new @biohub initiative, including the combination of internal & external investment in tech-dev, open data, modeling and collaborative science, all good things!
Scaling laws are powering AI. It’s time to scale biology.
Today we’re launching the Virtual Biology Initiative to generate the data to unlock scaling laws in biology and build accurate predictive models of the cell.
Digital representations of proteins are already expanding our understanding of life at the molecular level, and accelerating the design of molecules and medicines. Accurate digital representations of the cell could reveal the mechanisms that are responsible for disease, and show how to reverse them.
The protein data bank, and worldwide repositories of protein sequence biodiversity were created through decades of work by the scientific community. The advances in artificial intelligence for proteins would not have been possible without them.
The cell is orders of magnitude more complex, and we will need to create the data in just a few years rather than decades.
This will require a coordinated global effort. We're partnering with Broad, Wellcome Sanger, Arc, Allen, Human Cell Atlas, Human Protein Atlas, NVIDIA, and Renaissance Philanthropy.
Biohub is contributing to this effort as both a funder and a builder. We are developing microscopy to observe millions of cells in living organisms, and cryo-ET to resolve the cell in atomic detail. We're building instruments that expand the range of modalities and parameters that can be simultaneously measured. We’re developing molecular, cellular, and tissue engineering to create models of disease and design interventions.
The data we generate will be available to the worldwide scientific community.
We’re also committing $100M over the next five years to support work beyond Biohub.
We invite other scientific teams and funders to join.
Link: biohub.org/news/virtual-biol…