Antigen-specific Antibody Multi-modal Foundation Model for Functional Antibody Design 1 AAMFM is an antigen-specific antibody multimodal foundation model built on ESM3 that jointly models antibody sequence and structure while being explicitly conditioned on antigen context, aiming to close the gap where general protein language models struggle: epitope-level, paired antibody–antigen design. 2 The core idea is to put antibody sequence tokens, antibody structure tokens, and antigen-derived signals into a shared latent space, so generation is driven by the actual complex context (antigen geometry + epitope annotations), not just antibody priors. 3 A key engineering choice is a lightweight Antigen Geometric–Epitope-aware Adapter (about 5.1M params, ~0.36% of the 1.4B base model) inserted after the last attention block, enabling antigen conditioning without retraining the entire foundation model from scratch. 4 Antigen conditioning is multi-source: (a) antigen 3D geometry is encoded as a graph and embedded via GearNet to produce geometric features; (b) epitope sites are provided as a binary mask and encoded with embeddings + positional encoding + a Transformer encoder to capture interface context; (c) these signals are fused with the antibody backbone representation via down-projection/concat/MLP/up-projection with a residual update. 5 Training is hierarchical: (a) start from ESM3-open (1.4B); (b) antibody domain adaptation by continued pretraining on ~1.4M paired antibody sequence–structure examples from OAS (structures predicted by AbodyBuilder2/IgFold, with coordinate noise augmentation); (c) supervised fine-tuning on high-resolution SAbDab antibody–antigen complexes, learning more realistic interaction patterns. 6 The masking strategy is antibody-aware: besides random masking, they frequently mask entire CDR regions (and in SFT sometimes mask only within CDRs), forcing the model to learn CDR completion/design under fixed frameworks—aligned with how antibodies vary in nature and how design is practiced. 7 The functional alignment step is AF3-driven preference optimization: candidate CDR sequences sampled from the SFT model are evaluated by Protenix (an open AlphaFold3 reproduction) to obtain AF3 ranking score, pTM, and ipTM as structure-and-binding confidence signals; these scores are used as an oracle to define preferences for post-training. 8 To reduce “reward hacking” against the structural oracle, preferences are constrained by a second, nearly orthogonal plausibility signal: AntiBERTy pseudo-log-likelihood (PLL). Preference pairs are kept only when the preferred sequence beats the rejected one by at least 0.2 AF3-score and 0.1 PLL (Spearman correlation between AF3-score and PLL reported as 0.13). 9 Preference learning uses Calibrated Direct Preference Optimization (Cal-DPO), optimizing only over CDR tokens (length-normalized). Compared to standard DPO, Cal-DPO adds a calibration term so likelihood ratios reflect absolute quality margins, empirically stabilizing training dynamics (preferred likelihood stays steadier while dispreferred decreases). 10 Results on SAbDab design benchmarks show consistent gains in plausibility and complex-level functional metrics: for full antibody (all 6 CDRs) co-design, AAMFM-CalDPO improves AF3-score, pTM, and ipTM over ESM3-open and antibody-specific baselines (DiffAb, dyMEAN, AbX). In epitope-binding CDR-H3 co-design, AAMFM variants lead on PLL and AF3-derived functional scores, with competitive AAR/RMSD tradeoffs typical of shifting from memorization (SFT) to preference-guided exploration (DPO). 💻Code: github.com/XL-S224/AAMFM. 📜Paper: arxiv.org/abs/2607.20057 #ComputationalBiology #ProteinDesign #AntibodyDesign #ProteinLanguageModels #MultimodalAI #AlphaFold3 #ESM3 #DPO #Bioinformatics #MachineLearning