Multi-modal Sequence Encoding for AMPs

Benchmarking peptide sequence encoders and multi-modal transfer learning for out-of-distribution generalization. CIS 5200 (Machine Learning).

Overview

For this project, we asked whether models trained on modern antimicrobial peptides could predict the activity of peptides from extinct organisms—an idea known as molecular de-extinction.

The main challenge was distribution shift: good predictions on familiar sequences might not carry over to ancient ones. We tested whether adding AlphaFold structural features to a sequence model would help it generalize.

Notebook · Report

Approach

We formulated the prediction of log2-transformed Minimum Inhibitory Concentration (MIC) as a supervised regression task, using the publicly available Database of Antimicrobial Activity and Structure of Peptides (DBAASP) with 25,306 unique peptide-target interactions across 6,233 sequences and 50 bacterial targets.

We compared three approaches:

  1. Encoder Development: We implemented an APEX-based Sequence Encoder combining GRU layers with attention mechanisms to translate primary amino acid sequences into high-dimensional latent representations.
  2. Multi-modal Transfer Learning: To improve OOD stability, we integrated five explicit structural features (pLDDT, helix fraction, disordered state, etc.) extracted from AlphaFold via ColabFold. These were concatenated into a multi-modal layer to provide the model with geometric context.
  3. Baseline Comparison & Interpretability: We benchmarked against tree-based models (XGBoost, Random Forest) and used feature importance to see how much the structural inputs contributed.
Diagram of the multi-modal pipeline, showing the integration of sequence embeddings and AlphaFold structural features.

Results

Performance changed substantially when we moved to extinct peptides:

  • Modern peptides: the sequence encoder reached Spearman rho = 0.79 and R² = 0.62, with comparable performance from XGBoost and Random Forest.
  • Extinct peptides: on 69 peptides, including 41 validated AMPs, Spearman rho fell to 0.36.
  • Structural features: these received less than 1% feature weight in our analysis and added little useful signal in this setup.
Correlation analysis of predicted vs. actual activity, highlighting the generalization challenges in OOD datasets.

Adding a few structural summaries wasn’t enough to solve the generalization problem. I’d be interested in testing richer representations, such as graph neural networks, that retain more of the predicted structure.

Skills Used

  • Programming: PyTorch, scikit-learn, Pandas, NumPy, modlAMP, ColabFold
  • ML concepts: GRU networks, attention mechanisms, transfer learning, multi-modal learning, XGBoost, Random Forest, regression, out-of-distribution generalization, feature importance analysis
  • Domain-specific: AlphaFold, protein structure prediction, antimicrobial peptides, DBAASP database, sequence encoding