Cited by 335Protein Structure and DynamicsMachine Learning in BioinformaticsEnzyme Structure and Function

Benchmarking AlphaFold for protein complex modeling reveals accuracy determinants

Open access
Yin, Rui, Feng, Brandon Y., Varshney, Amitabh, Pierce, Brian G.·2022·Protein Science

Published version of a preprint

Abstract

High-resolution experimental structural determination of protein-protein interactions has led to valuable mechanistic insights, yet due to the massive number of interactions and experimental limitations there is a need for computational methods that can accurately model their structures. Here we explore the use of the recently developed deep learning method, AlphaFold, to predict structures of protein complexes from sequence. With a benchmark of 152 diverse heterodimeric protein complexes, multiple implementations and parameters of AlphaFold were tested for accuracy. Remarkably, many cases (43%) had near-native models (medium or high critical assessment of predicted interactions accuracy) generated as top-ranked predictions by AlphaFold, greatly surpassing the performance of unbound protein-protein docking (9% success rate for near-native top-ranked models), however AlphaFold modeling of antibody-antigen complexes within our set was unsuccessful. We identified sequence and structural features associated with lack of AlphaFold success, and we also investigated the impact of multiple sequence alignment input. Benchmarking of a multimer-optimized version of AlphaFold (AlphaFold-Multimer) with a set of recently released antibody-antigen structures confirmed a low rate of success for antibody-antigen complexes (11% success), and we found that T cell receptor-antigen complexes are likewise not accurately modeled by that algorithm, showing that adaptive immune recognition poses a challenge for the current AlphaFold algorithm and model. Overall, our study demonstrates that end-to-end deep learning can accurately model many transient protein complexes, and highlights areas of improvement for future developments to reliably model any protein-protein interaction of interest.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1002/pro.4379

Cites

72 of 72 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 2

  1. [69]Wickham H. ggplot2: Elegant graphics for data analysis. New York: Springer‐Verlag; 2016.
  2. [72]Thiele C, Hirschfeld G. Cutpointr: Improved estimation and validation of optimal Cutpoints in R. J Stat Softw. 2021; 98(11): 1– 27.

Cited by

Sorted by citations

Authors
Yin, R., Feng, B. Y., Varshney, A., & Pierce, B. G.
Affiliations
  • Institute for Bioscience and Biotechnology Research (US)
  • University of Maryland, College Park (US)
  • University of Maryland, Baltimore (US)
Corresponding author
Pierce, Brian G.
Venue
Protein Science
Publisher
Wiley
Published
2022-07-13
Volume
31
Issue
8
Pages
e4379-e4379
ISSN
  • 0961-8368 (ISSN-L)
  • 1469-896X
OpenAlex
W4285392187
Cited by
335
References
72
Citations by year
2026: 47 · 2025: 67 · 2024: 111 · 2023: 90 · 2022: 19 · 2013: 1
FWCI
23.43
Type
journal article
Language
en
Open access
Locations
  • Protein Science (publishedVersion, cc-by)
  • bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nd)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • University Libraries (University of Maryland) (submittedVersion)
  • Digital Repository at the University of Maryland (University of Maryland College Park)
Funders
National Institutes of Health (R01GM126299, R35GM144083)
Topics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
  • Physical Sciences › Materials Science › Materials Chemistry › Enzyme Structure and Function
Keywords
AlphaFold, AlphaFold-Multimer, protein complex structure prediction, protein-protein interaction, immune complexes, protein-protein docking, benchmarking, multiple sequence alignment, deep learning
MeSH terms
Algorithms, Protein Binding, Protein Conformation, Proteins, Sequence Alignment, Benchmarking, Molecular Docking Simulation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review