Cited by 24Protein Structure and DynamicsComputational Drug Discovery Methodsthermodynamics and calorimetric analyses

Robust prediction of relative binding energies for protein-protein complex mutations using free energy perturbation calculations

Open access
Sampson, Jared M., Cannon, Daniel A., Duan, Jianxin, Epstein, Jordan C.K., Sergeeva, Alina P., Katsamba, Phinikoula S., Mannepalli, Seetha M., Bahna, Fabiana A. +15 more·2024·Journal of Molecular Biology

Published version of a preprint

Abstract

Computational free energy-based methods have the potential to significantly improve throughput and decrease costs of protein design efforts. Such methods must reach a high level of reliability, accuracy, and automation to be effectively deployed in practical industrial settings in a way that impacts protein design projects. Here, we present a benchmark study for the calculation of relative changes in protein-protein binding affinity for single point mutations across a variety of systems from the literature, using free energy perturbation (FEP+) calculations. We describe a method for robust treatment of alternate protonation states for titratable amino acids, which yields improved correlation with and reduced error compared to experimental binding free energies. Following careful analysis of the largest outlier cases in our dataset, we assess limitations of the default FEP+ protocols and introduce an automated script which identifies probable outlier cases that may require additional scrutiny and calculates an empirical correction for a subset of charge-related outliers. Through a series of three additional case study systems, we discuss how Protein FEP+ can be applied to real-world protein design projects, and suggest areas of further study.

Integrity signals

Registry

Crossref · 10.1016/j.jmb.2024.168640

Cites

59 of 59 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

14 of 22 cited sources checked · 7 supported · 0 flagged · accuracy not yet measured

  • Findingcitation

    FEP+ results from longer simulation and rigorous treatment of titratable residues yielded accuracy consistent with previous studies, with RMSE about 1.0-1.5 kcal/mol.

    “The FEP+ results from longer simulation time and rigorous treatment of titratable residues yielded accuracy consistent with results from previous studies, with RMSE in the range of about 1.0–1.5 kcal/mol [17,18].”
    Quote in the paperSubstantial claim2 of 2 sources checked
    • cites [17] source

      The Extended sampling section reports that longer simulations improved accuracy for tryptophan mutations. But the Summary of results using recommended protocol gives an overall RMSE of 0.70 kcal/mol, not about 1.0–1.5 kcal/mol. The Discussion treats methods for handling protonation-state changes as work still needed, rather than a rigorous treatment used to obtain these results. Page numbers are unavailable in our copy. (LLM, conf=0.88)

      Experimental — accuracy not yet measured
    • cites [18] source

      Table 2 (Results and Discussion; page unavailable) reports FEP RMSEs of 1.41 kcal/mol overall and 1.23 kcal/mol for non-buried mutations, within the claimed range, but 1.79 kcal/mol for buried mutations. The Methods describe extended simulations, and the discussion of buried sites describes explicit modeling of protonation states; however, it calls the pKa correction approximate, so “rigorous treatment” overstates that aspect of the source. (LLM, conf=0.85)

      Experimental — accuracy not yet measured
    • Rests on an assumption: Experimental binding affinity values from ITC or SPR are accurate enough to serve as ground truth for assessing FEP+ prediction error.
  • Premisecitation

    FEP+ outperformed several other computational methods in probing effects of mutations on the SARS-CoV-2 spike protein and human ACE2 interaction.

    “It has also been used recently to probe the effects of mutations on the interaction between the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) viral spike protein and the human angiotensin converting enzyme 2 (ACE2) receptor, where it outperformed several other computational methods, as determined both by the agreement of its results with experiment and by its ability to identify st”
    Quote in the paperSubstantial claim1 of 1 sources checked
  • Interpretationcitation

    The large unfavorable experimental binding ΔΔG for 6SBA B:T20A suggests that a substantial rearrangement occurs for the unbound mutant peptide, not sampled in simulation, incurring a reorganization penalty to adopt the binding-competent conformation.

    “The large unfavorable experimental binding ΔΔG, in this case, suggests that—as previously hypothesized [47]—a substantial rearrangement occurs for the unbound mutant peptide in the physical system, which was not sampled in simulation, and which would incur a reorganization penalty to adopt the binding-competent conformation present in the complex structure and FEP+ calculations.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [47] source

      Split into 2 atoms — 0 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 1 unverifiable. Expand the claim to see per-atom verdicts.

      Experimental — accuracy not yet measured
  • Premisecitation

    The intrinsic uncertainty expected from experimental protein-protein and protein-ligand SPR binding measurements has been proposed to be 0.4-0.9 kcal/mol.

    “This level of accuracy is approaching the intrinsic uncertainty expected from experimental results, which has been proposed to be in the range of 0.4–0.9 kcal/mol for protein-protein and protein-ligand SPR binding measurements [34,39,48].”
    Quote in the paperSubstantial claim2 of 3 sources checked
    • cites [34] source
    • cites [39] source

      The Methods (Surface plasmon resonance; page unavailable) estimate an experimental error of approximately 0.4 kcal/mol for this study’s RBD–ACE2 SPR binding measurements, and the Discussion calls that a typical reproducibility error even for high-quality SPR results. The paper does not propose a 0.4–0.9 kcal/mol uncertainty range for protein-protein and protein-ligand SPR measurements; its nearby 0.5–1 kcal/mol range describes computational RMS errors, not experimental uncertainty. (LLM, conf=0.88)

      Experimental — accuracy not yet measured
    • cites [48] source

      The paper reports protein–ligand experimental variability near the proposed range: 0.41 kcal/mol for repeated measurements in the background discussion and 0.91 kcal/mol for relative-affinity reproducibility in the experimental survey (Validation of the experimental error estimates; page unavailable). It does not establish a 0.4–0.9 kcal/mol range specifically for SPR measurements or for protein–protein binding. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
  • Interpretationcitation

    These results are consistent with previous protein FEP+ results where accuracy has been at or near 1 kcal/mol.

    “These results are consistent with previous protein FEP+ results where accuracy has been at or near the 1 kcal/mol level[17,18,35,36,39,40].”
    Quote in the paperNot a substantial claim4 of 6 sources checked
    • cites [17] source
    • cites [18] source
    • cites [35] source
    • cites [36] source
    • cites [39] source
    • cites [40] source
  • Classified as probably not a substantial claim. 10
  • May combine two claims. 38
  • The quote only partly states this claim. 2
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    FEP+ has demonstrated high accuracy and reliability in predicting binding potencies of small molecules with their protein targets.

    “Over the past decade FEP+ has demonstrated a high level of accuracy and reliability in predicting the binding potencies of small molecules with their protein targets, both via relative binding FEP for congeneric ligands in lead optimization and via absolute binding FEP for diverse ligands in hit discovery [31,32].”
    Quote in the paperNot a substantial claim1 of 2 sources checked
    • cites [31] source

      The Introduction (p. 2) describes free-energy calculations as a preferred approach to estimating ligand binding affinity, and Section 2 (p. 3) discusses studies assessing their accuracy and reliability across many ligands and targets. The review does not establish the citing sentence’s more specific claim that FEP+ demonstrated high accuracy and reliability through both relative calculations in lead optimization and absolute calculations for diverse ligands in hit discovery; its absolute-affinity example in Section 3 (p. 4) is a toy model. (LLM, conf=0.82)

      Experimental — accuracy not yet measured
    • cites [32]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim.

    Premisecitation

    FEP+ has been adapted to predict effects of neutral and charge-changing perturbations on protein-protein binding affinity.

    “We have previously published proof-of-concept studies for the application of FEP+ to predict the effects of neutral and charge-changing perturbations on protein-protein binding affinity [17,18]”
    Quote in the paperSubstantial claim2 of 2 sources checked
    • cites [17] source

      The Conclusions describe a proof of concept for predicting protein–protein binding affinities from alanine-scan mutations. They explicitly state that the paper had not yet demonstrated treatment of mutations that change a protein’s net charge, so this source supports the neutral-perturbation part but not the charge-changing part. (LLM, conf=0.91)

      Experimental — accuracy not yet measured
    • cites [18] source
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Proline mutations have been treated before in protein stability FEP+ with reasonable success.

    “Proline mutations have been treated before in protein stability FEP+ with reasonable success [35].”
    Quote in the paperNot a substantial claim1 of 1 sources checked
  • May combine two claims.

    Findingown data

    In a hypothetical restrictive application on the full benchmark set, testing only cases with calculated ΔΔG of −0.5 kcal/mol or less would result in 13 variants tested, a 31% positive hit rate, and recovery of 4 of 8 actives (50%).

    “if we were to take the full benchmark set as a hypothetical example and consider the most restrictive application of these classification results with only cases having calculated ΔΔG of −0.5 kcal/mol or less (using the unflagged 100 ns cases after FEP+ Groups treatment of protonation states and automated outlier flagging and empirical correction) to be tested by experiment, this would result in”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    1JRH I:G50A (experimental +4.53, predicted +1.49), I:E55A (experimental -0.43, predicted +1.74), and I:E55P (experimental -3.79, predicted +1.63) involve mutations on the flexible I:50-56 loop of IFNγR1.

    “The 1JRH I:G50A (expt. +4.53, pred. +1.49), I:E55A (expt. −0.43, pred. +1.74), and I:E55P (expt. −3.79, pred +1.63) cases, which are also depicted in Supplemental Figure 4, each involve mutation of a residue on the flexible I:50–56 loop of the interferon γ receptor 1 (IFNγR1) antigen.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Experimental binding affinity values from ITC or SPR are accurate enough to serve as ground truth for assessing FEP+ prediction error.
  • May combine two claims.

    Findingown data

    In the case studies, 4 of 5 outlier cases were automatically flagged and 3 of those were empirically corrected.

    “The effect of the automated outlier flagging and empirical correction procedure was also similar, with 4 out of 5 outlier cases being automatically flagged and 3 of those empirically corrected.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    The Protein FEP+ Groups treatment improves correlation with and reduces error relative to experimental binding free energies.

    “We describe a method for robust treatment of alternate protonation states for titratable amino acids, which yields improved correlation with and reduced error compared to experimental binding free energies.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    On average, 100 ns FEP+ simulations produced smaller overall errors but similar correlation coefficients compared to 10 ns simulations.

    “We found that on average the longer 100 ns simulations resulted in smaller overall errors, but similar correlation coefficients, compared to the more typical 10 ns simulations.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    Naïve FEP+ treatment at 10 ns yielded RMSE 1.65 kcal/mol and R2 0.29 against experimental binding ΔΔG values.

    “Overall root mean square error (RMSE) for these naïve calculated ΔΔG values compared to experimentally measured values are 1.65 and 1.42 kcal/mol, and coefficient of determination (R2) values are 0.29 and 0.27, for 10 and 100 ns results, respectively.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Experimental binding affinity values from ITC or SPR are accurate enough to serve as ground truth for assessing FEP+ prediction error.
  • May combine two claims.

    Findingown data

    Six of the salt-bridge cases disrupted trans salt bridges (1IAR A:R81A/E, A:R85A/E, A:R88A/Q), while one (1VFB C:D119A) disrupted a cis salt bridge.

    “Six of the cases (which were all mutations at one of three Arg positions in the 1IAR system: A:R81A/E, A:R85A/E, and A:R88A/Q) disrupted trans salt bridges, that is, salt bridges involving residues from both protein binding components, whereas one case (1VFB C:D119A) disrupted a cis salt bridge, between two residues from the same side of the interface.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    FEP+ with explicit polarization improved the predicted ΔΔG for 1JRH I:N53A to +3.09 kcal/mol (error -0.8), suggesting explicit polarization may improve similar cases.

    “The FEP+ results with explicit polarization resulted in substantial improvement of the predicted binding ΔΔG, to +3.09 kcal/mol (an error of −0.8 compared to experiment). This preliminary result reflects an increased stabilization of the wt complex by more accurately modeling the energetics of the π-hydrogen bond between the Asn and Trp sidechains, and suggests the inclusion of explicit”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Experimental binding affinity values from ITC or SPR are accurate enough to serve as ground truth for assessing FEP+ prediction error.
  • May combine two claims.

    Findingown data

    All seven salt-bridge-elimination cases were unfavorable experimentally, with ΔΔG from +0.43 to +3.75 kcal/mol, but FEP+ predicted them too unfavorable with signed errors from +2.3 to +4.5 kcal/mol.

    “All seven cases were unfavorable experimentally, with binding ΔΔG s ranging from +0.43 to +3.75 kcal/mol, but FEP+ consistently predicted them to be too unfavorable with signed errors ranging from +2.3 to +4.5 kcal/mol.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Experimental binding affinity values from ITC or SPR are accurate enough to serve as ground truth for assessing FEP+ prediction error.
  • May combine two claims.

    Findingown data

    For 1IAR A:W91D, experimental ΔΔG was +1.31 kcal/mol, naïve prediction +3.42, and FEP+ Groups prediction +1.99, reducing absolute error from 2.11 to 0.68 kcal/mol.

    “Experimentally the Asp mutation yielded a binding ΔΔG of +1.31 kcal/mol, but the naïve prediction of +3.42 was overly unfavorable.”
    • Also stated in: § Effect of Protein FEP+ Groups treatment · plain
      “Accounting for this change in dominant protonation state, the FEP+ Groups treatment yielded a more accurate predicted ΔΔG of +1.99 kcal/mol, corresponding to a decrease in absolute error from 2.11 to 0.68 kcal/mol.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Experimental binding affinity values from ITC or SPR are accurate enough to serve as ground truth for assessing FEP+ prediction error.
    • Rests on an assumption: The initial protonation states assigned to titratable residues in the prepared models are appropriate for the experimental pH.
  • May combine two claims.

    Findingown data

    Naïve FEP+ treatment at 100 ns yielded RMSE 1.42 kcal/mol and R2 0.27 against experimental binding ΔΔG values.

    “Overall root mean square error (RMSE) for these naïve calculated ΔΔG values compared to experimentally measured values are 1.65 and 1.42 kcal/mol, and coefficient of determination (R2) values are 0.29 and 0.27, for 10 and 100 ns results, respectively.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Experimental binding affinity values from ITC or SPR are accurate enough to serve as ground truth for assessing FEP+ prediction error.
  • May combine two claims.

    Findingown data

    For 6NRQ C:Q138D, experimental ΔΔG was +1.54 kcal/mol, naïve prediction +4.09, and FEP+ Groups prediction +0.75, reducing error from 2.55 to 0.79 kcal/mol.

    “This mutation was also unfavorable experimentally (+1.54 kcal/mol), but the naïve predicted value was again too highly destabilizing (+4.09).”
    • Also stated in: § Effect of Protein FEP+ Groups treatment · plain
      “FEP+ Groups treatment produced a predicted ΔΔG of +0.75, reducing the error from 2.55 to 0.79 kcal/mol.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Experimental binding affinity values from ITC or SPR are accurate enough to serve as ground truth for assessing FEP+ prediction error.
    • Rests on an assumption: The initial protonation states assigned to titratable residues in the prepared models are appropriate for the experimental pH.
  • May combine two claims.

    Findingown data

    For 1DVF B:E98A, experimental ΔΔG was +4.19 kcal/mol and predicted -0.05, an absolute error of 4.24 kcal/mol; wild-type B:Glu98 forms a salt bridge with A:Arg96 across the VH-VL interface.

    “Specifically, for 1DVF B:E98A (expt. +4.19; pred. −0.05), in the wt structure residue B:Glu98 forms a salt bridge with A:Arg96 across the internal VH−VL heterodimerization interface, between the heavy and light chain variable domains of the antibody Fv region.”
    • Also stated in: § Outliers due to insufficient sampling and other conformational issues · plain
      “resulting in an underprediction of the unfavorability of the mutation and a large absolute error of 4.24 kcal/mol.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Experimental binding affinity values from ITC or SPR are accurate enough to serve as ground truth for assessing FEP+ prediction error.
    • Rests on an assumption: The PDB crystal structures used as starting models accurately represent the relevant experimental protein-protein complexes.
  • May combine two claims.

    Findingown data

    1JRH I:G50 has a high-energy backbone conformation resembling a γ-turn and is a Ramachandran outlier; two other crystal structures show IFNγR1 adopting an alternative conformation of the I:50-56 loop.

    “The first of these, 1JRH I:G50, is located in a tight turn with geometry resembling a γ-turn, and backbone φ and ψ angles that make it an outlier in the Ramachandran plot for the crystal structure (Supplemental Figure 4 a–b) This high-energy backbone conformation would require substantial relaxation to achieve the true lowest energy conformation. Indeed, two other crystal structures (1FYH chain B,”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The PDB crystal structures used as starting models accurately represent the relevant experimental protein-protein complexes.
  • May combine two claims.

    Findingown data

    Charge-flipping cases 1IAR A:K12E and A:K84D were unfavorable experimentally and gave signed errors greater than +1 kcal/mol, suggesting buried unpaired charges from charge flipping also tend to be undersampled.

    “the other two (1IAR A:K12E and A:K84D), were also unfavorable experimentally and gave signed errors greater than +1 kcal/mol, suggesting that these types of buried (or mostly buried) unpaired charges resulting from charge flipping perturbations also tend to be undersampled.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Experimental binding affinity values from ITC or SPR are accurate enough to serve as ground truth for assessing FEP+ prediction error.
  • May combine two claims.

    Findingown data

    The case studies dataset had an initial RMSE of 1.32 and an initial R2 of 0.15 for the full FEP+ Groups-treated 100 ns dataset, with the low R2 due to a large proline mutation outlier.

    “The results for the case studies dataset were similar to those from the benchmark set, with initial RMSE of 1.32, though due to a large proline mutation outlier, a relatively low initial R2 of 0.15 for the full FEP+ Groups-treated 100 ns dataset.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingreasoning

    For 1JRH I:N53A, experimental ΔΔG was +3.89 kcal/mol and predicted +1.37; the mutation affects a π-hydrogen bond between the Asn sidechain amide and the π cloud of I:Trp96, and fixed-charge OPLS4 could not account for this interaction, making the prediction too favorable by 2.5 kcal/mol.

    “the case in question, 1JRH I:N53A (expt. +3.89, pred. +1.37), involved mutation of a residue that participates in a well-defined π interaction in the wt crystal structure.”
    • Also stated in: § Outlier attributed to force field limitations · plain
      “Due to the fixed-charge nature of the OPLS4 force field, it was not possible for the FEP+ calculation to account for the contribution of this π interaction to the stability of the wt complex, which resulted in a predicted binding ΔΔG value that was too favorable by 2.5 kcal/mol.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Experimental binding affinity values from ITC or SPR are accurate enough to serve as ground truth for assessing FEP+ prediction error.
    • Rests on an assumption: The PDB crystal structures used as starting models accurately represent the relevant experimental protein-protein complexes.
  • May combine two claims.

    Findingown data

    1DVF D:Y98A (experimental +4.74, predicted +1.32) and D:R100BY (experimental +4.09, predicted +1.2) are outliers in the CDR H3 loop; reprediction identified a lower-energy loop conformation in the unbound antibody.

    “A final sampling-related outlier class, shown in Figure 4(f), involved mutations in the 1DVF antibody CDR H3 loop, D:Y98A (expt. +4.74, pred. +1.32) and D:R100BY (expt. +4.09, pred. +1.2).”
    • Also stated in: § Outliers due to insufficient sampling and other conformational issues · plain
      “For each of these cases, we repredicted the conformation of the CDR H3 loop in the unbound free antibody, and in each case identified a different loop conformation with lower energy than the crystal structure-like conformation (Supplemental Figure 5).”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Experimental binding affinity values from ITC or SPR are accurate enough to serve as ground truth for assessing FEP+ prediction error.
    • Rests on an assumption: The PDB crystal structures used as starting models accurately represent the relevant experimental protein-protein complexes.
  • May combine two claims. The quote only partly states this claim.

    Findingown data

    In a hypothetical less stringent application including neutral mutations (calculated ΔΔG of +0.5 kcal/mol or less), testing would result in 100 variants measured, a 7% positive hit rate, 88% recovery, 7 of 8 true actives classified as neutral or favorable, 1 false negative, and a 50% reduction in experimental mutations.

    “If we were to use a less stringent approach and include neutral mutations—that is, proceeding with experimental testing of all cases with calculated ΔΔG of +0.5 kcal/mol or less, this would result in experimental measurement of 100 variants, a positive hit rate of 7%, and recovery of 88%, with 7 out of 8 true actives in the experimental dataset classified as neutral or favorable in FEP+”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    Applying the FEP+ Groups treatment decreased RMSE and increased PCC and R2 at both 10 and 100 ns.

    “After applying the FEP+ Groups treatment to the retrospective calculations, we observed decreased RMSE and increased Pearson correlation coefficient (PCC) and R2 at both 10 and 100 ns time points.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Experimental binding affinity values from ITC or SPR are accurate enough to serve as ground truth for assessing FEP+ prediction error.
  • May combine two claims. The quote only partly states this claim.

    Findingown data

    For 1DVF D:H33A, experimental ΔΔG was +1.86 kcal/mol and predicted -0.26; using the final frame of an MD trajectory as input, FEP+ predicted +1.56 kcal/mol, within 0.30 of experimental.

    “In the 1DVF D:H33A case (expt. +1.86, pred. −0.26), which is presented in more detail in Supplemental Figure 3, the geometry of the wt D:His55 residue (modeled as HIP) from the crystal structure showed its Nε proton interacting with two side chain carboxylate groups, from D:Asp52 (in cis) and B:Asp100 (in trans).”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Experimental binding affinity values from ITC or SPR are accurate enough to serve as ground truth for assessing FEP+ prediction error.
    • Rests on an assumption: The PDB crystal structures used as starting models accurately represent the relevant experimental protein-protein complexes.
  • May combine two claims.

    Findingown data

    1JRH I:E55 has a highly solvent-exposed side chain making no direct contact with antibody in the wild-type structure; it is about 6 Å from H:Lys64 and the N-terminal L:Ser1, and FEP+/MD trajectories show it interacting with the charged N-terminus of chain L.

    “The second mutation site on this loop, 1JRH I:E55, has a side chain that is highly solvent exposed in the complex, making no direct contact with any antibody residue in the wt crystal structure. It is approximately 6 Å from two different primary amino groups—namely the solvent-exposed (and therefore flexible) H:Lys64 side chain and the N-terminal amino group from L:Ser1—and FEP+ and MD”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The PDB crystal structures used as starting models accurately represent the relevant experimental protein-protein complexes.
  • May combine two claims.

    Findingown data

    After empirical correction of charged outlier cases and exclusion of the flagged proline mutation case, the case studies RMSE improved to 0.79 kcal/mol and R2 increased to 0.46.

    “After empirical correction of the charged outlier cases and exclusion of the flagged proline mutation case, RMSE was improved to 0.79 kcal/mol, and R2 increased substantially to 0.46.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    Among cases involving titratable residues, FEP+ Groups treatment decreased RMSE from 2.33 to 1.48 kcal/mol at 10 ns and from 1.83 to 1.35 kcal/mol at 100 ns.

    “among these cases involving titratable residues, FEP+ Groups treatment resulted in decreased RMSE at both timepoints, from 2.33 to 1.48 kcal/mol at 10 ns, and from 1.83 to 1.35 kcal/mol at 100 ns.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Experimental binding affinity values from ITC or SPR are accurate enough to serve as ground truth for assessing FEP+ prediction error.
  • May combine two claims.

    Findingown data

    For 1IAR A:W91D, FEP+ Groups pKa calculations indicated the unbound ASP form was dominant with pKa 6.5, while the bound state preferred neutral ASH with pKa 8.5.

    “In the unbound state, the deprotonated ASP form was calculated to be dominant, with an A:D91 side chain pKa of 6.5; whereas in the bound state, a higher complex pKa of 8.5 indicated a preference for the neutral ASH form.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The initial protonation states assigned to titratable residues in the prepared models are appropriate for the experimental pH.
  • May combine two claims.

    Findingown data

    A number of mutations exhibited absolute errors in binding ΔΔG larger than 2 kcal/mol compared to experiment, which would reduce prediction accuracy in prospective studies.

    “However, a number of mutations exhibited absolute errors in binding ΔΔG larger than 2 kcal/mol compared to experiment, which would reduce the prediction accuracy in prospective studies.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Experimental binding affinity values from ITC or SPR are accurate enough to serve as ground truth for assessing FEP+ prediction error.
  • May combine two claims.

    Findingreasoning

    Three cases (1JRH L:Y91A and H:H100BA; 1VFB B:W52A) involved mutation of larger buried aromatic residues to alanine; all were modestly unfavorable experimentally (+0.6 to +1.7 kcal/mol), but FEP+ overpredicted unfavorability by 3-5 kcal/mol.

    “Figure 4(d) depicts an outlier class consisting of three cases (1JRH L:Y91A and H:H100BA; and 1VFB B:W52A) which involved mutations of larger aromatic residues, buried in the complex structure, to much smaller side chains, namely alanine. All three were modestly unfavorable experimentally (+0.6 to +1.7 kcal/mol), but FEP+ overpredicted the unfavorability by 3–5 kcal/mol.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Experimental binding affinity values from ITC or SPR are accurate enough to serve as ground truth for assessing FEP+ prediction error.
  • May combine two claims.

    Findingown data

    For 6NRQ C:Q138D, FEP+ Groups pKas were 8.4 unbound and 10.9 bound, elevated relative to isolated Asp pKa 3.67, indicating neutral ASH was preferred in both states at pH 7.2.

    “Due to hydrophobic contacts and the proximity of C:Glu136, FEP+ Groups-calculated pKas for both physical macrostates were significantly elevated compared to the pKa value of 3.67 for an isolated Asp in solution [45], with calculated pKa values of 8.4 (unbound) and 10.9 (bound), thus indicating the neutral ASH form was preferred in both unbound and bound states at the experimental pH of 7.2.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [45] source
    • Rests on an assumption: The initial protonation states assigned to titratable residues in the prepared models are appropriate for the experimental pH.
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Both proline cases in the datasets considered here were mutations to or from proline in flexible loops at the binding interface.

    “We note that both proline cases in the datasets considered here (one each in the benchmark and case studies sets) were mutations to or from proline in flexible loops at the binding interface.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    In a hypothetical less stringent approach for the case studies, testing would result in 28 variants, a 25% positive hit rate, 100% recovery, 7 of 7 true actives classified as neutral or favorable, no false negatives, and a 32% reduction in experimental mutations.

    “Using the less stringent approach and including neutral mutations would result in experimental measurement of 28 variants, a positive hit rate of 25%, and recovery of 100%, with 7 out of 7 true actives in the experimental subset corresponding to neutral or favorable FEP+ predictions, and leaving no false negatives, while reducing the number of experimental mutations by 32%.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingreasoning

    In a hypothetical strict-cutoff design project for the case studies, testing would result in 8 variants, a 50% positive hit rate, and recovery of 4 of 7 actives (57%).

    “Treated collectively as a hypothetical design project, using the same strict cutoff criteria as described in the previous section, this classification would result in the experimental testing of only 8 variants, with a positive hit rate of 50%, and a recovery of 4 out of 7 actives, or 57%.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    Typical standard deviations among triplicate model leg FEP calculations were 0.02-0.03 kcal/mol, indicating good convergence among replicates.

    “Typical standard deviations were on the order of 0.02–0.03 kcal/mol, indicating good convergence among the replicates.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    FEP+ has been demonstrated to calculate protein residue pKa values and free energies between different receptor conformations with high accuracy.

    “FEP+ has been demonstrated to calculate both protein residue pKa values and free energies between different receptor conformations with high accuracy [37,38].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [37] source
    • cites [38] source
  • May combine two claims.

    Interpretationreasoning

    For mutation of any non-contact residue to yield a binding ΔΔG significantly different from zero requires conformational rearrangement, and accurate simulation would need sampling beyond the standard FEP+ workflow, making such cases not typically suitable for standard protocols.

    “More generally, for mutation of any non-contact residue in a crystal structure to yield a binding ΔΔG significantly different from zero essentially requires some conformational rearrangement, either in the unbound form, the bound form, or both. Accurate simulation of such conformational changes would necessitate sampling beyond what is achievable in the standard protein binding FEP+ workflow, thus”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim.

    Interpretationnone given

    The preferred protonation states of titratable residues may differ between the bound protein-protein complex and the unbound protein in solution.

    “The preferred protonation states of these titratable residues might differ between the relevant physical macrostates, namely the bound protein-protein complex and the unbound protein in solution.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationreasoning

    For 1DVF B:E98A, the antibody Fv must retain or adopt the binding-competent crystal conformation to bind antigen, but the unbound mutant Fv would relax to a lower-energy conformation to alleviate the buried charge at the VH−VL interface, which was not sampled on the simulation timescale.

    “Conversely, for 1DVF B:E98A (which was the only example among these cases with a negative signed error), the antibody Fv must retain or adopt the binding-competent conformation observed in the crystal structure in order to bind the antigen, but in order to alleviate the buried charge at the VH−VL interface, the unbound antibody Fv region in the physical system would readily relax to a lower-energy”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationown data

    For a majority of mutations from a variety of systems, protein binding FEP+ calculations can produce accurate results and may provide insight suitable for guiding the experimental component of biologics design projects.

    “In summary, the results presented here demonstrate that, for a majority of mutations from a variety of systems, protein binding FEP+ calculations are able to produce accurate results and may provide insight suitable for guiding the experimental component of biologics design projects.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The benchmark dataset is representative of protein-protein complex mutations encountered in biologics design.
  • May combine two claims.

    Premisereasoning

    The default OPLS4 force field used in FEP+ calculations uses fixed charges only and cannot account for π-based interactions resulting from induced polarization.

    “The current default OPLS4 force field used in FEP+ calculations uses fixed charges only. This leaves it unable to account for π-based interactions, which result from induced polarization upon movement of electronic negative charge relative to the atomic nuclei.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    One reason proline is challenging may be that binding FEP requires two different protein contexts in the solvent and complex legs, whereas stability calculations have a linear peptide in the unbound solvent leg with fewer degrees of freedom and greater likelihood of adequate conformational sampling.

    “One reason may be that the solvent and complex legs of binding FEP calculations necessitate two different protein contexts, whereas stability calculations have a linear peptide in the unbound solvent leg with far fewer degrees of freedom and therefore a greater likelihood of achieving adequate sampling of conformational space, similarly to the unbound ligand in a small molecule FEP calculation.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Interpretationreasoning

    The lack of explicit π interaction treatment has two effects: π interaction energetics do not contribute to FEP+ reported energies, and the lack of stabilization could increase sampling of less relevant conformations, adding energetic noise.

    “The effect here is twofold: first, the energetics of the interactions do not contribute to the energies reported by FEP+ calculations; and second, the lack of explicit stabilization of these interactions in simulation could lead to increased sampling of less relevant conformations, thereby adding energetic noise to the results.”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationown data

    The automatic probable outlier flagging and empirical correction script may serve to prioritize FEP+ results, allowing unflagged or corrected cases to be treated as higher confidence and used more directly in downstream experimental validation.

    “The automatic probable outlier flagging and empirical correction script may serve as a means of prioritizing FEP+ results, such that unflagged or corrected cases may be treated as higher confidence and used more directly as input to a downstream experimental validation pipeline.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Protein reorganization FEP has been applied successfully to account for loop conformation changes in absolute binding FEP calculations with protein-ligand complexes.

    “Specifically, protein reorganization FEP has been applied successfully to account for changes in loop conformation in absolute binding FEP calculations with protein-ligand complexes [38].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [38] source
45 of 45
  • The paper relies on this without support.

    Experimental binding affinity values from ITC or SPR are accurate enough to serve as ground truth for assessing FEP+ prediction error.

    “To increase the reliability of the experimental values, we limited the dataset to include only measurements made by isothermal calorimetry (ITC) or surface plasmon resonance (SPR).”
  • The paper relies on this without support.

    The PDB crystal structures used as starting models accurately represent the relevant experimental protein-protein complexes.

    “using all-atom structural models derived from the structures deposited with the RCSB Protein Data Bank (PDB), and prepared as described in the Methods section”
  • The paper relies on this without support.

    The initial protonation states assigned to titratable residues in the prepared models are appropriate for the experimental pH.

    “including the addition of hydrogen atoms and assignment of protonation states expected to be dominant in the bound complex at the pH of the experimental measurements.”
  • The paper relies on this without support.

    The experimental binding affinities used as reference are accurate enough to evaluate the FEP+ predictions.

    “For comparison to FEP-calculated binding free energies, experimental binding affinities (KD) were converted to ΔG values (in kcal/mol) via the standard relation”
    “The experimental dataset included SPR measurements for wt and single-residue mutation variants for two antibody-antigen systems and one peptide-target system.”
  • The paper relies on this without support.

    The benchmark dataset is representative of protein-protein complex mutations encountered in biologics design.

    “Here, we have presented results from protein binding FEP+ calculations for a benchmark dataset with a variety of systems.”
    “In summary, the results presented here demonstrate that, for a majority of mutations from a variety of systems, protein binding FEP+ calculations are able to produce accurate results”
5 of 5

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Different residue numbers are given for the same outlier

    Minor

    The outlier is identified as mutation D:H33A, but the description calls the wild-type residue D:His55. The same account later identifies the starting conformation as D:His33, so the residue number is inconsistent.

  • Force-field outlier has conflicting figure references

    Minor

    The force-field outlier is assigned to Figure 4(g) in Results but to Figure 4(e) in Discussion. Results assigns Figure 4(e) to a different outlier class involving starting conformations.

2 of 2

Cites

Flagged first, then by citations

1 of the 60 references couldn’t be matched to a known paper.

Research works printed but not linked: 1

  1. Allaire, 2022, Quarto

Cited by

Sorted by citations

Authors
Sampson, J. M., Cannon, D. A., Duan, J., Epstein, J. C. K., Sergeeva, A. P., Katsamba, P. S., Mannepalli, S. M., Bahna, F. A., Adihou, H., Guéret, S. M., Gopalakrishnan, R., Geschwindner, S., Rees, D. G., Sigurdardottir, A., Wilkinson, T., Dodd, R. B., De Maria, L., Mobarec, J. C., Shapiro, L., Honig, B., Buchanan, A., Friesner, R. A., & Wang, L.
Affiliations
  • Schrodinger (United States) (US)
  • Columbia University (US)
  • Mortimer B. Zuckerman Mind Brain Behavior Institute (US)
  • Max Planck Institute of Molecular Physiology (DE)
  • AstraZeneca (Sweden) (SE)
  • AstraZeneca (United Kingdom) (GB)
Corresponding author
  • Friesner, Richard A.
  • Wang, Lingle
Venue
Journal of Molecular Biology
Publisher
Elsevier BV
Published
2024-08
Volume
436
Issue
16
Pages
168640
ISSN
0022-2836 (ISSN-L)
OpenAlex
W4399356565
Cited by
24
References
60
Citations by year
2026: 10 · 2025: 10 · 2024: 3 · 2022: 1
FWCI
3.12
Type
journal article
Language
en
Open access
Locations
  • Journal of Molecular Biology (acceptedVersion)
  • bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • PubMed Central (acceptedVersion)
  • PubMed Central (submittedVersion, cc-by-nc-nd)
Funders
  • National Institutes of Health (R35-GM139585)
  • Bill & Melinda Gates Foundation (INV-016167)
Topics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
  • Physical Sciences › Computer Science › Computational Theory and Mathematics › Computational Drug Discovery Methods
  • Physical Sciences › Chemistry › Physical and Theoretical Chemistry › thermodynamics and calorimetric analyses
Keywords
protein-protein binding affinity, point mutations, free energy perturbation, FEP, protonation states, benchmarking, outlier detection, protein design, binding free energy

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read under definition version 3 (extract-v3) on 2026-09-28.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-09-28.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review