Reinforcement Learning Based Adaptive Sampling: REAPing Rewards by Exploring Protein Conformational Landscapes
Open accessThe DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.
Abstract
One of the key limitations of Molecular Dynamics (MD) simulations is the computational intractability of sampling protein conformational landscapes associated with either large system size or long time scales. To overcome this bottleneck, we present the REinforcement learning based Adaptive samPling (REAP) algorithm that aims to efficiently sample conformational space by learning the relative importance of each order parameter as it samples the landscape. To achieve this, the algorithm uses concepts from the field of reinforcement learning, a subset of machine learning, which rewards sampling along important degrees of freedom and disregards others that do not facilitate exploration or exploitation. We demonstrate the effectiveness of REAP by comparing the sampling to long continuous MD simulations and least-counts adaptive sampling on two model landscapes (L-shaped and circular) and realistic systems such as alanine dipeptide and Src kinase. In all four systems, the REAP algorithm consistently demonstrates its ability to explore conformational space faster than the other two methods when comparing the expected values of the landscape discovered for a given amount of time. The key advantage of REAP is on-the-fly estimation of the importance of collective variables, which makes it particularly useful for systems with limited structural information.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083
Cell Signaling by Receptor Tyrosine Kinases
Lemmon, M. A., & Schlessinger, J. (2010) · Cell · doi:10.1016/j.cell.2010.06.011 · Cited by 7,929
How fast-folding proteins fold
Lindorff-Larsen, K., Piana, S., Dror, R. O., & Shaw, D. E. (2011) · Science · doi:10.1126/science.1208351 · Cited by 1,980
Sequence co-evolution gives 3D contacts and structures of protein complexes
Hopf, T. A., Schärfe, C. P. I., Rodrigues, J. P. G. L. M., Green, A. G., Kohlbacher, O., Sander, C., Bonvin, A. M. J. J., & Marks, D. S. (2014) · eLife · doi:10.7554/elife.03430 · Cited by 515
Chen, L.-Q., Cheung, L. S., Feng, L., Tanner, W., & Frommer, W. B. (2015) · Annual Review of Biochemistry · doi:10.1146/annurev-biochem-060614-033904 · Cited by 507 · cited 1×
“Furthermore, kinases are not the only biological systems that can be projected onto an L-shaped landscape using two reaction coordinates, these include membrane transporter proteins 30,31 and protein folding 1,4,8,9 (see Fig.”
BackgroundTo milliseconds and beyond: challenges in the simulation of protein folding
Lane, T. J., Shukla, D., Beauchamp, K. A., & Pande, V. S. (2013) · Current Opinion in Structural Biology · doi:10.1016/j.sbi.2012.11.002 · Cited by 409
Self-citationActivation pathway of Src kinase reveals intermediate states as targets for drug design
Shukla, D., Meng, Y., Roux, B., & Pande, V. S. (2014) · Nature Communications · doi:10.1038/ncomms4397 · Cited by 333 · cited 5×
“16,27 This system is therefore well suited to show that our algorithm is not only successful in a model system, differentiate between ECs that are actually relevant for conformational dynamics from those that are only important for protein folding.”
BackgroundMethodInfluentialSelf-citationIntrinsic map dynamics exploration for uncharted effective free-energy landscapes
Chiavazzo, E., Covino, R., Coifman, R. R., Gear, C. W., Georgiou, A. S., Hummer, G., & Kevrekidis, I. G. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1621481114 · Cited by 125
Cited by
Unsupervised Learning Methods for Molecular Simulation Data
Glielmo, A., Husic, B. E., Rodriguez, A., Clementi, C., Noé, F., & Laio, A. (2021) · Chemical Reviews · doi:10.1021/acs.chemrev.0c01195 · Cited by 402
Enhanced Sampling Methods for Molecular Dynamics Simulations [Article v1.0]
Hénin, J., Lelièvre, T., Shirts, M. R., Valsson, O., & Delemotte, L. (2022) · Living Journal of Computational Molecular Science · doi:10.33011/livecoms.4.1.1583 · Cited by 372
Machine learning approaches for analyzing and enhancing molecular dynamics simulations
Wang, Y., Lamim Ribeiro, J. M., & Tiwary, P. (2020) · Current Opinion in Structural Biology · doi:10.1016/j.sbi.2019.12.016 · Cited by 303
“[39] learns relevant CVs on-the-fly as exploration of the landscape is carried out.”
BackgroundVerkhivker, G. M., Agajanian, S., Hu, G., & Tao, P. (2020) · Frontiers in Molecular Biosciences · doi:10.3389/fmolb.2020.00136 · Cited by 76
Distinct Binding Mechanisms for Allosteric Sodium Ion in Cannabinoid Receptors
Dutta, S., Selvam, B., & Shukla, D. (2022) · ACS Chemical Neuroscience · doi:10.1021/acschemneuro.1c00760 · Cited by 31
Self-citationMolecular Mechanism of Brassinosteroid Perception by the Plant Growth Receptor BRI1
Aldukhi, F., Deb, A., Zhao, C., Moffett, A. S., & Shukla, D. (2020) · The Journal of Physical Chemistry B · doi:10.1021/acs.jpcb.9b09377 · Cited by 29
Self-citationMarkov State Model Approach to Simulate Self-Assembly
Trubiano, A., & Hagan, M. F. (2024) · Physical Review X · doi:10.1103/physrevx.14.041063 · Cited by 7
Metadata
- Authors
- Shamsi, Z., Cheng, K. J., & Shukla, D.
- Venue
- The Journal of Physical Chemistry B
- Publisher
- American Chemical Society (ACS)
- Published
- 2018-08-20
- Volume
- 122
- Issue
- 35
- Pages
- 8386-8395
- ISSN
- 1520-5207 (ISSN-L)
- 1520-6106
- OpenAlex
- W2762377283
- Cited by
- 123
- References
- 8
- FWCI
- 3.45
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.