Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 123 of the 123 references it cites were found in a registry or an index.

Differentiable biology: using deep learning for biophysics-based and data-driven modeling of molecular mechanisms

Open access
AlQuraishi, Mohammed, Sorger, Peter K. · 2021 · Nature Methods

Abstract

Deep learning using neural networks relies on a class of machine-learnable models constructed using 'differentiable programs'. These programs can combine mathematical equations specific to a particular domain of natural science with general-purpose, machine-learnable components trained on experimental data. Such programs are having a growing impact on molecular and cellular biology. In this Perspective, we describe an emerging 'differentiable biology' in which phenomena ranging from the small and specific (for example, one experimental assay) to the broad and complex (for example, protein folding) can be modeled effectively and efficiently, often by exploiting knowledge about basic natural phenomena to overcome the limitations of sparse, incomplete and noisy data. By distilling differentiable biology into a small set of conceptual primitives and illustrative vignettes, we show how it can help to address long-standing challenges in integrating multimodal data from diverse experiments across biological scales. This promises to benefit fields as diverse as biophysics and functional genomics.

Overview

Integrity signals

Registry

Crossref · 10.1038/s41592-021-01283-4

Cites

123 of 123 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 115 references

Research works printed but not linked: 42

  1. Zhang, R. Making convolutional networks shift-invariant again. Preprint at https://arxiv.org/abs/1904.11486 (2019). · arXiv
  2. Wallach, I., Dzamba, M. & Heifets, A. AtomNet: a deep convolutional neural network for bioactivity prediction in structure-based drug discovery. Preprint at https://arxiv.org/abs/1510.02855 (2015). · arXiv
  3. Gomes, J., Ramsundar, B., Feinberg, E. N. & Pande, V. S. Atomic convolutional networks for predicting protein–ligand binding affinity. Preprint at https://arxiv.org/abs/1703.10603 (2017). · arXiv
  4. Hall, B. Lie Groups, Lie Algebras, and Representations: An Elementary Introduction (Springer, 2004).
  5. Cohen, T. S., Geiger, M. & Weiler, M. A general theory of equivariant CNNs on homogeneous spaces. In Advances in Neural Information Processing Systems vol. 32 (Curran, 2019).
  6. Weiler, M., Geiger, M., Welling, M., Boomsma, W. & Cohen, T. S. 3D Steerable CNNs: learning rotationally equivariant features in volumetric data. In Advances in Neural Information Processing Systems vol. 31 (Curran, 2018).
  7. Townshend, R., Bedi, R., Suriana, P. & Dror, R. End-to-end learning on 3D protein structure for interface prediction. In Advances in Neural Information Processing Systems vol. 32 (Curran, 2019).
  8. Bialek, W. Biophysics: Searching for Principles (Princeton Univ. Press, 2012).
  9. Chen, T. Q., Rubanova, Y., Bettencourt, J. & Duvenaud, D. K. Neural ordinary differential equations. In Advances in Neural Information Processing Systems (eds. Bengio, S. et al.) 6571–6583 (Curran, 2018).
  10. Fuchs, F., Worrall, D., Fischer, V. & Welling, M. SE(3)-transformers: 3D roto-translation equivariant attention networks. In Advances in Neural Information Processing Systems vol. 33 1970–1981 (Curran, 2020).
  11. Devlin, J., Chang, M.-W., Lee, K. & Toutanova, K. BERT: pre-training of deep bidirectional transformers for language understanding. In Proc. 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers) (eds. Burstein, J., Doran, C. & Solorio, T.) 4171–4186 (Association for Computational Linguistics, 2019).
  12. Martins, A. & Astudillo, R. From Softmax to Sparsemax: a sparse model of attention and multi-label classification. In International Conference on Machine Learning 1614–1623 (PMLR, 2016).
  13. Rasmussen, C. E. & Williams, C. K. I. Gaussian Processes for Machine Learning (MIT Press, 2005). · DOI · Paper page
  14. Maclaurin, D., Duvenaud, D. & Adams, R. Gradient-based hyperparameter optimization through reversible learning. In International Conference on Machine Learning 2113–2122 (PMLR, 2015).
  15. Lorraine, J. & Duvenaud, D. Stochastic hyperparameter optimization through hypernetworks. Preprint at https://arxiv.org/abs/1802.09419 (2018). · arXiv
  16. Neil, D., Pfeiffer, M. & Liu, S.-C. Phased LSTM: accelerating recurrent network training for long or event-based sequences. In Advances in Neural Information Processing Systems vol. 29 (Curran, 2016).
  17. Bepler, T. & Berger, B. Learning protein sequence embeddings using information from structure. In International Conference on Learning Representations (2019).
  18. Liu, S., Demirel, M. F. & Liang, Y. N-gram graph: simple unsupervised representation for graphs, with applications to molecules. In Advances in Neural Information Processing Systems vol. 32 (Curran, 2019).
  19. Chithrananda, S., Grand, G. & Ramsundar, B. ChemBERTa: large-scale self-supervised pretraining for molecular property prediction. Preprint at https://arxiv.org/abs/2010.09885 (2020). · arXiv
  20. Wang, Y., Wang, J., Cao, Z. & Farimani, A. B. MolCLR: molecular contrastive learning of representations via graph neural networks. Preprint at https://arxiv.org/abs/2102.10056 (2021). · arXiv
  21. Goodfellow, I. et al. Generative adversarial nets. In Advances in Neural Information Processing Systems (eds. Ghahramani, Z. et al.) 2672–2680 (Curran, 2014).
  22. Kingma, D. P. & Welling, M. Auto-encoding variational Bayes. Preprint at https://arxiv.org/abs/1312.6114 (2013). · arXiv
  23. Karras, T., Aila, T., Laine, S. & Lehtinen, J. Progressive growing of GANs for improved quality, stability, and variation. In International Conference on Learning Representations (2018).
  24. De Cao, N. & Kipf, T. MolGAN: an implicit generative model for small molecular graphs. Preprint at https://arxiv.org/abs/1805.11973 (2018). · arXiv
  25. Killoran, N., Lee, L. J., Delong, A., Duvenaud, D. & Frey, B. J. Generating and designing DNA with deep generative models. Preprint at https://arxiv.org/abs/1712.06148 (2017). · arXiv
  26. Anand, N., Eguchi, R. & Huang, P.-S. Fully differentiable full-atom protein backbone generation. In International Conference on Learning Representations Workshop (2019).
  27. Ingraham, J., Garg, V., Barzilay, R. & Jaakkola, T. Generative models for graph-based protein design. In Advances in Neural Information Processing Systems vol. 32 (Curran, 2019).
  28. Martin, R. M. Electronic Structure: Basic Theory and Practical Methods (Cambridge University Press, 2008).
  29. OpenAI et al. Solving Rubik’s cube with a robot hand. Preprint at https://arxiv.org/abs/1910.07113 (2019). · arXiv
  30. Kulkarni, T. D., Whitney, W. F., Kohli, P. & Tenenbaum, J. B. Deep convolutional inverse graphics network. in Proc. 28th International Conference on Neural Information Processing Systems Vol. 2, 2539–2547 (MIT Press, 2015).
  31. Ingraham, J., Riesselman, A., Sander, C. & Marks, D. Learning protein structure with a differentiable simulator. In International Conference on Learning Representations (2019).
  32. Rezende, D. J., Mohamed, S. & Wierstra, D. Stochastic backpropagation and approximate inference in deep generative models. in Proceedings of the 31st International Conference on Machine Learning Vol. 32, II-1278–II-1286 (JMLR.org, 2014).
  33. Sutton, R. S. & Barto, A. G. Reinforcement Learning: An Introduction (MIT Press, 2018).
  34. Suarez, J., Du, Y., Mordach, I. & Isola, P. Neural MMO v1.3: a massively multiagent game environment for training and evaluating neural networks. In Proc. 19th International Conference on Autonomous Agents and MultiAgent Systems 2020–2022 (International Foundation for Autonomous Agents and Multiagent Systems, 2020).
  35. Cho, J., Lee, K., Shin, E., Choy, G. & Do, S. How much data is needed to train a medical image deep learning system to achieve necessary high accuracy? Preprint at https://arxiv.org/abs/1511.06348 (2016). · arXiv
  36. Lample, G. & Charton, F. Deep learning for symbolic mathematics. In International Conference for Learning Representations (2020).
  37. Grefenstette, E., Hermann, K. M., Suleyman, M. & Blunsom, P. Learning to transduce with unbounded memory. In Advances in Neural Information Processing Systems vol. 28 (Curran, 2015).
  38. Grover, A., Wang, E., Zweig, A. & Ermon, S. Stochastic optimization of sorting networks via continuous relaxations. In International Conference on Learning Representations (2018).
  39. Graves, A. Adaptive computation time for recurrent neural networks. Preprint at https://arxiv.org/abs/1603.08983 (2016). · arXiv
  40. Jin, W., Barzilay, R. & Jaakkola, T. Junction tree variational autoencoder for molecular graph generation. In International Conference on Machine Learning 2323–2332 (PMLR, 2018).
  41. Chakraborty, S. et al. Interpretability of deep learning models: a survey of results. in 2017 IEEE SmartWorld, Ubiquitous Intelligence Computing, Advanced Trusted Computed, Scalable Computing Communications, Cloud Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI) 1–6 (IEEE, 2017).
  42. Han, K. J., Chandrashekaran, A., Kim, J. & Lane, I. The CAPIO 2017 conversational speech recognition system. Preprint at https://arxiv.org/abs/1801.00059 (2018). · arXiv

Cited by

See all 96

Metadata

Authors
AlQuraishi, M., & Sorger, P. K.
Affiliations
Harvard University (US)
Corresponding author
  • AlQuraishi, Mohammed
  • Sorger, Peter K.
Venue
Nature Methods
Published
2021-10
Volume
18
Issue
10
Pages
1169-1180
ISSN
  • 1548-7091 (ISSN-L)
  • 1548-7105
OpenAlex
W3201866154
Cited by
96
References
115
Citations by year
2026: 11 · 2025: 20 · 2024: 26 · 2023: 23 · 2022: 15 · 2021: 1
FWCI
1.71
Type
journal article
Language
en
Open access
Licence
Locations
  • Nature Methods (publishedVersion)
  • PubMed (publishedVersion)
  • PubMed Central (acceptedVersion)
Funders
  • U.S. Department of Health & Human Services | NIH | National Cancer Institute (U54-CA225088)
  • United States Department of Defense | Defense Advanced Research Projects Agency (HR0011-19-2-0022)
Topics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene Regulatory Network Analysis
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
Keywords
differentiable programming, deep learning, molecular mechanisms, protein folding, multimodal data integration, functional genomics, knowledge-based modeling, sparse noisy data, artificial neural networks
MeSH terms
Deep Learning, Computational Chemistry, Biophysics, Models, Chemical, Pattern Recognition, Automated, Protein Conformation, Proteins, Neural Networks, Computer, Computational Biology

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Differentiable biology: using deep learning for biophysics-based and data-driven modeling of molecular mechanisms. https://app.cochat.ai/papers/10.1038/s41592-021-01283-4
The paper
AlQuraishi, M., & Sorger, P. K. (2021). Differentiable biology: using deep learning for biophysics-based and data-driven modeling of molecular mechanisms. Nature Methods. https://doi.org/10.1038/s41592-021-01283-4

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.