The DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.
A review of automatic differentiation and its efficient implementation
Open accessAbstract
Derivatives play a critical role in computational statistics, examples being Bayesian inference using Hamiltonian Monte Carlo sampling and the training of neural networks. Automatic differentiation (AD) is a powerful tool to automate the calculation of derivatives and is preferable to more traditional methods, especially when differentiating complex algorithms and mathematical functions. The implementation of AD, however, requires some care to insure efficiency. Modern differentiation packages deploy a broad range of computational techniques to improve applicability, run time, and memory management. Among these techniques are operation overloading, region‐based memory, and expression templates. There also exist several mathematical techniques which can yield high performance gains when applied to complex algorithms. For example, semi‐analytical derivatives can reduce by orders of magnitude the runtime required to numerically solve and differentiate an algebraic equation. Open and practical problems include the extension of current packages to provide more specialized routines, and finding optimal methods to perform higher‐order differentiation. This article is categorized under: Algorithmic Development > Scalable Statistical Methods
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ggplot2: Elegant Graphics for Data Analysis
Wickham, H. (2009) · doi:10.1007/978-0-387-98141-3 · Cited by 14,447
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408
Stan : A Probabilistic Programming Language
Carpenter, B., Gelman, A., Hoffman, M. D., Lee, D., Goodrich, B., Betancourt, M., Brubaker, M., Guo, J., Li, P., & Riddell, A. (2017) · Journal of Statistical Software · doi:10.18637/jss.v076.i01 · Cited by 7,754 · cited 2×
“NUTS, supported by AD, indeed provides the algorithmic foundation for the probabilistic languages Stan (Carpenter et al., 2017) and PyMC3 (Salvatier, Wiecki, & Fonnesbeck, 2016).”
BackgroundCasADi: a software framework for nonlinear optimization and optimal control
Andersson, J. A. E., Gillis, J., Horn, G., Rawlings, J. B., & Diehl, M. (2019) · Mathematical Programming Computation · doi:10.1007/s12532-018-0139-4 · Cited by 3,868
The No-U-turn sampler: adaptively setting path lengths in Hamiltonian Monte CarloPreprint
Homan, M. D., & Gelman, A. (2014) · arXiv (Cornell University) · Cited by 3,405
MCMC Using Hamiltonian Dynamics
Neal, R. M. (2011) · Handbook of Markov Chain Monte Carlo · doi:10.1201/b10905-6 · Cited by 2,574
Probabilistic programming in Python using PyMC3
Salvatier, J., Wiecki, T. V., & Fonnesbeck, C. (2016) · PeerJ Computer Science · doi:10.7717/peerj-cs.55 · Cited by 2,544
Nineteen Dubious Ways to Compute the Exponential of a Matrix, Twenty-Five Years Later
Moler, C., & Van Loan, C. (2003) · SIAM Review · doi:10.1137/s00361445024180 · Cited by 2,357
30 years of adaptive neural networks: perceptron, Madaline, and backpropagation
Widrow, B., & Lehr, M. A. (1990) · Proceedings of the IEEE · doi:10.1109/5.58323 · Cited by 2,242
Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation
Griewank, A., & Walther, A. (1987) · Cited by 2,161
Cited by
DeepXDE: A Deep Learning Library for Solving Differential Equations
Lu, L., Meng, X., Mao, Z., & Karniadakis, G. E. (2021) · SIAM Review · doi:10.1137/19m1274067 · Cited by 2,512
Gradient-enhanced physics-informed neural networks for forward and inverse PDE problems
Yu, J., Lu, L., Meng, X., & Karniadakis, G. E. (2022) · Computer Methods in Applied Mechanics and Engineering · doi:10.1016/j.cma.2022.114823 · Cited by 722
DiSECt: A Differentiable Simulation Engine for Autonomous Robotic Cutting
Heiden, E., Macklin, M., Narang, Y., Fox, D., Garg, A., & Ramos, F. (2021) · Robotics: Science and Systems XVII · doi:10.15607/rss.2021.xvii.067 · Cited by 66
A computational toolbox for the assembly yield of complex and heterogeneous structures
Curatolo, A. I., Kimchi, O., Goodrich, C. P., Krueger, R. K., & Brenner, M. P. (2023) · Nature Communications · doi:10.1038/s41467-023-43168-4 · Cited by 15
Metadata
- Authors
- Margossian, C. C.
- Venue
- WIREs Data Mining and Knowledge Discovery
- Publisher
- Wiley
- Published
- 2019-03-20
- Volume
- 9
- Issue
- 4
- ISSN
- 1942-4787 (ISSN-L)
- 1942-4795
- OpenAlex
- W2901943374
- Cited by
- 321
- References
- 27
- FWCI
- 14.73
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A review of automatic differentiation and its efficient implementation. https://app.cochat.ai/papers/10.1002/widm.1305
- The paper
- Margossian, C. C. (2019). A review of automatic differentiation and its efficient implementation. WIREs Data Mining and Knowledge Discovery. https://doi.org/10.1002/widm.1305
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.