Explainable Machine Learning for Scientific Insights and Discoveries
Open accessThe DOI matches its Crossref record; 135 of the 137 references it cites were found in a registry or an index.
Abstract
Machine learning methods have been remarkably successful for a wide range of application areas in the extraction of essential information from data. An exciting and relatively recent development is the uptake of machine learning in the natural sciences, where the major goal is to obtain novel scientific insights and discoveries from observational or simulated data. A prerequisite for obtaining a scientific outcome is domain knowledge, which is needed to gain explainability, but also to enhance scientific consistency. In this article, we review explainable machine learning in view of applications in the natural sciences and discuss three core elements that we identified as relevant in this context: transparency, interpretability, and explainability. With respect to these core elements, we provide a survey of recent scientific works that incorporate machine learning and the way that explainable machine learning is used in combination with domain knowledge from the application areas.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Several powerful AI algorithms are employing a so-called “black box” approach, where it is difficult or even impossible to understand how the obtained results have been achieved.66”
From Addressing bias in big data and AI for health care: A call for open science (2021) · cites this paper as [66]
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 1×
“image or a video. Lerer et al. [2016] and Li et al. [2016] use video simulations to learn intuitive physics, for example about the stability of wooden block towers. Lerer et al. [2016] use ResNet-34 [He et al., 2016] and Googlenet [Szegedy et al., 2015] to predict the fall of towers of wooden blocks, as well as DeepMa…”
MethodGradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 1×
“o clips containing 101 objects of dierent characteristics, which was built for the task of deriving physical parameters such as velocity and mass. In their work, they use the LeNet CNN architecture [LeCun et al., 1998] to capture visual as well as physical characteristics while explicitly integrating physical laws bas…”
MethodGoing deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Gaussian Processes for Machine Learning
Rasmussen, C. E., & Williams, C. K. I. (2005) · doi:10.7551/mitpress/3206.001.0001 · Cited by 11,705
(2016) · Cited by 11,679
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294 · cited 1×
“Further details, types of interpretation, and specific realization can be found in recent surveys [7], [8], [20].”
BackgroundDeep learning and process understanding for data-driven Earth system science
Reichstein, M., Camps-Valls, G., Stevens, B., Jung, M., Denzler, J., Carvalhais, N., & Prabhat (2019) · Nature · doi:10.1038/s41586-019-0912-1 · Cited by 5,673
Lipton, Z. C. (2018) · Queue · doi:10.1145/3236386.3241340 · Cited by 5,427
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
Machine learning for molecular and materials science
Butler, K. T., Davies, D. W., Cartwright, H., Isayev, O., & Walsh, A. (2018) · Nature · doi:10.1038/s41586-018-0337-2 · Cited by 4,968 · cited 1×
“Reference [108] gives on overview on recent research using ML for molecular and materials science.”
Discovering governing equations from data by sparse identification of nonlinear dynamical systems
Brunton, S. L., Proctor, J. L., & Kutz, J. N. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1517384113 · Cited by 4,796
Towards A Rigorous Science of Interpretable Machine Learning
Doshi-Velez, F., & Kim, B. (2017) · arXiv · doi:10.48550/arxiv.1702.08608 · Cited by 3,121 · cited 1×
“Generally, to interpret means ‘‘to explain the meaning of’’ or ‘‘present in understandable terms;’’(1) see also [6]–[8].”
Methods for interpreting and understanding deep neural networks
Montavon, G., Samek, W., & Müller, K.-R. (2018) · Digital Signal Processing · doi:10.1016/j.dsp.2017.10.011 · Cited by 2,665
Explaining Explanations: An Overview of Interpretability of Machine Learning
Gilpin, L. H., Bau, D., Yuan, B. Z., Bajwa, A., Specter, M., & Kagal, L. (2018) · 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA) · doi:10.1109/dsaa.2018.00018 · Cited by 2,507 · cited 2×
“Ginsburg et al. [2016], for example, proposes FINE (feature importance in nonlinear embeddings) for the analysis of cancer patterns in ER+ breast cancer tissue slides. This approach relates original and automatically derived features to each other by estimating the relative contributions of the original features to th…”
BackgroundMachine learning and the physical sciences
Carleo, G., Cirac, I., Cranmer, K., Daudet, L., Schuld, M., Tishby, N., Vogt-Maranto, L., & Zdeborová, L. (2019) · Reviews of Modern Physics · doi:10.1103/revmodphys.91.045002 · Cited by 2,417 · cited 1×
“A review on the manifold recent research topics in the physical sciences is given by [109], with applications in particle physics and cosmology, quantum many-body physics, quantum computing, and chemical and material physics.”
BackgroundOpportunities and obstacles for deep learning in biology and medicine
Ching, T., Himmelstein, D. S., Beaulieu-Jones, B. K., Kalinin, A. A., Do, B. T., Way, G. P., Ferrero, E., Agapow, P.-M., Zietz, M., Hoffman, M. M., et al. (2018) · Journal of The Royal Society Interface · doi:10.1098/rsif.2017.0387 · Cited by 2,310 · cited 1×
“Several ML approaches have been used in biology and medicine to derive new insights, as described in [111] for the broad class of deep learning methods.”
MethodDefinitions, methods, and applications in interpretable machine learning
Murdoch, W. J., Singh, C., Kumbier, K., Abbasi-Asl, R., & Yu, B. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1900654116 · Cited by 2,167
European Union Regulations on Algorithmic Decision Making and a “Right to Explanation”
Goodman, B., & Flaxman, S. (2017) · AI Magazine · doi:10.1609/aimag.v38i3.2741 · Cited by 2,131 · cited 1×
“One component of these rules can be summed up by the phrase “right to an explanation” [Goodman and Flaxman, 2017].”
BackgroundAttention gated networks: learning to leverage salient regions in medical images
Schlemper, J., Oktay, O., Schaap, M., Heinrich, M., Kainz, B., Glocker, B., & Rueckert, D. (2019) · Medical Image Analysis · doi:10.1016/j.media.2019.01.012 · Cited by 2,014 · cited 2×
“Furthermore, [73] builds an attentive gated network that gradually fitted its attention weights with respect to targeted organ boundaries in segmenting tasks.”
BackgroundMethodCausability and explainability of artificial intelligence in medicine
Holzinger, A., Langs, G., Denk, H., Zatloukal, K., & Müller, H. (2019) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1312 · Cited by 1,885 · cited 1×
“Reference [113] discusses the difference between explainability and causality for medical applications, and the necessity of a person to be involved.”
BackgroundSensitivity Analysis in Practice: A Guide to Assessing Scientific Models
McCulloch, A. (2005) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2005.358_16.x · Cited by 1,842 · cited 1×
“A sensitivity analysis can be used to inspect how a model output (locally) depends upon the different input parameters [19].”
MethodSensitivity Analysis in Practice: A Guide to Assessing Scientific Models
Saltelli, A., Tarantola, S., Campolongo, F., & Ratto, M. (2004) · Cited by 1,797
Theory-Guided Data Science: A New Paradigm for Scientific Discovery from Data
Karpatne, A., Atluri, G., Faghmous, J. H., Steinbach, M., Banerjee, A., Ganguly, A., Shekhar, S., Samatova, N., & Kumar, V. (2017) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2017.2720168 · Cited by 1,752 · cited 2×
“Reference [35] underlines the importance of consistency by defining it as an essential component to measure performance:”
BackgroundKernel methods in machine learning
Hofmann, T., Schölkopf, B., & Smola, A. J. (2008) · The Annals of Statistics · doi:10.1214/009053607000000677 · Cited by 1,651 · cited 2×
“Namely, it is the unique solution of a convex optimization problem that can be reproducibly obtained [12], [13], resulting in algorithmic transparency.”
BackgroundMethodReynolds averaged turbulence modelling using deep neural networks with embedded invariance
Ling, J., Kurzawski, A., & Templeton, J. (2016) · Journal of Fluid Mechanics · doi:10.1017/jfm.2016.615 · Cited by 1,624 · cited 1×
“In [62], a deep learning approach for Reynolds-averaged”
BackgroundData-driven discovery of partial differential equations
Rudy, S. H., Brunton, S. L., Proctor, J. L., & Kutz, J. N. (2017) · Science Advances · doi:10.1126/sciadv.1602614 · Cited by 1,557
Cichocki, A., Zdunek, R., Phan, A. H., & Amari, S. (2009) · doi:10.1002/9780470747278 · Cited by 1,540
Nonlinear Dimensionality Reduction
(2007) · Information Science and Statistics · doi:10.1007/978-0-387-39351-3 · Cited by 1,517
Quantum-chemical insights from deep tensor neural networks
Schütt, K. T., Arbabzadah, F., Chmiela, S., Müller, K. R., & Tkatchenko, A. (2017) · Nature Communications · doi:10.1038/ncomms13890 · Cited by 1,445
Deep learning for universal linear embeddings of nonlinear dynamics
Lusch, B., Kutz, J. N., & Brunton, S. L. (2018) · Nature Communications · doi:10.1038/s41467-018-07210-0 · Cited by 1,424
KERNEL METHODS IN MACHINE LEARNING
Hofmann, T., Schölkopf, B., & Smola, A. J. (2008) · Cited by 1,408
Hidden physics models: Machine learning of nonlinear partial differential equations
Raissi, M., & Karniadakis, G. E. (2018) · Journal of Computational Physics · doi:10.1016/j.jcp.2017.11.039 · Cited by 1,349 · cited 1×
“Reference [43] proposes a nonlinear regression approach employing NNs to learn closed form representations of partial differential equations (PDEs) from scattered data collected in space and time, thereby uncovering the dynamic dependencies and obtaining a model that can be subsequently used to forecast”
Survey and critique of techniques for extracting rules from trained artificial neural networks
Andrews, R., Diederich, J., & Tickle, A. B. (1995) · Knowledge-Based Systems · doi:10.1016/0950-7051(96)81920-4 · Cited by 1,225
Data-Driven Science and Engineering: Machine Learning, Dynamical Systems, and Control
Brunton, S. L., & Kutz, J. N. (2019) · Cambridge University Press eBooks · doi:10.1017/9781108380690 · Cited by 1,154 · cited 1×
“As stated by [5], Data science is not replacing mathematical physics and engineering, but is instead augmenting it for the twenty-first century, resulting in more of a renaissance than a revolution.”
BackgroundSamek, W., Wiegand, T., & Müller, K.-R. (2017) · arXiv · doi:10.48550/arxiv.1708.08296 · Cited by 1,080 · cited 1×
“this context, research directions such as explainable artificial intelligence (AI) [1], informed ML [2], or intelligible intelligence [3] have emerged.”
BackgroundUnmasking Clever Hans predictors and assessing what machines really learn
Lapuschkin, S., Wäldchen, S., Binder, A., Montavon, G., Samek, W., & Müller, K.-R. (2019) · Nature Communications · doi:10.1038/s41467-019-08987-4 · Cited by 1,047 · cited 1×
“While the methods for interpretation allow the inspection of a single datum, [27] observes that it quickly becomes very time consuming to investigate large numbers of individual interpretations.”
MethodRETAIN: an interpretable predictive model for healthcare using reverse time attention mechanism
Choi, E., Bahadori, M. T., Kulas, J. A., Schuetz, A., Stewart, W. F., & Sun, J. (2016) · arXiv · doi:10.48550/arxiv.1608.05745 · Cited by 1,029
Causality: models, reasoning, and inference
(2010) · Choice Reviews Online · doi:10.5860/choice.47-3771 · Cited by 1,014
Ouyang, R., Curtarolo, S., Ahmetcik, E., Scheffler, M., & Ghiringhelli, L. M. (2018) · Physical Review Materials · doi:10.1103/physrevmaterials.2.083802 · Cited by 944
Raissi, M., Perdikaris, P. G., & Karniadakis, G. E. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.10561 · Cited by 925
Rethinking drug design in the artificial intelligence era
Schneider, P., Walters, W. P., Plowright, A. T., Sieroka, N., Listgarten, J., Goodnow, R. A., Fisher, J., Jansen, J. M., Duca, J. S., Rush, T. S., et al. (2020) · Nature Reviews Drug Discovery · doi:10.1038/s41573-019-0050-3 · Cited by 894 · cited 1×
“For the successful application of ML for drug design, [114] identifies five ‘‘grand challenges’’: obtaining”
MethodMittelstadt, B., Russell, C., & Wachter, S. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287574 · Cited by 794 · cited 1×
“Concerning explanations, it has also been argued that there is a gap of expectations between ML and so-called explanation sciences such as law, cognitive science, philosophy, and the social sciences [28].”
Backgroundvon Rueden, L., Mayer, S., Beckh, K., Georgiev, B., Giesselbach, S., Heese, R., Kirsch, B., Walczak, M., Pfrommer, J., Pick, A., et al. (2021) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3079836 · Cited by 793
Self-citationScientific Representation: Paradoxes of Perspective
van Fraassen, B. C. (2008) · doi:10.1093/acprof:oso/9780199278220.001.0001 · Cited by 754
Machine Learning for the Geosciences: Challenges and Opportunities
Karpatne, A., Ebert-Uphoff, I., Ravela, S., Babaie, H. A., & Kumar, V. (2019) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2018.2861006 · Cited by 661
Learning to Segment Object Candidates
Pinheiro, P. O., Collobert, R., & Dollár, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.06204 · Cited by 648 · cited 1×
“, 2015] to predict the fall of towers of wooden blocks, as well as DeepMask [Pinheiro et al., 2015] and a custom network called PhysNet to predict the trajectory of the wooden blocks in case the tower is collapsing.”
BackgroundRevealing the vectors of cellular identity with single-cell genomics
Wagner, A., Regev, A., & Yosef, N. (2016) · Nature Biotechnology · doi:10.1038/nbt.3711 · Cited by 632
A Survey of Methods for Explaining Black Box Models
Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., & Pedreschi, D. (2019) · ACM Computing Surveys · doi:10.1145/3236009 · Cited by 629 · cited 5×
“As written in [8], ‘‘[in explainable ML] these definitions assume implicitly that the concepts expressed in the understandable terms composing an explanation are self-contained and do not need further explanations.”
BackgroundMethodInfluentialDiscovering phase transitions with unsupervised learning
Wang, L. (2016) · Physical Review B · doi:10.1103/physrevb.94.195105 · Cited by 618 · cited 1×
“similar way, [83] and [84] use principal component analysis and the derived interpretable principal components for exploration of different phases, phase-transition, and crossovers in classical spin models.”
MethodThe Building Blocks of Interpretability
Olah, C., Satyanarayan, A., Johnson, I., Carter, S., Schubert, L., Ye, K., & Mordvintsev, A. (2018) · Distill · doi:10.23915/distill.00010 · Cited by 591
Machine learning of linear differential equations using Gaussian processes
Raissi, M., Perdikaris, P., & Karniadakis, G. E. (2017) · Journal of Computational Physics · doi:10.1016/j.jcp.2017.07.050 · Cited by 584
McCloskey, M. (1983) · Scientific American · doi:10.1038/scientificamerican0483-122 · Cited by 583 · cited 1×
“Intuitive physics are everyday-observed rules of nature that help us to predict the outcome of events even with a relatively untrained human perception [38].”
Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontiers
Hohman, F., Kahng, M., Pienta, R., & Chau, D. H. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2843369 · Cited by 582
Unmasking Clever Hans predictors and assessing what machines really learn
Lapuschkin, S., Wäldchen, S., Binder, A., Montavon, G., Samek, W., & Müller, K. (2019) · RePEc: Research Papers in Economics · Cited by 580
Explanation, imagination, and confidence in judgment.
Koehler, D. J. (1991) · Psychological Bulletin · doi:10.1037//0033-2909.110.3.499 · Cited by 561 · cited 1×
“Further, a review of studies of experimental manipulations that require people to generate explanations or imagine scenarios indicates that people express greater confidence in a possibility, although false, when asked to generate explanations for it or imagine the possibility [34].”
BackgroundAn explainable deep machine vision framework for plant stress phenotyping
Ghosal, S., Blystone, D., Singh, A. K., Ganapathysubramanian, B., Singh, A., & Sarkar, S. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1716999115 · Cited by 553 · cited 1×
“Reference [66] uses interpretation tools for image-based plant stress phenotyping.”
MethodDiscovering Physical Concepts with Neural Networks
Iten, R., Metger, T., Wilming, H., del Rio, L., & Renner, R. (2020) · Physical Review Letters · doi:10.1103/physrevlett.124.010508 · Cited by 458 · cited 1×
“Reference [100] introduces SciNet, a modified variational autoencoder that learns a representation from experimental data and uses the learned representation to derive physical concepts from it rather than from the experimental input”
BackgroundInferring biological networks by sparse identification of nonlinear dynamics
Mangan, N. M., Brunton, S. L., Proctor, J. L., & Kutz, J. N. (2016) · IEEE Transactions on Molecular, Biological and Multi-Scale Communications · doi:10.1109/tmbmc.2016.2633265 · Cited by 436
Using deep learning to model the hierarchical structure and function of a cell
Ma, J., Yu, M. K., Fong, S., Ono, K., Sage, E., Demchak, B., Sharan, R., & Ideker, T. (2018) · Nature Methods · doi:10.1038/nmeth.4627 · Cited by 435
Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations
Raissi, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.06637 · Cited by 419
A Compositional Object-Based Approach to Learning Physical DynamicsPreprint
Chang, M. B., Ullman, T. D., Torralba, A. B., & Tenenbaum, J. B. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.00341 · Cited by 388
Machine learning strategies for systems with invariance properties
Ling, J., Jones, R., & Templeton, J. (2016) · Journal of Computational Physics · doi:10.1016/j.jcp.2016.05.003 · Cited by 386 · cited 1×
“A related approach for RANS-modeled Reynolds stresses for high-speedflat-plate turbulent boundary layers was presented in [63], which uses a systematic approach with basis tensor invariants proposed by [64].”
MethodHu, W., Singh, R. R. P., & Scalettar, R. T. (2017) · Physical Review E · doi:10.1103/physreve.95.062122 · Cited by 332 · cited 1×
“similar way, [83] and [84] use principal component analysis and the derived interpretable principal components for exploration of different phases, phase-transition, and crossovers in classical spin models.”
MethodRaissi, M., Perdikaris, P. G., & Karniadakis, G. E. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.10566 · Cited by 312 · cited 1×
“See also the related works of [35], who use the term theoryguided data science, or physics-informed learning by [36].”
BackgroundSelf-Regulation of Cognitive Inference and Decision Processes
Baumeister, R. F., & Newman, L. S. (1994) · Personality and Social Psychology Bulletin · doi:10.1177/0146167294201001 · Cited by 292 · cited 1×
“For illustration, [33] distinguishes between the intuitive scientist, who seeks to make the most accurate or otherwise optimal decision, and the intuitive lawyer,”
BackgroundOuyang, R., Stefano Curtarolo, A., Ahmetcik, E., Scheffler, M., & Ghiringhelli, L. M. (2018) · MPG.PuRe (Max Planck Society) · Cited by 289
Learning Physical Intuition of Block Towers by ExamplePreprint
Lerer, A. K., Gross, S., & Fergus, R. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.01312 · Cited by 287
Accelerating Eulerian Fluid Simulation With Convolutional Networks
Tompson, J., Schlachter, K., Sprechmann, P., & Perlin, K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.03597 · Cited by 280 · cited 2×
“The idea in [49] is to use a transparent cost function design by reformulating the condition of divergence-free velocity fields into an unsupervised learning problem at each time step.”
MethodData-driven fluid simulations using regression forests
Ladický, L., Jeong, S., Solenthaler, B., Pollefeys, M., & Gross, M. (2015) · ACM Transactions on Graphics · doi:10.1145/2816795.2818129 · Cited by 275 · cited 1×
“[2017] and Jeong et al. [2015] use physics-informed approaches for applications such as fluid simulations based on the incompressible Navier-Stokes equations, where physicsbased losses are introduced to achieve plausible results.”
BackgroundLabel-Free Supervision of Neural Networks with Physics and Domain Knowledge
Stewart, R., & Ermon, S. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10934 · Cited by 274 · cited 1×
“For example, [53] detects and tracks objects in videos in an unsupervised way.”
BackgroundInterpretations are useful: penalizing explanations to align neural networks with prior knowledgePreprint
Rieger, L., Singh, C., Murdoch, W. J., & Yu, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.13584 · Cited by 264 · cited 1×
“Reference [79] proposes contextual decomposition explanation penalization, which constrains a classification or regression result to more accurate and more scientifically plausible results by leveraging the explained outcome of interpretation tools.”
BackgroundAccelerating the Design of Functional Glasses through Modeling
Mauro, J. C., Tandia, A., Vargheese, K. D., Mauro, Y. Z., & Smedskjaer, M. M. (2016) · Chemistry of Materials · doi:10.1021/acs.chemmater.6b01054 · Cited by 256
The challenge of crafting intelligible intelligence
Weld, D. S., & Bansal, G. (2019) · Communications of the ACM · doi:10.1145/3282486 · Cited by 253
Applications of tensor (multiway array) factorizations and decompositions in data mining
Mørup, M. (2011) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1 · Cited by 244
Design Principles and Top Non-Fullerene Acceptor Candidates for Organic Photovoltaics
Lopez, S. A., Sanchez-Lengeling, B., de Goes Soares, J., & Aspuru-Guzik, A. (2017) · Joule · doi:10.1016/j.joule.2017.10.006 · Cited by 225
Conditioning by adaptive sampling for robust designPreprint
Brookes, D. H., Park, H., & Listgarten, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.10060 · Cited by 217 · cited 1×
“In this way, it is possible to generate new synthetic data samples that optimize the value of the quantity or property by leveraging an adaptive sampling technique over the generative model; see also [48].”
BackgroundReliable and explainable machine-learning methods for accelerated material discovery
Kailkhura, B., Gallagher, B., Kim, S., Hiszpanski, A., & Han, T. Y.-J. (2019) · npj Computational Materials · doi:10.1038/s41524-019-0248-2 · Cited by 211 · cited 1×
“For example, [78] discusses explainable ML for scien-”
BackgroundCausality for Machine Learning
Schölkopf, B. (2022) · Probabilistic and Causal Inference · doi:10.1145/3501714.3501755 · Cited by 209
Syntax-Directed Variational Autoencoder for Structured Data
Dai, H., Tian, Y., Dai, B., Skiena, S., & Song, L. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.08786 · Cited by 202 · cited 2×
“A so-called syntax-direct variational autoencoder is introduced in [90], where syntax and semantic con-”
BackgroundDetecting Statistical Interactions from Neural Network WeightsPreprint
Tsang, M., Cheng, D., & Liu, Y. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.04977 · Cited by 188
Tomita, N., Abdollahi, B., Wei, J., Ren, B., Suriawinata, A., & Hassanpour, S. (2019) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.14645 · Cited by 187
Understanding tools: Task-oriented object modeling, learning and recognition
Zhu, Y., Zhao, Y., & Zhu, S. C. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298903 · Cited by 181 · cited 1×
“Reference [45] introduces a framework that calculates physical concepts from color-depth videos that estimates tool and tool-use such as cracking a nut.”
BackgroundIndividual Brain Charting, a high-resolution fMRI dataset for cognitive mapping
Pinho, A. L., Amadon, A., Ruest, T., Fabre, M., Dohmatob, E., Denghien, I., Ginisty, C., Becuwe-Desmidt, S., Roger, S., Laurier, L., et al. (2018) · Scientific Data · doi:10.1038/sdata.2018.105 · Cited by 176
Sparse dynamics for partial differential equations
Schaeffer, H., Caflisch, R., Hauck, C. D., & Osher, S. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1302752110 · Cited by 172
Exact Recovery of Chaotic Systems from Highly Corrupted Data
Tran, G., & Ward, R. (2017) · Multiscale Modeling & Simulation · doi:10.1137/16m1086637 · Cited by 168
Learning physical descriptors for materials science by compressed sensing
Ghiringhelli, L. M., Vybíral, J., Ahmetcik, E., Ouyang, R., Levchenko, S. V., Draxl, C., & Scheffler, M. (2017) · New Journal of Physics · doi:10.1088/1367-2630/aa57bf · Cited by 149 · cited 1×
“Feature selection schemes using embedded methods, similar to the previous group, have been used in areas such as material sciences [93], [94].”
MethodNewtonian Image Understanding: Unfolding the Dynamics of Objects in Static Images
Mottaghi, R., Bagherinezhad, H., Rastegari, M., & Farhadi, A. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.383 · Cited by 131
Yan, Y., Zhu, J., Duda, M., Solarz, E., Sripada, C., & Koutra, D. (2019) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3292500.3330921 · Cited by 127 · cited 1×
“Reference [102], for example, introduces a grouping layer in a graph-based NN called GroupINN to identify subgroups of”
MethodPhysics 101: Learning Physical Object Properties from Unlabeled Videos
Wu, J., Lim, J. J., Zhang, H., Tenenbaum, J. B., & Freeman, W. T. (2016) · British Machine Vision Conference (BMVC) · doi:10.5244/c.30.39 · Cited by 114
Physics-aware Gaussian processes in remote sensing
Camps‐Valls, G., Martino, L., Svendsen, D. H., Campos‐Taberner, M., Muñoz-Marı́, J., Laparra, V., Luengo, D., & Garcı́a-Haro, F. J. (2018) · Applied Soft Computing · doi:10.1016/j.asoc.2018.03.021 · Cited by 107
Scientific Domain Knowledge Improves Exoplanet Transit Classification with Deep Learning
Ansdell, M., Ioannou, Y., Osborn, H. P., Sasdelli, M., (2018 NASA Frontier Development Lab Exoplanet Team), Smith, J. C., Caldwell, D. A., Jenkins, J. M., Raïssi, C., Angerhausen, D., et al. (2018) · The Astrophysical Journal Letters · doi:10.3847/2041-8213/aaf23b · Cited by 101 · cited 1×
“Reference [44] designed a multi-source NN for exoplanet transit classification.”
MethodPyzer‐Knapp, E. O., Simm, G. N. C., & Aspuru‐Guzik, A. (2016) · Materials Horizons · doi:10.1039/c5mh00282f · Cited by 95
Attend and Predict: Understanding Gene Regulation by Selective Attention on ChromatinPreprint
Singh, R., Lanchantin, J., Sekhon, A., & Qi, Y. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/329334 · Cited by 94 · cited 1×
“Reference [75] uses attention modules for genomics in their AttentiveChrome NN.”
MethodDsilva, C. J., Talmon, R., Coifman, R. R., & Kevrekidis, I. G. (2015) · Applied and Computational Harmonic Analysis · doi:10.1016/j.acha.2015.06.008 · Cited by 91 · cited 1×
“rent end-to-end long-term prediction network. Therefore, a simulation software and the explicit modelling of the underlying physical laws is not necessary. Another broad framework [Yair et al., 2017, Dsilva et al., 2018, Holidaya et al., 2019] leverages unsupervised learning approaches to learn low-complexity represen…”
BackgroundShapeStacks: Learning Vision-Based Physical Intuition for Generalised Object Stacking
Groth, O., Fuchs, F. B., Posner, I., & Vedaldi, A. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01246-5_43 · Cited by 82
Atlases of cognition with large-scale human brain mapping
Varoquaux, G. P., Schwartz, Y., Poldrack, R. A., Gauthier, B., Bzdok, D., Poline, J., & Thirion, B. (2018) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006565 · Cited by 78
Probing hidden spin order with interpretable machine learning
Greitemann, J., Liu, K., & Pollet, L. (2019) · Physical review. B./Physical review. B · doi:10.1103/physrevb.99.060404 · Cited by 76 · cited 1×
“So-called order parameters in condensed matter physics are analysed in [106], [107].”
BackgroundFrom deep learning to mechanistic understanding in neuroscience: the structure of retinal prediction
Tanaka, H., Nayebi, A., Maheswaranathan, N., McIntosh, L., Baccus, S. A., & Ganguli, S. (2019) · PubMed · doi:10.48550/arxiv.1912.06207 · Cited by 76
Learning physical descriptors for materials science by compressed sensing
Ghiringhelli, L. M., Vybiral, J., Ahmetcik, E., Ouyang, R., Levchenko, S., Draxl, C., & Scheffler, M. (2017) · MPG.PuRe (Max Planck Society) · Cited by 75
Design by adaptive samplingPreprint
Brookes, D. H., & Listgarten, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.03714 · Cited by 68 · cited 1×
“Reference [47] has proposed a data-centric approach for scientific design based on the combination of a generative model for the data being considered, e.”
Learning multiple order parameters with interpretable machines
Liu, K., Greitemann, J., & Pollet, L. (2019) · Physical review. B./Physical review. B · doi:10.1103/physrevb.99.104410 · Cited by 66 · cited 1×
“So-called order parameters in condensed matter physics are analysed in [106], [107].”
BackgroundScientific Domain Knowledge Improves Exoplanet Transit Classification with Deep Learning
Ansdell, M., Ioannou, Y., Osborn, H. P., Sasdelli, M., Smith, J., Caldwell, D., Jenkins, J., Räissi, C., Angerhausen, D., et al. (2018) · Cambridge University Engineering Department Publications Database · Cited by 65
To Fall Or Not To Fall: A Visual Approach to Physical Stability PredictionPreprint
Li, W., Azimi, S. M., Leonardis, A., & Fritz, M. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1604.00066 · Cited by 63 · cited 1×
“In a similar way, but with more complex scenes or differently shaped objects, [40] predicts the physical stability of stacked objects using various popular convolutional neural network (CNN) architectures.”
BackgroundWang, J., Huang, J., Duan, L., & Xiao, H. (2018) · Theoretical and Computational Fluid Dynamics · doi:10.1007/s00162-018-0480-2 · Cited by 63
Reconstruction of normal forms by learning informed observation geometries from data
Yair, O., Talmon, R., Coifman, R. R., & Kevrekidis, I. G. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1620045114 · Cited by 61
Interpretable Intuitive Physics Model
Ye, T., Wang, X., Davidson, J., & Gupta, A. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01258-8_6 · Cited by 58
Manifold learning for parameter reduction
Holiday, A., Kooshkbaghi, M., Bello‐Rivas, J. M., Gear, C. W., Zagaris, A., & Kevrekidis, I. G. (2019) · Journal of Computational Physics · doi:10.1016/j.jcp.2019.04.015 · Cited by 54 · cited 1×
“Lerer et al. [2016] and Li et al. [2016] use video simulations to learn intuitive physics, for example about the stability of wooden block towers.”
BackgroundLearning A Physical Long-term PredictorPreprint
Ehrhardt, S., Monszpart, Á., Mitra, N. J., & Vedaldi, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.00247 · Cited by 52 · cited 2×
“Further details, types of interpretation, and specific realization can be found in recent surveys [7], [8], [20].”
Background“Explain” in scientific discourse
Overton, J. A. (2012) · Synthese · doi:10.1007/s11229-012-0109-8 · Cited by 48 · cited 1×
“For scientific explanations, [30] identifies five broad categories to classify the large majority of objects that are explained in science: data, entities, kinds, models, and theories.”
BackgroundChen, K., Yao, L., Wang, X., Zhang, D., Gu, T., Yu, Z., & Yang, Z. (2018) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2018.8489767 · Cited by 43
Hyperspectral Plant Disease Forecasting Using Generative Adversarial Networks
Förster, A., Behley, J., Behmann, J., & Roscher, R. (2019) · IEEE International Geoscience and Remote Sensing Symposium (IGARSS) · doi:10.1109/igarss.2019.8898749 · Cited by 43 · cited 1×
“Reference [41] predicts the spread of diseases on barley plants in micro-”
BackgroundSelf-citationInterpretable machine learning for inferring the phase boundaries in a nonequilibrium system
Casert, C., Vieijra, T., Nys, J., & Ryckebusch, J. (2019) · Physical review. E · doi:10.1103/physreve.99.023304 · Cited by 42 · cited 1×
“Ideally, one could answer the question from [16]: ‘‘Can we understand what the ML algo-”
BackgroundFrom genotype to phenotype: augmenting deep learning with networks and systems biology
Gazestani, V., & Lewis, N. E. (2019) · Current Opinion in Systems Biology · doi:10.1016/j.coisb.2019.04.001 · Cited by 42 · cited 3×
“of deep learning approaches in systems biology is given in Gazestani and Lewis [2019]. They describe how one can design DNNs that encode the extensive, existing networkand systems-level knowledge that is generated by combing diverse data types. It is said that such designs inform the model on aspects of the hierarchic…”
BackgroundInfluentialA Rate-Distortion Framework for Explaining Neural Network DecisionsPreprint
Macdonald, J., Wäldchen, S., Hauch, S., & Gitta Kutyniok, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.11092 · Cited by 41 · cited 1×
“[23] introduces a formal notion for interpreting NNs where a set of input features is deemed relevant for a classi-”
Backgroundvon Rueden, L., Mayer, S., Garcke, J., Bauckhage, C., & Schücker, J. (2019) · arXiv (Cornell University) · Cited by 40
Self-citationFeature Importance in Nonlinear Embeddings (FINE): Applications in Digital Pathology
Ginsburg, S. B., Lee, G., Ali, S., & Madabhushi, A. (2015) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2015.2456188 · Cited by 33 · cited 1×
“Reference [65], for example, proposes FINE (feature importance in nonlinear embeddings) for the analysis of”
An information-based machine learning approach to elasticity imaging
Hoerig, C. L., Ghaboussi, J., & Insana, M. F. (2016) · Biomechanics and Modeling in Mechanobiology · doi:10.1007/s10237-016-0854-6 · Cited by 33 · cited 1×
“An information-based ML approach using NNs to solve an inverse problem in biomechanical applications was presented in [58].”
MethodThe DeepTune framework for modeling and characterizing neurons in visual cortex area V4Preprint
Abbasi-Asl, R., Chen, Y., Bloniarz, A. E., Oliver, M., Willmore, B. D. B., Gallant, J. L., & Yu, B. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/465534 · Cited by 33 · cited 2×
“Abbasi-Asl et al. [2018] introduce DeepTune, a visualization framework for CNNs, for applications in neuroscience. DeepTune consists of an ensemble of CNNs that learn multiple complementary representations of natural images. The features from these CNNs are fed into regression models to predict the firing rates of neu…”
BackgroundMethodGraphical dynamical systems and their applications to bio-social systems
Adiga, A., Kuhlman, C. J., MARATHE, M. V., Mortveit, H. S., Ravi, S. S., & Vullikanti, A. K. S. (2018) · International Journal of Advances in Engineering Sciences and Applied Mathematics · doi:10.1007/s12572-018-0237-6 · Cited by 30 · cited 1×
“For example, in [95], an application of ML for epidemiology leverages a”
BackgroundNeural-Guided Symbolic Regression with Asymptotic ConstraintsPreprint
Li, L., Fan, M., Singh, R., & Riley, P. F. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.07714 · Cited by 29
Molloy, E. K., & Warnow, T. J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00834-5_15 · Cited by 25
Automated, predictive, and interpretable inference of Caenorhabditis elegans escape dynamics
Daniels, B. C., Ryu, W. S., & Nemenman, I. M. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1816531116 · Cited by 24 · cited 1×
“Reference [99] use theML algorithm ‘Sir Isaac’ to infer a dynamical model of biological time-series data to understand and predict dynamics of worm escape behavior.”
MethodEigel, M., Schneider, R., Trunschke, P., & Wolf, S. (2019) · Advances in Computational Mathematics · doi:10.1007/s10444-019-09723-8 · Cited by 24 · cited 1×
“A tensor-based approach to ML for uncertainty quantification problems can be found in [57], where the solutions to parametric convection-diffusion PDEs are learned based on a few samples.”
BackgroundHagensieker, R., Roscher, R., Rosentreter, J., Jakimow, B., & Waske, B. (2017) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2017.07.019 · Cited by 22 · cited 1×
“Reference [51] classifies land use and land cover and their changes based on remote sensing satellite timeseries data.”
BackgroundSelf-citationSMASH: physics-guided reconstruction of collisions from videos
Monszpart, Á., Thuerey, N., & Mitra, N. J. (2016) · ACM Transactions on Graphics · doi:10.1145/2980179.2982421 · Cited by 19
A comparison of hierarchical multi-output recognition approaches for anuran classification
Colonna, J. G., da Gama, J. M. P., & Nakamura, E. F. (2018) · Machine Learning · doi:10.1007/s10994-018-5739-8 · Cited by 19
Genetic Architect: Discovering Genomic Structure with Learned Neural ArchitecturesPreprint
Deming, L., Targ, S., Sauder, N., Almeida, D., & Ye, C. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.07156 · Cited by 13 · cited 1×
“For example, [74] applies attention modules in NNs trained on genomic sequences for the identification of important sequence motifs by visualizing the attention mask weights.”
BackgroundInterpreting Deep Learning: The Machine Learning Rorschach Test?Preprint
Charles, A. S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.00148 · Cited by 13
A regression approach for explaining manifold embedding coordinatesPreprint
Meilă, M., Koelle, S., & Zhang, H. (2018) · arXiv (Cornell University) · doi:10.13140/rg.2.2.20959.71841 · Cited by 12
Liao, Q., & Poggio, T. (2017) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 9
Chen, K., Yao, L., Wang, X., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.07233 · Cited by 8 · cited 1×
“Attention modules in recurrent NNs for multi-modal sensorbased activity recognition have been used by [77].”
MethodNeural-Guided Symbolic Regression with Semantic Prior.Preprint
Li, L., Fan, M., Singh, R., & Riley, P. F. (2019) · arXiv (Cornell University) · Cited by 7
Deep Learning for Plasma Diagnostics
Matos, F., Hendrich, F., Jenko, F., & Odstrčil, T. (2017) · Max Planck Digital Library · Cited by 1
Deep Learning Model Integration of Remotely Sensed and SWAT-Simulated Regional Soil Moisture
Breen, K., James, S. C., & White, J. D. (2018) · AGUFM · Cited by 1
Research works printed but not linked: 3
- 10.1037/0033-2909.110.3.499 · doi:10.1037/0033-2909.110.3.499 · Paper page
- frigg, 2018, Scientific representation, The Stanford Encyclopedia of Philosophy
- 10.1145/3233231 · doi:10.1145/3233231 · Paper page
Cited by
Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications
Samek, W., Montavon, G., Lapuschkin, S., Anders, C. J., & Muller, K.-R. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3060483 · Cited by 1,477
“Only recently, the technologies have become available to extract scientific insights from complex nonlinear models [121], [150], [166].”
BackgroundAddressing bias in big data and AI for health care: A call for open science
Norori, N., Hu, Q., Aellen, F. M., Faraci, F. D., & Tzovara, A. (2021) · Patterns · doi:10.1016/j.patter.2021.100347 · Cited by 903
“66 Explainable AI by contrast includes interpretable AI models, where the strengths and weaknesses of a decision-making process are transparent.”
Backgroundvon Rueden, L., Mayer, S., Beckh, K., Georgiev, B., Giesselbach, S., Heese, R., Kirsch, B., Walczak, M., Pfrommer, J., Pick, A., et al. (2021) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3079836 · Cited by 793
“With machine learning models becoming more and more complex, there is also a growing need for models to be interpretable and explainable [9].”
BackgroundSelf-citationMachine learning for digital soil mapping: Applications, challenges and suggested solutions
Wadoux, A. M. J.-C., Minasny, B., & McBratney, A. B. (2020) · Earth-Science Reviews · doi:10.1016/j.earscirev.2020.103359 · Cited by 731
Jusup, M., Holme, P., Kanazawa, K., Takayasu, M., Romić, I., Wang, Z., Geček, S., Lipić, T., Podobnik, B., Wang, L., et al. (2022) · Physics Reports · doi:10.1016/j.physrep.2021.10.005 · Cited by 643
Metadata
- Authors
- Roscher, R., Bohn, B., Duarte, M. F., & Garcke, J.
- Affiliations
- University of Bonn (DE)
- University of Massachusetts Amherst (US)
- Venue
- IEEE Access
- Published
- 2020
- Volume
- 8
- Pages
- 42200-42216
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W2945526235
- Cited by
- 1,040
- References
- 137
- Citations by year
- 2026: 152 · 2025: 178 · 2024: 168 · 2023: 184 · 2022: 174 · 2021: 130 · 2020: 47
- FWCI
- 52.58
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- IEEE Access (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Publikationsdatenbank der Fraunhofer-Gesellschaft (Fraunhofer-Gesellschaft) (submittedVersion, other-oa)
- Fraunhofer-Publica (Fraunhofer-Gesellschaft) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Fraunhofer Cluster of Excellence Cognitive Internet Technologies
- Deutsche Forschungsgemeinschaft
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Decision Sciences › Information Systems and Management › Scientific Computing and Data Management
- Physical Sciences › Materials Science › Materials Chemistry › Machine Learning in Materials Science
- Keywords
- explainable machine learning, scientific discovery, transparency, interpretability, explainability, domain knowledge, observational data, simulated data
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.