The DOI matches its Crossref record; 334 of the 334 references it cites were found in a registry or an index.
Interpretable machine learning: Fundamental principles and 10 grand challenges
Open accessAbstract
Interpretability in machine learning (ML) is crucial for high stakes decisions and troubleshooting. In this work, we provide fundamental principles for interpretable ML, and dispel common misunderstandings that dilute the importance of this crucial topic. We also identify 10 technical challenge areas in interpretable machine learning and provide history and background on each problem. Some of these problems are classically important, and some are recent problems that have arisen in the last few years. These problems are: (1) Optimizing sparse logical models such as decision trees; (2) Optimization of scoring systems; (3) Placing constraints into generalized additive models to encourage sparsity and better interpretability; (4) Modern case-based reasoning, including neural networks and matching for causal inference; (5) Complete supervised disentanglement of neural networks; (6) Complete or even partial unsupervised disentanglement of neural networks; (7) Dimensionality reduction for data visualization; (8) Machine learning models that can incorporate physics and other generative or causal constraints; (9) Characterization of the “Rashomon set” of good models; and (10) Interpretable reinforcement learning. This survey is suitable as a starting point for statisticians and computer scientists interested in working in interpretable machine learning.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“For tabular data, very small interpretable models can often perform as well as black box models on benchmark datasets if they are optimized carefully.”
From Interpretable algorithmic forensics (2023) · cites this paper as [22]
The cited source says what the claim attributes to it.
In the citing paper
“Thus, for either tabular data or images, interpretable models are generally as accurate as the best of the black box models when applied to benchmark datasets (22).”
From Interpretable algorithmic forensics (2023) · cites this paper as [22]
The cited source says what the claim attributes to it.
In the citing paper
“For image data, where neural networks are the only technique that performs well currently, interpretable neural networks perform as well as black box neural networks.”
From Interpretable algorithmic forensics (2023) · cites this paper as [22]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016
(2008) · Journal of Machine Learning Research · Cited by 33,051
Greedy function approximation: A gradient boosting machine
Friedman, J. H. (2001) · The Annals of Statistics · doi:10.1214/aos/1013203451 · Cited by 30,731
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
UCI Machine Learning Repository
Asuncion, A. (2007) · Medical Entomology and Zoology · Cited by 24,155
Classification and Regression Trees.
Ryzin, J. V., Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1986) · Journal of the American Statistical Association · doi:10.2307/2288003 · Cited by 20,825
A decision-theoretic generalization of on-line learning and an application to boosting
Freund, Y., & Schapire, R. E. (1997) · Journal of Computer and System Sciences · doi:10.1006/jcss.1997.1504 · Cited by 20,466
Research works printed but not linked: 15
- Bousquet, O. and Elisseeff, A. (2002). Stability and generalization. Journal of Machine Learning Research 2 499–526.
- Dobkin, D., Fulton, T., Gunopulos, D., Kasif, S. and Salzberg, S. (1997). Induction of shallow decision trees. IEEE Transactions on Pattern Analysis and Machine Intelligence.
- Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A. and Bengio, Y. (2014). Generative Adversarial Nets. In Proceedings of Conference on Neural Information Processing Systems (NeurIPS) 27 2672–2680.
- Hammond, K. (1989). Proceedings of the Second Case-Based Reasoning Workshop (DARPA). Morgan Kanfmann Publishers, Inc., San Mateo, CA.
- Hu, X., Rudin, C. and Seltzer, M. (2019). Optimal Sparse Decision Trees. In Proceedings of Conference on Neural Information Processing Systems (NeurIPS).
- Juozapaitis, Z., Koul, A., Fern, A., Erwig, M. and Doshi-Velez, F. (2019). Explainable reinforcement learning via reward decomposition. In Proceedings of the IJCAI/ECAI Workshop on Explainable Artificial Intelligence.
- Kodratoff, Y. (1994). The comprehensibility manifesto. KDD Nugget Newsletter 94.
- Kolodner, J. L. (1988). Proceedings of the Case-Based Reasoning Workshop (DARPA). Morgan Kaufmmnn Publishers, Inc., San Mateo, CA.
- Kosiorek, A., Sabour, S., Teh, Y. W. and Hinton, G. E. (2019). Stacked capsule autoencoders. In Proceedings of Conference on Neural Information Processing Systems (NeurIPS) 15512–15522.
- Quinlan, J. R. (1993). C4.5: Programs for Machine Learning. Morgan Kaufmann.
- Sabour, S., Frosst, N. and Hinton, G. E. (2017). Dynamic routing between capsules. In Proceedings of Conference on Neural Information Processing Systems (NeurIPS) 3856–3866.
- Srebro, N., Sridharan, K. and Tewari, A. (2010). Smoothness, low noise and fast rates. In Proceedings of Conference on Neural Information Processing Systems (NeurIPS) 2199–2207.
- Yang, H., Rudin, C. and Seltzer, M. (2017). Scalable Bayesian rule lists. In Proceedings of International Conference on Machine Learning (ICML) 3921–3930. · DOI
- Zhu, Z., Luo, P., Wang, X. and Tang, X. (2014). Multi-view perceptron: a deep model for learning face identity and view representations. In Proceedings of Conference on Neural Information Processing Systems (NeurIPS) 217–225.
- Hastie, T. J. and Tibshirani, R. J. (1990). Generalized additive models 43. CRC press.
Cited by
Scientific Machine Learning Through Physics–Informed Neural Networks: Where we are and What’s Next
Cuomo, S., Di Cola, V. S., Giampaolo, F., Rozza, G., Raissi, M., & Piccialli, F. (2022) · Journal of Scientific Computing · doi:10.1007/s10915-022-01939-z · Cited by 2,784
Interpreting Black-Box Models: A Review on Explainable Artificial Intelligence
Hassija, V., Chamola, V., Mahapatra, A., Singal, A., Goel, D., Huang, K., Scardapane, S., Spinelli, I., Mahmud, M., & Hussain, A. (2024) · Cognitive Computation · doi:10.1007/s12559-023-10179-8 · Cited by 2,495
Ali, S., Abuhmed, T., El-Sappagh, S., Muhammad, K., Alonso-Moral, J. M., Confalonieri, R., Guidotti, R., Del Ser, J., Díaz-Rodríguez, N., & Herrera, F. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101805 · Cited by 1,762
Aligning artificial intelligence with climate change mitigation
Kaack, L. H., Donti, P. L., Strubell, E., Kamiya, G., Creutzig, F., & Rolnick, D. (2022) · Nature Climate Change · doi:10.1038/s41558-022-01377-7 · Cited by 570
Bottom-up Coarse-Graining: Principles and Perspectives
Jin, J., Pak, A. J., Durumeric, A. E. P., Loose, T. D., & Voth, G. A. (2022) · Journal of Chemical Theory and Computation · doi:10.1021/acs.jctc.2c00643 · Cited by 305
Metadata
- Authors
- Rudin, C., Chen, C., Chen, Z., Huang, H., Semenova, L., & Zhong, C.
- Venue
- Statistics Surveys
- Publisher
- Institute of Mathematical Statistics
- Published
- 2022-01-01
- Volume
- 16
- Issue
- none
- Pages
- 1-85
- ISSN
- 1935-7516 (ISSN-L)
- OpenAlex
- W3137125108
- Cited by
- 958
- References
- 334
- Citations by year
- 2026: 202 · 2025: 247 · 2024: 205 · 2023: 179 · 2022: 91
- FWCI
- 59.17
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by
- Locations
- Statistics Surveys (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1934964, 2022040)
- U.S. Department of Energy (DESC0021358)
- National Institute on Drug Abuse (DA054994-01)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- model interpretability, decision tree, scoring system, generalized additive models, case-based reasoning, dimensionality reduction, physics-informed machine learning, Rashomon set, interpretable reinforcement learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Interpretable machine learning: Fundamental principles and 10 grand challenges. https://app.cochat.ai/papers/10.1214/21-ss133
- The paper
- Rudin, C., Chen, C., Chen, Z., Huang, H., Semenova, L., & Zhong, C. (2022). Interpretable machine learning: Fundamental principles and 10 grand challenges. Statistics Surveys. https://doi.org/10.1214/21-ss133
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.