Interpretable machine learning: Fundamental principles and 10 grand challenges
Open accessThe DOI matches its Crossref record; 335 of the 335 references it cites were found in a registry or an index.
Abstract
Interpretability in machine learning (ML) is crucial for high stakes decisions and troubleshooting. In this work, we provide fundamental principles for interpretable ML, and dispel common misunderstandings that dilute the importance of this crucial topic. We also identify 10 technical challenge areas in interpretable machine learning and provide history and background on each problem. Some of these problems are classically important, and some are recent problems that have arisen in the last few years. These problems are: (1) Optimizing sparse logical models such as decision trees; (2) Optimization of scoring systems; (3) Placing constraints into generalized additive models to encourage sparsity and better interpretability; (4) Modern case-based reasoning, including neural networks and matching for causal inference; (5) Complete supervised disentanglement of neural networks; (6) Complete or even partial unsupervised disentanglement of neural networks; (7) Dimensionality reduction for data visualization; (8) Machine learning models that can incorporate physics and other generative or causal constraints; (9) Characterization of the “Rashomon set” of good models; and (10) Interpretable reinforcement learning. This survey is suitable as a starting point for statisticians and computer scientists interested in working in interpretable machine learning.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“For tabular data, very small interpretable models can often perform as well as black box models on benchmark datasets if they are optimized carefully.”
From Interpretable algorithmic forensics (2023) · cites this paper as [22]
The cited source says what the claim attributes to it.
In the citing paper
“Thus, for either tabular data or images, interpretable models are generally as accurate as the best of the black box models when applied to benchmark datasets (22).”
From Interpretable algorithmic forensics (2023) · cites this paper as [22]
The cited source says what the claim attributes to it.
In the citing paper
“For image data, where neural networks are the only technique that performs well currently, interpretable neural networks perform as well as black box neural networks.”
From Interpretable algorithmic forensics (2023) · cites this paper as [22]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Items 201–335 of 335
Relational Deep Reinforcement Learning
Zambaldi, V., Raposo, D. C., Santoro, A., Bapst, V., Li, Y., Babuschkin, I., Tuyls, K., Reichert, D., Lillicrap, T., Lockhart, E., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.01830 · Cited by 143
Multistep Neural Networks for Data-driven Discovery of Nonlinear Dynamical Systems
Raissi, M., Perdikaris, P. G., & Karniadakis, G. E. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.01236 · Cited by 141
Xie, F., Chakraborty, B., Ong, M. E. H., Goldstein, B. A., & Liu, N. (2020) · JMIR Medical Informatics · doi:10.2196/21798 · Cited by 140
Learning Optimal Decision Trees with SAT
Narodytska, N., Ignatiev, A., Pereira, F., & Marques-Silva, J. P. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/189 · Cited by 134
Learning Optimal Decision Trees Using Caching Branch-and-Bound Search
Aglin, G., Nijssen, S., & Schaus, P. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.5711 · Cited by 129
Sharp Minima Can Generalize For Deep Nets
Dinh, L., Pascanu, R., Bengio, S., & Bengio, Y. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.04933 · Cited by 125
Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable ClaimsPreprint
Brundage, M., Avin, S., Wang, J., et al. (2020) · arXiv (Cornell University) · Cited by 124
Spatially Invariant Unsupervised Object Detection with Convolutional Neural Networks
Crawford, E., & Pineau, J. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33013412 · Cited by 122
The Bayesian Case Model: A Generative Approach for Case-Based Reasoning and Prototype Classification
Kim, B., Rudin, C. D., & Shah, J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1503.01161 · Cited by 117
Self-citationOptimized Scoring Systems: Toward Trust in Machine Learning for Healthcare and Criminal Justice
Rudin, C., & Ustun, B. (2018) · Interfaces · doi:10.1287/inte.2018.0957 · Cited by 117
Self-citationNeural Additive Models: Interpretable Machine Learning with Neural Nets
Agarwal, R., Melnick, L., Frosst, N., Zhang, X., Lengerich, B. J., Caruana, R., & Hinton, G. E. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.13912 · Cited by 117
Toward Interpretable Deep Reinforcement Learning with Linear Model U-Trees
Liu, G., Schulte, O., Wang, Z., & Li, Q. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-10928-8_25 · Cited by 115
Interpretable and Trustworthy Deepfake Detection via Dynamic Prototypes
Trinh, L., Tsang, M., Rambhatla, S., & Liu, Y. (2021) · 2021 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv48630.2021.00202 · Cited by 115
Machine learning for science and society
Rudin, C., & Wagstaff, K. L. (2014) · Machine Learning · doi:10.1007/s10994-013-5425-9 · Cited by 114
Self-citationReinforcement Learning in Healthcare: A SurveyPreprint
Yu, C., Liu, J., & Nemati, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.08796 · Cited by 114
Adaptive treatment of epilepsy via batch-mode reinforcement learning
Guez, A., Vincent, R. D., Avoli, M., & Pineau, J. (2008) · Innovative Applications of Artificial Intelligence · Cited by 113
Exploring the cloud of variable importance for the set of all good models
Dong, J., & Rudin, C. D. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-00264-0 · Cited by 113
Self-citationMargin-based Ranking and an Equivalence between AdaBoost and RankBoost
Rudin, C. D., & Schapire, R. E. (2009) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 109
Self-citationMarchand, M., & Shawe-Taylor, J. S. (2003) · ePrints Soton (University of Southampton) · doi:10.5555/944919.944952 · Cited by 108
Learning Certifiably Optimal Rule Lists for Categorical Data
Angelino, E., Larus-Stone, N., Alabi, D., Seltzer, M., & Rudin, C. (2017) · arXiv · doi:10.48550/arxiv.1704.01701 · Cited by 108
Self-citation(2017) · doi:10.1145/3097983.3098047 · Cited by 104
Self-citationGPU accelerated t-distributed stochastic neighbor embedding
Chan, D. M., Rao, R., Huang, F., & Canny, J. (2019) · Journal of Parallel and Distributed Computing · doi:10.1016/j.jpdc.2019.04.008 · Cited by 103
Deep Reinforcement Learning for Dynamic Treatment Regimes on Medical Registry Data
Liu, Y., Logan, B., Liu, N., Xu, Z., Tang, J., & Wang, Y. (2017) · 2017 IEEE International Conference on Healthcare Informatics (ICHI) · doi:10.1109/ichi.2017.45 · Cited by 102
Yang, Y., Morillo, I. G., & Hospedales, T. M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.06988 · Cited by 100
Clinical Prediction Models for Sleep Apnea: The Importance of Medical History over Symptoms
Ustun, B., Westover, M. B., Rudin, C., & Bianchi, M. T. (2016) · Journal of Clinical Sleep Medicine · doi:10.5664/jcsm.5476 · Cited by 99
Self-citationClassification tree analysis using TARGET
Gray, J. B., & Fan, G. (2007) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2007.03.014 · Cited by 97
Interpretable and Steerable Sequence Learning via Prototypes
Ming, Y., Xu, P., Qu, H., & Ren, L. (2019) · Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3292500.3330908 · Cited by 97
Exact Rule Learning via Boolean Compressed Sensing
Malioutov, D., & Varshney, K. R. (2013) · International Conference on Machine Learning · Cited by 91
ViCE: visual counterfactual explanations for machine learning models
Gomez, O., Holter, S., Yuan, J., & Bertini, E. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377536 · Cited by 91
Theory and Evaluation Metrics for Learning Disentangled RepresentationsPreprint
Do, K., & Tran, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.09961 · Cited by 87
Differentiable Physics-informed Graph NetworksPreprint
Seo, S., & Liu, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.02950 · Cited by 83
Learning Optimal Decision Trees with MaxSAT and its Integration in AdaBoost
Hu, H., Siala, M., Hébrard, E., & Huguet, M. (2020) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/163 · Cited by 79
Non-parametric policy gradients: a unified treatment of propositional and relational domains
Kersting, K., & Driessens, K. (2008) · International Conference on Machine Learning · doi:10.1145/1390156.1390214 · Cited by 75
Neural-net-induced Gaussian process regression for function approximation and PDE solution
Pang, G., Yang, L., & Karniadakis, G. E. (2019) · Journal of Computational Physics · doi:10.1016/j.jcp.2019.01.045 · Cited by 74
Conservative Q-Improvement: Reinforcement Learning for an Interpretable Decision-Tree PolicyPreprint
Roth, A. M., Topin, N., Jamshidi, P., & Veloso, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.01180 · Cited by 73
Interpretability Beyond Classification Output: Semantic Bottleneck NetworksPreprint
Losch, M., Fritz, M., & Schiele, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.10882 · Cited by 73
Knowledge Infused Learning (K-IL): Towards Deep Incorporation of Knowledge in Deep LearningPreprint
Kursuncu, U., Gaur, M., & Sheth, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.00512 · Cited by 71
Petersen, A. J., Witten, D., & Simon, N. (2015) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.2015.1073155 · Cited by 70
"Why Should You Trust My Explanation?" Understanding Uncertainty in LIME Explanations
Zhang, Y., Song, K., Sun, Y., Tan, S., & Udell, M. (2019) · arXiv · doi:10.48550/arxiv.1904.12991 · Cited by 68
Mining optimal decision trees from itemset lattices
Nijssen, S., & Fromont, E. (2007) · Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1281192.1281250 · Cited by 67
Attraction-Repulsion Spectrum in Neighbor EmbeddingsPreprint
Böhm, J. N., Berens, P., & Kobak, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.08902 · Cited by 67
Sparse Partially Linear Additive Models
Lou, Y., Bien, J., Caruana, R., & Gehrke, J. E. (2015) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.2015.1089775 · Cited by 66
Learning customized and optimized lists of rules with mathematical programming
Rudin, C. D., & Ertekin, Ş. (2018) · Mathematical Programming Computation · doi:10.1007/s12532-018-0143-8 · Cited by 65
Self-citationDisentangling Factors of Variation via Generative Entangling
Desjardins, G., Courville, A. C., & Bengio, Y. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1210.5474 · Cited by 64
Handling Model Uncertainty and Multiplicity in Explanations via Model Reconciliation
Sreedharan, S., Chakraborti, T., & Kambhampati, S. (2018) · Proceedings of the International Conference on Automated Planning and Scheduling · doi:10.1609/icaps.v28i1.13930 · Cited by 64
Spiegelhalter, D. (2020) · Harvard Data Science Review · doi:10.1162/99608f92.cb91a35a · Cited by 64
Additive models with trend filtering
Sadhanala, V., & Tibshirani, R. J. (2019) · The Annals of Statistics · doi:10.1214/19-aos1833 · Cited by 63
Combining Kernel and Model Based Learning for HIV Therapy Selection
Parbhoo, S., Bogojeska, J., Zazzi, M., Röth, V., & Doshi‐Velez, F. (2017) · PubMed · doi:10.5451/unibas-ep59234 · Cited by 63
Conditional deep surrogate models for stochastic, high-dimensional, and multi-fidelity systems
Yang, Y., & Perdikaris, P. (2019) · Computational Mechanics · doi:10.1007/s00466-019-01718-y · Cited by 63
Distilling Deep Reinforcement Learning Policies in Soft Decision Trees
Coppens, Y., Efthymiadis, K., Lenaerts, T., & Nowé, A. (2019) · Dépôt institutionnel de l'Université libre de Bruxelles (Université Libre de Bruxelles) · Cited by 63
Optimal constraint-based decision tree induction from itemset lattices
Nijssen, S., & Fromont, É. (2010) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-010-0174-x · Cited by 62
An Interpretable Model with Globally Consistent Explanations for Credit Risk
Chen, C., Lin, K., Rudin, C. D., Shaposhnik, Y., Wang, S., & Wang, T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.12615 · Cited by 61
Self-citationMurTree: Optimal Classification Trees via Dynamic Programming and SearchPreprint
Demirović, E., Lukina, A., Hébrard, E., Chan, J., Bailey, J. P., Leckie, C. A., Ramamohanarao, K., & Stuckey, P. J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.12652 · Cited by 61
A SAT-Based Approach to Learn Explainable Decision Sets
Ignatiev, A., Pereira, F., Narodytska, N., & Marques-Silva, J. P. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-94205-6_41 · Cited by 59
Semenova, L., & Rudin, C. D. (2019) · arXiv (Cornell University) · Cited by 59
Self-citation(2018) · ENLIGHTEN (Jurnal Bimbingan dan Konseling Islam) · Cited by 57
Learning with Decision Lists of Data-Dependent Features
Marchand, M., & Sokolova, M. (2005) · Journal of Machine Learning Research · Cited by 56
Learning Optimized Risk Scores
Ustun, B., & Rudin, C. D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.00168 · Cited by 56
Self-citationGraying the black box: Understanding DQNs
Zahavy, T., Ben-Zrihem, N., & Mannor, S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.02658 · Cited by 54
Hierarchical and Interpretable Skill Acquisition in Multi-task Reinforcement Learning
Shu, T., Xiong, C., & Socher, R. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.07294 · Cited by 54
In AI We Trust? Factors That Influence Trustworthiness of AI-infused Decision-Making Processes
Ashoori, M., & Weisz, J. D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.02675 · Cited by 53
GA Tree: genetically evolved decision trees
Papagelis, A., & Kalles, D. (2002) · IEEE International Conference on Tools with Artificial Intelligence (ICTAI) · doi:10.1109/tai.2000.889871 · Cited by 50
Silva, A., Gombolay, M., Killian, T. W., Jimenez, I., & Son, S.-H. (2020) · International Conference on Artificial Intelligence and Statistics · Cited by 50
Learning Theory Analysis for Association Rules and Sequential Event Prediction
Rudin, C. D., Letham, B., & Madigan, D. (2013) · DSpace@MIT (Massachusetts Institute of Technology) · doi:10.7916/d82n50c1 · Cited by 49
Self-citationA comparison of methods for the fitting of generalized additive models
Binder, H., & Tutz, G. (2007) · Statistics and Computing · doi:10.1007/s11222-007-9040-0 · Cited by 49
Boolean Decision Rules via Column Generation
Dash, S., Günlük, O., & Wei, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.09901 · Cited by 49
Getting a CLUE: A Method for Explaining Uncertainty Estimates
Antorán, J., Bhatt, U., Adel, T., Weller, A., & Hernández-Lobato, J. M. (2021) · Apollo - University of Cambridge Repository · doi:10.17863/cam.69929 · Cited by 49
BEAMES: Interactive Multimodel Steering, Selection, and Inspection for Regression Tasks
Das, S., Cashman, D., Chang, R., & Endert, A. (2019) · IEEE Computer Graphics and Applications · doi:10.1109/mcg.2019.2922592 · Cited by 47
This Looks Like That, Because ... Explaining Prototypes for Interpretable Image Recognition
Nauta, M., Jutte, A., Provoost, J., & Seifert, C. (2021) · Communications in Computer and Information Science · doi:10.1007/978-3-030-93736-2_34 · Cited by 47
You shouldn’t trust me: Learning models which conceal unfairness from multiple explanation methods.
Dimanov, B., Bhatt, U., Jamnik, M., & Weller, A. (2020) · Apollo (University of Cambridge) · doi:10.17863/cam.48825 · Cited by 45
Computing Optimal Decision Sets with SAT
Yu, J., Ignatiev, A., Stuckey, P. J., & Bodic, P. L. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58475-7_55 · Cited by 45
Ustun, B., & Rudin, C. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098161 · Cited by 44
Self-citationMLIC: A MaxSAT-Based Framework for Learning Interpretable Classification Rules
Malioutov, D., & Meel, K. S. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-98334-9_21 · Cited by 44
Atrey, A., Clary, K., & Jensen, D. D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.05743 · Cited by 44
Rudin, C. D., & Shaposhnik, Y. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3395422 · Cited by 43
Self-citationIn Defense of C4.5: Notes on Learning One-Level Decision Trees
Elomaa, T. (1994) · Elsevier eBooks · doi:10.1016/b978-1-55860-335-6.50016-7 · Cited by 41
Generalized and Scalable Optimal Sparse Decision Trees
Lin, J., Zhong, C., Hu, D. J., Rudin, C. D., & Seltzer, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.08690 · Cited by 41
Self-citationEthical Implementation of Artificial Intelligence to Select Embryos in In Vitro Fertilization
Afnan, M. A. M., Rudin, C. D., Conitzer, V., Savulescu, J., Mishra, A., Liu, Y., & Afnan, M. (2021) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3461702.3462589 · Cited by 41
Self-citationA Reinforcement Learning–Based Method for Management of Type 1 Diabetes: Exploratory Study
Javad, M. O. M., Agboola, S. O., Jethwani, K., Zeid, A., & Kamarthi, S. V. (2019) · JMIR Diabetes · doi:10.2196/12905 · Cited by 39
Physics-aware Difference Graph Networks for Sparsely-Observed Dynamics
Seo, S., Meng, C., & Liu, Y. (2020) · International Conference on Learning Representations · Cited by 39
Learning Optimal Classification Trees: Strong Max-Flow FormulationsPreprint
Aghaei, S., Gómez, A., & Vayanos, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.09142 · Cited by 39
Optimal decision trees for the algorithm selection problem: integer programming based approaches
Boas, M. G. V., Santos, H. G., Merschmann, L. H. D. C., & Berghe, G. V. (2019) · International Transactions in Operational Research · doi:10.1111/itor.12724 · Cited by 38
Getting a CLUE: A Method for Explaining Uncertainty EstimatesPreprint
Antorán, J., Bhatt, U., Adel, T., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.06848 · Cited by 38
ProtoPShare: Prototype Sharing for Interpretable Image Classification and Similarity DiscoveryPreprint
Rymarczyk, D., Struski, Ł., Tabor, J., & Zieliński, B. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.14340 · Cited by 38
Deep Weighted Averaging Classifiers
Card, D., Zhang, M., & Smith, N. A. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287595 · Cited by 37
PDE-Constrained Gaussian Process Model on Material Removal Rate of Wire Saw Slicing Process
Zhao, H., Jin, R., Wu, S., et al. (2011) · Journal of Manufacturing Science and Engineering · doi:10.1115/1.4003617 · Cited by 35
Box drawings for learning with imbalanced data
Goh, S. T., & Rudin, C. (2014) · Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2623330.2623648 · Cited by 35
Self-citationGuan, L., Verma, M., Guo, S., Zhang, R., & Subbarao Kambhampati, B. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.14804 · Cited by 35
Do Not Trust Additive Explanations
Gosiewska, A., & Biecek, P. (2019) · arXiv · doi:10.48550/arxiv.1903.11420 · Cited by 34
Physics informed deep learning for computational elastodynamics without labeled dataPreprint
Rao, C., Sun, H. Y., & Yang, L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.08472 · Cited by 33
How to see hidden patterns in metamaterials with interpretable machine learning
Chen, Z., Ogren, A. C., Daraio, C., Brinson, L. C., & Rudin, C. D. (2022) · Extreme Mechanics Letters · doi:10.1016/j.eml.2022.101895 · Cited by 32
Self-citationStrongly agree or strongly disagree?: Rating features in Support Vector Machines
Carrizosa, E., Nogales-Gómez, A., & Morales, D. R. (2015) · Information Sciences · doi:10.1016/j.ins.2015.09.031 · Cited by 30
The Thing That We Tried Didn't Work Very Well : Deictic Representation in Reinforcement LearningPreprint
Finney, S. J., Gardiol, N. H., Kaelbling, L. P., & Oates, T. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.0567 · Cited by 30
Ghosh, B., & Meel, K. S. (2019) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3306618.3314283 · Cited by 30
Learning Optimized Or's of And'sPreprint
Wang, T., & Rudin, C. D. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.02210 · Cited by 29
Self-citationAn Optimization Approach to Learning Falling Rule ListsPreprint
Chen, C., & Rudin, C. D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.02572 · Cited by 29
Self-citationAutonomous Self-Explanation of Behavior for Interactive Reinforcement Learning Agents
Fukuchi, Y., Osawa, M., Yamakawa, H., & Imai, M. (2017) · Proceedings of the 5th International Conference on Human Agent Interaction · doi:10.1145/3125739.3125746 · Cited by 28
Wang, J., Oh, J., Wang, H., & Wiens, J. (2018) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3219819.3220070 · Cited by 28
Gaining Free or Low-Cost Interpretability with Interpretable Partial Substitute
Wang, T. (2019) · International Conference on Machine Learning · Cited by 28
Predictive Multiplicity in Classification
Marx, C. T., Calmon, F. P., & Ustun, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.06677 · Cited by 28
FLAME: A Fast Large-scale Almost Matching Exactly Approach to Causal InferencePreprint
Wang, T., Morucci, M., Awan, M. U., Liu, Y., Roy, S., Rudin, C. D., & Volfovsky, A. (2021) · arXiv (Cornell University) · Cited by 27
Self-citationOn Equivalence Relationships Between Classification and Ranking Algorithms
Ertekin, Ş., & Rudin, C. D. (2011) · DSpace@MIT (Massachusetts Institute of Technology) · doi:10.5555/1953048.2078199 · Cited by 26
Self-citationA Unifying Perspective on Neighbor Embeddings along the Attraction-Repulsion SpectrumPreprint
Böhm, J. N., Berens, P., & Kobak, D. (2021) · arXiv (Cornell University) · Cited by 26
FLAME: A Fast Large-scale Almost Matching Exactly Approach to Causal InferencePreprint
Wang, T., Morucci, M., Awan, M. U., Liu, Y., Roy, S., Rudin, C. D., & Volfovsky, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.06315 · Cited by 24
Self-citationAtSNE: Efficient and Robust Visualization on GPU through Hierarchical Optimization
Fu, C., Zhang, Y., Cai, D., & Ren, X. (2019) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3292500.3330834 · Cited by 24
Structure of association rule classifiers: a review
Vanhoof, K., & Depaire, B. (2010) · International Conference on Intelligent Systems and Knowledge Engineering (ISKE) · doi:10.1109/iske.2010.5680784 · Cited by 22
Learning Explainable Decision Rules via Maximum Satisfiability
Cao, H. E. C., Sarlin, R., & Jung, A. (2020) · IEEE Access · doi:10.1109/access.2020.3041040 · Cited by 22
Sokolova, M., Marchand, M., Japkowicz, N., & Shawe-Taylor, J. S. (2002) · ePrints Soton (University of Southampton) · Cited by 20
Supersparse Linear Integer Models for Predictive Scoring SystemsPreprint
Ustun, B., Tracà, S., & Rudin, C. D. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1306.5860 · Cited by 19
Self-citationOptimal Generalized Decision Trees via Integer ProgrammingPreprint
Günlük, O., Kalagnanam, J., Li, M., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.03225 · Cited by 19
Learning Fair Rule ListsPreprint
Aïvodji, U., Ferry, J., Gambs, S., Huguet, M., & Siala, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.03977 · Cited by 18
ProtoryNet - Interpretable Text Classification Via Prototype TrajectoriesPreprint
Hong, D., Wang, T., & Baek, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.01777 · Cited by 17
Forward stability and model path selection
Kissel, N., & Mentch, L. (2024) · Statistics and Computing · doi:10.1007/s11222-024-10395-8 · Cited by 17
Rounding Methods for Discrete Linear Classification
Chevaleyre, Y., Koriche, F., & Zucker, J. (2013) · Cited by 15
A Provable Algorithm for Learning Interpretable Scoring Systems
Sokolovska, N., Chevaleyre, Y., & Zucker, J. (2018) · Base Institutionnelle de Recherche de l'université Paris-Dauphine (BIRD) (University Paris-Dauphine) · Cited by 15
Prototypal Analysis and Prototypal RegressionPreprint
Wu, C., & Tabak, E. G. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1701.08916 · Cited by 14
Fitted Q-Learning for Relational DomainsPreprint
Das, S., Natarajan, S., Roy, K., Parr, R., & Kersting, K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.05595 · Cited by 14
Characterizing Fairness Over the Set of Good Models Under Selective Labels
Coston, A., Rambachan, A., & Chouldechova, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.00352 · Cited by 13
Interval Coded Scoring: a toolbox for interpretable scoring systems
Billiet, L., Huffel, S. V., & Belle, V. V. (2018) · PeerJ Computer Science · doi:10.7717/peerj-cs.150 · Cited by 12
Barnett, A. J., Schwartz, F. R., Tao, C., Chen, C., Ren, Y., Lo, J. Y.-C., & Rudin, C. D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.12308 · Cited by 12
Self-citationHow to represent part-whole hierarchies in a neural networkPreprint
Hinton, G. E. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.12627 · Cited by 11
Optimal Decision Lists using SATPreprint
Yu, J., Ignatiev, A., Bodic, P. L., & Stuckey, P. J. (2020) · arXiv (Cornell University) · doi:10.4230/lipics.cp.2021.44 · Cited by 10
The fused lasso penalty for learning interpretable medical scoring systems
Sokolovska, N., Chevaleyre, Y., Clément, K., & Zucker, J. (2017) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2017.7966427 · Cited by 9
Designing Interpretable Approximations to Deep Reinforcement LearningPreprint
Dahlin, N., Kalagarla, K. C., Naik, N., Jain, R., & Nuzzo, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.14785 · Cited by 9
A Fast Way to Produce Optimal Fixed-Depth Decision Trees.
Farhangfar, A., Greiner, R., & Zinkevich, M. (2008) · ISAIM · Cited by 8
Interval Coded Scoring extensions for larger problems
Billiet, L., Huffel, S. V., & Belle, V. V. (2017) · IEEE Symposium on Computers and Communications (ISCC) · doi:10.1109/iscc.2017.8024529 · Cited by 8
Generalized Sparse Additive Models
Haris, A., Simon, N., & Shojaie, A. (2019) · PubMed · doi:10.48550/arxiv.1903.04641 · Cited by 8
Zhang, K., Wang, Y., Du, J., Chu, B. K., Celi, L. A., Kindle, R. D., & Doshi‐Velez, F. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.03309 · Cited by 8
Partially Interpretable Estimators (PIE): Black-Box-Refined Interpretable Machine LearningPreprint
Wang, T., Yang, J., Li, Y., & Wang, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.02410 · Cited by 7
RAPID: Rapid and Precise Interpretable Decision Sets
Dhamnani, S., Singal, D., Sinha, R., Mohandoss, T., & Dash, M. (2019) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata47090.2019.9006159 · Cited by 5
Interpretable-AI Policies using Evolutionary Nonlinear Decision Trees for Discrete Action Systems.Preprint
Dhebar, Y., Deb, K., Nageshrao, S. P., Zhu, L., & Filev, D. (2020) · arXiv (Cornell University) · Cited by 5
Parikh, H., Rudin, C. D., & Volfovsky, A. (2019) · Observational Studies · doi:10.1353/obs.2019.0006 · Cited by 4
Self-citationConvex-constrained Sparse Additive Modeling and Its ExtensionsPreprint
Yin, J., & Yu, Y. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.00687 · Cited by 3
Forward Stability and Model Path SelectionPreprint
Kissel, N., & Mentch, L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.03462 · Cited by 2
(2008) · ScholarlyCommons (University of Pennsylvania)
Research works printed but not linked: 15
- Bousquet, O. and Elisseeff, A. (2002). Stability and generalization. Journal of Machine Learning Research 2 499–526.
- Dobkin, D., Fulton, T., Gunopulos, D., Kasif, S. and Salzberg, S. (1997). Induction of shallow decision trees. IEEE Transactions on Pattern Analysis and Machine Intelligence.
- Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A. and Bengio, Y. (2014). Generative Adversarial Nets. In Proceedings of Conference on Neural Information Processing Systems (NeurIPS) 27 2672–2680.
- Hammond, K. (1989). Proceedings of the Second Case-Based Reasoning Workshop (DARPA). Morgan Kanfmann Publishers, Inc., San Mateo, CA.
- Hu, X., Rudin, C. and Seltzer, M. (2019). Optimal Sparse Decision Trees. In Proceedings of Conference on Neural Information Processing Systems (NeurIPS).
- Juozapaitis, Z., Koul, A., Fern, A., Erwig, M. and Doshi-Velez, F. (2019). Explainable reinforcement learning via reward decomposition. In Proceedings of the IJCAI/ECAI Workshop on Explainable Artificial Intelligence.
- Kodratoff, Y. (1994). The comprehensibility manifesto. KDD Nugget Newsletter 94.
- Kolodner, J. L. (1988). Proceedings of the Case-Based Reasoning Workshop (DARPA). Morgan Kaufmmnn Publishers, Inc., San Mateo, CA.
- Kosiorek, A., Sabour, S., Teh, Y. W. and Hinton, G. E. (2019). Stacked capsule autoencoders. In Proceedings of Conference on Neural Information Processing Systems (NeurIPS) 15512–15522.
- Quinlan, J. R. (1993). C4.5: Programs for Machine Learning. Morgan Kaufmann.
- Sabour, S., Frosst, N. and Hinton, G. E. (2017). Dynamic routing between capsules. In Proceedings of Conference on Neural Information Processing Systems (NeurIPS) 3856–3866.
- Srebro, N., Sridharan, K. and Tewari, A. (2010). Smoothness, low noise and fast rates. In Proceedings of Conference on Neural Information Processing Systems (NeurIPS) 2199–2207.
- Yang, H., Rudin, C. and Seltzer, M. (2017). Scalable Bayesian rule lists. In Proceedings of International Conference on Machine Learning (ICML) 3921–3930. · DOI
- Zhu, Z., Luo, P., Wang, X. and Tang, X. (2014). Multi-view perceptron: a deep model for learning face identity and view representations. In Proceedings of Conference on Neural Information Processing Systems (NeurIPS) 217–225.
- Hastie, T. J. and Tibshirani, R. J. (1990). Generalized additive models 43. CRC press.
Cited by
Scientific Machine Learning Through Physics–Informed Neural Networks: Where we are and What’s Next
Cuomo, S., Di Cola, V. S., Giampaolo, F., Rozza, G., Raissi, M., & Piccialli, F. (2022) · Journal of Scientific Computing · doi:10.1007/s10915-022-01939-z · Cited by 2,784
Interpreting Black-Box Models: A Review on Explainable Artificial Intelligence
Hassija, V., Chamola, V., Mahapatra, A., Singal, A., Goel, D., Huang, K., Scardapane, S., Spinelli, I., Mahmud, M., & Hussain, A. (2024) · Cognitive Computation · doi:10.1007/s12559-023-10179-8 · Cited by 2,495
Ali, S., Abuhmed, T., El-Sappagh, S., Muhammad, K., Alonso-Moral, J. M., Confalonieri, R., Guidotti, R., Del Ser, J., Díaz-Rodríguez, N., & Herrera, F. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101805 · Cited by 1,762
Aligning artificial intelligence with climate change mitigation
Kaack, L. H., Donti, P. L., Strubell, E., Kamiya, G., Creutzig, F., & Rolnick, D. (2022) · Nature Climate Change · doi:10.1038/s41558-022-01377-7 · Cited by 570
Bottom-up Coarse-Graining: Principles and Perspectives
Jin, J., Pak, A. J., Durumeric, A. E. P., Loose, T. D., & Voth, G. A. (2022) · Journal of Chemical Theory and Computation · doi:10.1021/acs.jctc.2c00643 · Cited by 305
Metadata
- Authors
- Rudin, C., Chen, C., Chen, Z., Huang, H., Semenova, L., & Zhong, C.
- Venue
- Statistics Surveys
- Publisher
- Institute of Mathematical Statistics
- Published
- 2022-01-01
- Volume
- 16
- Issue
- none
- Pages
- 1-85
- ISSN
- 1935-7516 (ISSN-L)
- OpenAlex
- W3137125108
- Cited by
- 958
- References
- 335
- Citations by year
- 2026: 202 · 2025: 247 · 2024: 205 · 2023: 179 · 2022: 91
- FWCI
- 59.17
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by
- Locations
- Statistics Surveys (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1934964, 2022040)
- U.S. Department of Energy (DESC0021358)
- National Institute on Drug Abuse (DA054994-01)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- model interpretability, decision tree, scoring system, generalized additive models, case-based reasoning, dimensionality reduction, physics-informed machine learning, Rashomon set, interpretable reinforcement learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.