Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model
Open accessAbstract
We aim to produce predictive models that are not only accurate, but are also interpretable to human experts. Our models are decision lists, which consist of a series of if…then…statements (e.g., if high blood pressure, then stroke) that discretize a high-dimensional, multivariate feature space into a series of simple, readily interpretable decision statements. We introduce a generative model called Bayesian Rule Lists that yields a posterior distribution over possible decision lists. It employs a novel prior structure to encourage sparsity. Our experiments show that Bayesian Rule Lists has predictive accuracy on par with the current top algorithms for prediction in machine learning. Our method is motivated by recent developments in personalized medicine, and can be used to produce highly accurate and interpretable medical scoring systems. We demonstrate this by producing an alternative to the CHADS$_{2}$ score, actively used in clinical practice for estimating the risk of stroke in patients that have atrial fibrillation. Our model is as interpretable as CHADS$_{2}$, but more accurate.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Bayesian Rule Lists extract the comprehensible global predictor c_g as a decision list, which consists of a series of if-then statements discretizing the whole feature space into simple and directly interpretable decision statements.
“To this end, they propose Bayesian Rule Lists (BRL) f to extract the comprehensible global predictor c g as a decision list . A decision list consists of a series of if-then statements discretizing the whole feature space into a set of simple and directly interpretable decision statements.”
From A Survey of Methods for Explaining Black Box Models (2019) · cites this paper as [57]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538 · see citing works
LIBSVM: A library for support vector machines
Chang, C.-C., & Lin, C.-J. (2011) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/1961189.1961199 · Cited by 40,918 · see citing works
The Nature of Statistical Learning Theory
Vapnik, V. N. (1995) · doi:10.1007/978-1-4757-2440-0 · Cited by 39,138 · see citing works
UCI Machine Learning Repository
Asuncion, A. (2007) · Medical Entomology and Zoology · Cited by 24,190 · see citing works
Classification and Regression Trees.
Gordon, A. D., Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984) · Biometrics · doi:10.2307/2530946 · Cited by 23,834 · see citing works
Classification and Regression Trees.
Ryzin, J. V., Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1986) · Journal of the American Statistical Association · doi:10.2307/2288003 · Cited by 21,028 · see citing works
The magical number seven, plus or minus two: Some limits on our capacity for processing information.
Miller, G. A. (1956) · Psychological Review · doi:10.1037/h0043158 · Cited by 17,812 · see citing works
The magical number seven, plus or minus two: Some limits on our capacity for processing information.
Miller, G. A. (1994) · Psychological Review · doi:10.1037/0033-295x.101.2.343 · Cited by 17,469 · see citing works
Inference from iterative simulation using multiple sequences
Gelman, A., & Rubin, D. B. (1992) · Statistical Science · doi:10.1214/ss/1177011136 · Cited by 16,627 · see citing works
Breiman, L. (1996) · Machine Learning · doi:10.1007/bf00058655 · Cited by 16,325 · see citing works
Knaus, W. A., Draper, E. A., Wagner, D. P., & Zimmerman, J. E. (1985) · Critical Care Medicine · doi:10.1097/00003246-198510000-00009 · Cited by 15,333 · see citing works
APACHE II-A Severity of Disease Classification System
Knaus, W. A., Draper, E. A., Wagner, D. P., & Zimmerman, J. E. (1986) · Critical Care Medicine · doi:10.1097/00003246-198608000-00028 · Cited by 13,379 · see citing works
Fast algorithms for mining association rules
Agrawal, R., & Srikant, R. (1998) · Cited by 10,706 · see citing works
Binary Codes capable of currecting deletions, insertions, and reversals
Levenshtein, V. I. (1965) · Soviet physics. Doklady · Cited by 10,387 · see citing works
Lip, G. Y. H., Nieuwlaat, R., Pisters, R., Lane, D. A., & Crijns, H. J. G. M. (2010) · Chest · doi:10.1378/chest.09-1584 · Cited by 6,879 · see citing works
LIBLINEAR: A Library for Large Linear Classification
Fan, R.-E., Chang, K., Hsieh, C., Xiang-rui, W., & Lin, C. (2008) · doi:10.5555/1390681.1442794 · Cited by 6,680 · see citing works
Gage, B. F., Waterman, A. D., Shannon, W., Boechler, M., Rich, M. W., & Radford, M. J. (2001) · JAMA · doi:10.1001/jama.285.22.2864 · Cited by 4,832 · see citing works
Statistical Modeling: The Two Cultures (with comments and a rejoinder by the author)
Breiman, L. (2001) · Statistical Science · doi:10.1214/ss/1009213726 · Cited by 4,340 · see citing works
The TIMI Risk Score for Unstable Angina/Non–ST Elevation MI
Antman, E. M., Cohen, M. C., Bernink, P. J. L. M., McCabe, C. H., Horacek, T., Papuchis, G., Mautner, B., Corbalán, R., Radley, D. R., & D, E. B. M. (2000) · JAMA · doi:10.1001/jama.284.7.835 · Cited by 3,221 · see citing works
Galit Shmueli, A. (2010) · Statistical Science · doi:10.1214/10-sts330 · Cited by 3,121 · see citing works
Lim, W., van der Eerden, M. M., Laing, R., Boersma, W., Karalus, N., Town, G., Lewis, S., & Macfarlane, J. (2003) · Thorax · doi:10.1136/thorax.58.5.377 · Cited by 3,059 · see citing works
Multi-Interval Discretization of Continuous-Valued Attributes for Classification Learning
Fayyad, U. M., & Irani, K. B. (1993) · International Joint Conference on Artificial Intelligence · Cited by 2,870 · see citing works
The Nature of Statistical Learning
Vapnik, V. N. (1995) · Medical Entomology and Zoology · Cited by 2,295 · see citing works
Integrating classification and association rule mining
Liu, B., Hsu, W., & Ma, Y. (1998) · Cited by 2,247 · see citing works
The robust beauty of improper linear models in decision making.
Dawes, R. M. (1979) · American Psychologist · doi:10.1037/0003-066x.34.7.571 · Cited by 1,970 · see citing works
Supervised and Unsupervised Discretization of Continuous Features
Dougherty, J., Kohavi, R., & Sahami, M. (1995) · Elsevier eBooks · doi:10.1016/b978-1-55860-377-6.50032-3 · Cited by 1,946 · see citing works
BART: Bayesian additive regression trees
Chipman, H. A., George, E. I., & McCulloch, R. E. (2010) · The Annals of Applied Statistics · doi:10.1214/09-aoas285 · Cited by 1,875 · see citing works
Very Simple Classification Rules Perform Well on Most Commonly Used Datasets
Holte, R. C. (1993) · Machine Learning · doi:10.1023/a:1022631118932 · Cited by 1,800 · see citing works
Scalable algorithms for association mining
Zaki, M. J. (2000) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/69.846291 · Cited by 1,769 · see citing works
Inductive Logic Programming: Theory and methods
Muggleton, S., & de Raedt, L. (1994) · The Journal of Logic Programming · doi:10.1016/0743-1066(94)90035-3 · Cited by 1,537 · see citing works
Mining quantitative association rules in large relational tables
Srikant, R., & Agrawal, R. (1996) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/233269.233311 · Cited by 1,490 · see citing works
Predictive learning via rule ensembles
Friedman, J. H., & Popescu, B. E. (2008) · The Annals of Applied Statistics · doi:10.1214/07-aoas148 · Cited by 1,315 · see citing works
Statistical modeling: The two cultures
Breiman, L. (2001) · Cited by 1,314 · see citing works
CMAR: accurate and efficient classification based on multiple class-association rules
Li, W., Han, J., & Pei, J. (2002) · Proceedings · doi:10.1109/icdm.2001.989541 · Cited by 1,207 · see citing works
Heuristics of instability and stabilization in model selection
Breiman, L. (1996) · The Annals of Statistics · doi:10.1214/aos/1032181158 · Cited by 1,144 · see citing works
Rivest, R. L. (1987) · Machine Learning · doi:10.1023/a:1022607331053 · Cited by 894 · see citing works
BART: Bayesian Additive Regression Trees
Chipman, H., George, E. I., & McCulloch, R. E. (2006) · ScholarlyCommons (University of Pennsylvania) · Cited by 889 · see citing works
CPAR: Classification based on Predictive Association Rules
Yin, X., & Han, J. (2003) · Proceedings of the 2003 SIAM International Conference on Data Mining · doi:10.1137/1.9781611972733.40 · Cited by 833 · see citing works
Chipman, H., George, E. I., & McCulloch, R. E. (1998) · Journal of the American Statistical Association · doi:10.1080/01621459.1998.10473750 · Cited by 738 · see citing works
Comprehensible classification models
Freitas, A. A. (2014) · ACM SIGKDD Explorations Newsletter · doi:10.1145/2594473.2594475 · Cited by 575 · see citing works
Identifying Patients at High Risk for Stroke Despite Anticoagulation
Lip, G. Y. H., Frison, L., Halperin, J. L., & Lane, D. A. (2010) · Stroke · doi:10.1161/strokeaha.110.590257 · Cited by 493 · see citing works
Efficient mining of both positive and negative association rules
Wu, X., Zhang, C., & Zhang, S. (2004) · ACM Transactions on Information Systems · doi:10.1145/1010614.1010616 · Cited by 466 · see citing works
Stang, P. E., Ryan, P. B., Racoosin, J. A., Overhage, J. M., Hartzema, A. G., Reich, C., Welebob, E., Scarnecchia, T., & Woodcock, J. (2010) · Annals of Internal Medicine · doi:10.7326/0003-4819-153-9-201011020-00010 · Cited by 441 · see citing works
Huysmans, J., Dejaeger, K., Mues, C., Vanthienen, J., & Baesens, B. (2011) · Decision Support Systems · doi:10.1016/j.dss.2010.12.003 · Cited by 425 · see citing works
An implementation of the FP-growth algorithm
Borgelt, C. (2005) · doi:10.1145/1133905.1133907 · Cited by 327 · see citing works
Dension, D. (1998) · Biometrika · doi:10.1093/biomet/85.2.363 · Cited by 318 · see citing works
Informal covariation assessment: Data-based versus theory-based judgments
Jennings, D. L., Amabile, T. M., & Ross, L. D. (1982) · Cambridge University Press eBooks · doi:10.1017/cbo9780511809477.016 · Cited by 314 · see citing works
Bayesian Inference on Network Traffic Using Link Count Data
Tebaldi, C., & West, M. (1998) · Journal of the American Statistical Association · doi:10.1080/01621459.1998.10473707 · Cited by 312 · see citing works
(2012) · Cited by 253 · see citing works
Bayesian Additive Regression Trees
Chipman, H., George, E. I., & McCulloch, R. E. (2006) · Cited by 232 · see citing works
Computer‐Assisted Keyword and Document Set Discovery from Unstructured Text
King, G., Lam, P., & Roberts, M. E. (2017) · American Journal of Political Science · doi:10.1111/ajps.12291 · Cited by 216 · see citing works
Chipman, H., George, E. I., & McCulloch, R. E. (2002) · Machine Learning · doi:10.1023/a:1013916107446 · Cited by 185 · see citing works
Souillard‐Mandar, W., Davis, R., Rudin, C. D., Au, R., Libon, D. J., Swenson, R. A., Price, C. C., Lamar, M., & Penney, D. L. (2015) · Machine Learning · doi:10.1007/s10994-015-5529-5 · Cited by 163 · see citing works
Self-citation(2015) · International Conference on Artificial Intelligence and Statistics · Cited by 162 · see citing works
Self-citationDynamic Trees for Learning and Design
Taddy, M. A., Gramacy, R. B., & Polson, N. G. (2011) · Journal of the American Statistical Association · doi:10.1198/jasa.2011.ap09769 · Cited by 113 · see citing works
Meinshausen, N. (2010) · The Annals of Applied Statistics · doi:10.1214/10-aoas367 · Cited by 107 · see citing works
Using Decision Lists to Construct Interpretable and Parsimonious Treatment Regimes
Zhang, Y., Laber, E. B., Tsiatis, A. A., & Davidian, M. (2015) · Biometrics · doi:10.1111/biom.12354 · Cited by 105 · see citing works
Weighted Substructure Mining for Image Analysis
Nowozin, S., Tsuda, K., Uno, T., Kudo, T., & Bakır, G. (2007) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2007.383171 · Cited by 89 · see citing works
Expert systems : the technology of knowledge management and decision making for the 21st century
Leondes, C. T. (2002) · Cited by 88 · see citing works
Bayesian CART: Prior Specification and Posterior Simulation
Wu, Y., Tjelmeland, H., & West, M. (2007) · Journal of Computational and Graphical Statistics · doi:10.1198/106186007x180426 · Cited by 74 · see citing works
Stefan Rüping, A. (2006) · Technische Universität Dortmund Eldorado (Technische Universität Dortmund) · doi:10.17877/de290r-8863 · Cited by 69 · see citing works
Machine Learning: Between Accuracy and Interpretability
Bratko, I. (1997) · Learning, Networks and Statistics · doi:10.1007/978-3-7091-2668-4_10 · Cited by 67 · see citing works
Graphical Explanation in Belief Networks
Madigan, D., Mosurski, K., & Almond, R. G. (1997) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.1997.10474735 · Cited by 62 · see citing works
Self-citationLearning with Decision Lists of Data-Dependent Features
Marchand, M., & Sokolova, M. (2005) · Cited by 47 · see citing works
Bayesian hierarchical rule modeling for predicting medical conditions
McCormick, T. H., Rudin, C. D., & Madigan, D. (2012) · The Annals of Applied Statistics · doi:10.1214/11-aoas522 · Cited by 42 · see citing works
Self-citationWang, T., Rudin, C., Doshi-Velez, F., Liu, Y.-M., Klampfl, E., & MacNeille, P. (2015) · arXiv.org · doi:10.48550/arxiv.1504.07614 · Cited by 41 · see citing works
Self-citationAn Interpretable Stroke Prediction Model using Rules and Bayesian Analysis
Letham, B., Rudin, C. D., McCormick, T. H., & Madigan, D. (2013) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 40 · see citing works
Self-citationBox drawings for learning with imbalanced data
Goh, S. T., & Rudin, C. D. (2014) · ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2623330.2623648 · Cited by 37 · see citing works
Self-citationBeyond predictive accuracy : what?
Giraud-Carrier, C. G. (1998) · Cited by 31 · see citing works
Efficient sequential decision-making algorithms for container inspection operations
Madigan, D., Mittal, S., & Roberts, F. S. (2011) · Naval Research Logistics (NRL) · doi:10.1002/nav.20472 · Cited by 11 · see citing works
Self-citationLearning Theory Analysis for Association Rules and Sequential Event Prediction
Rudin, C. D., Letham, B., & Madigan, D. (2013) · doi:10.7916/d82n50c1 · Cited by 9 · see citing works
Self-citationYi, Y., & Hüllermeier, E. (2005) · Cited by 6 · see citing works
Integer optimization methods for machine learning
Chang, A. (2012) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 5 · see citing works
Flagged first, then by citations
2 of the 75 references couldn’t be matched to a known paper.
Research works printed but not linked: 9
- Morgan Kaufmann, San Francisco, CA.
- Morgan Kaufmann, San Francisco, CA.
- Morgan Kaufmann, San Francisco, CA.
- GAGE, B. F., WATERMAN, A. D., SHANNON, W., BOECHLER, M., RICH, M. W. and RAD- FORD, M. J. (2001). Validation of clinical classification schemes for predicting stroke. Journal of the American Medical Association 285 2864-2870.
- JENNINGS, D. L., AMABILE, T. M. and ROSS, L. (1982). Informal covariation assessments: Data- based versus theory-based judgements. In Judgment Under Uncertainty: Heuristics and Biases, (D. Kahneman, P. Slovic and A. Tversky, eds.) 211-230. Cambridge Univ. Press, Cambridge, MA.
- LETHAM, B., RUDIN, C., MCCORMICK, T. H. and MADIGAN, D. (2014). An interpretable model for stroke prediction using rules and Bayesian analysis. In Proceedings of 2014 KDD Workshop on Data Science for Social Good. MIT, Cambridge, MA.
- LETHAM, B., RUDIN, C., MCCORMICK, T. H. and MADIGAN, D. (2015). Supplement to "Inter- pretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model." DOI:10.1214/15-AOAS848SUPPA, DOI:10.1214/15-AOAS848SUPPB. · DOI
- Morgan Kaufmann, San Mateo.
- RUDIN, C. and ERTEKIN, Ş. (2015). Learning optimized lists of classification rules. Technical re- port, MIT, Cambridge, MA.
Cited by
Sorted by citations
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555 · see citing works
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423 · see citing works
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294 · see citing works
“[66] proposed a model called Bayesian Rule Lists (BRL) based on decision tree, the authors claimed that preliminary interpretable models provide a concise and convincing capabilities to gain domain experts trust.”
BackgroundMethods for interpreting and understanding deep neural networks
Montavon, G., Samek, W., & Müller, K.-R. (2018) · Digital Signal Processing · doi:10.1016/j.dsp.2017.10.011 · Cited by 2,665 · see citing works
A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI
Tjoa, E., & Guan, C. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3027314 · Cited by 2,524 · see citing works
Definitions, methods, and applications in interpretable machine learning
Murdoch, W. J., Singh, C., Kumbier, K., Abbasi-Asl, R., & Yu, B. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1900654116 · Cited by 2,167 · see citing works
Anchors: High-Precision Model-Agnostic Explanations
Ribeiro, M. T., Singh, S., & Guestrin, C. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11491 · Cited by 2,162 · see citing works
“A number of approaches construct globally interpretable models, many based on rules (Lakkaraju, Bach, and Leskovec 2016; Letham et al. 2015; Wang and Rudin 2015; Wang et al. 2015).”
BackgroundInterpreting Black-Box Models: A Review on Explainable Artificial Intelligence
Hassija, V., Chamola, V., Mahapatra, A., Singal, A., Goel, D., Huang, K., Scardapane, S., Spinelli, I., Mahmud, M., & Hussain, A. (2024) · Cognitive Computation · doi:10.1007/s12559-023-10179-8 · Cited by 2,062 · see citing works
“Letham et al. [126] proposed a model called Bayesian Rule Lists (BRL), which is based on decision trees as mentioned in [127].”
MethodInvestigating Ad Transparency Mechanisms in Social Media: A Case Study of Facebook's Explanations
Andreou, A., Venkatadri, G., Goga, O., Gummadi, K. P., Loiseau, P., & Mislove, A. E. (2018) · Network and Distributed System Security Symposium · doi:10.14722/ndss.2018.23191 · Cited by 1,505 · see citing works
DARPA's Explainable Artificial Intelligence Program
Gunning, D., & Aha, D. W. (2019) · AI Magazine · doi:10.1609/aimag.v40i2.2850 · Cited by 1,360 · see citing works
KING, G., PAN, J., & ROBERTS, M. E. (2017) · American Political Science Review · doi:10.1017/s0003055417000144 · Cited by 1,028 · see citing works
A Survey on the Explainability of Supervised Machine Learning
Burkart, N., & Huber, M. F. (2021) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.12228 · Cited by 1,008 · see citing works
Next-Generation Machine Learning for Biological Networks
Camacho, D. M., Collins, K. M., Powers, R. K., Costello, J. C., & Collins, J. J. (2018) · Cell · doi:10.1016/j.cell.2018.05.015 · Cited by 974 · see citing works
Interpretable machine learning: Fundamental principles and 10 grand challenges
Rudin, C., Chen, C., Chen, Z., Huang, H., Semenova, L., & Zhong, C. (2022) · Statistics Surveys · doi:10.1214/21-ss133 · Cited by 958 · see citing works
Self-citationDesigning Theory-Driven User-Centric Explainable AI
Wang, D., Yang, Q., Abdul, A., & Lim, B. Y. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300831 · Cited by 906 · see citing works
Shneiderman, B. (2020) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3419764 · Cited by 768 · see citing works
Accountability in algorithmic decision making
Diakopoulos, N. (2016) · Communications of the ACM · doi:10.1145/2844110 · Cited by 763 · see citing works
Abdul, A., Vermeulen, J., Wang, D., Lim, B. Y., & Kankanhalli, M. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174156 · Cited by 743 · see citing works
Rationalizing Neural Predictions
Lei, T., Barzilay, R., & Jaakkola, T. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1011 · Cited by 777 · see citing works
Yang, G., Ye, Q., & Xia, J. (2022) · Information Fusion · doi:10.1016/j.inffus.2021.07.016 · Cited by 717 · see citing works
Datta, A., Sen, S., & Zick, Y. (2016) · 2016 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2016.42 · Cited by 728 · see citing works
Interpretable Decision Sets: A Joint Framework for Description and Prediction
Lakkaraju, H., Bach, S. H., & Leskovec, J. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939874 · Cited by 687 · see citing works
What do we need to build explainable AI systems for the medical domain?
Holzinger, A., Biemann, C., Pattichis, C. S., & Kell, D. B. (2017) · arXiv · doi:10.48550/arxiv.1712.09923 · Cited by 636 · see citing works
Interpretable Machine Learning – A Brief History, State-of-the-Art and Challenges
Molnar, C., Casalicchio, G., & Bischl, B. (2020) · Communications in Computer and Information Science · doi:10.1007/978-3-030-65965-3_28 · Cited by 677 · see citing works
"Why Should I Trust You?": Explaining the Predictions of Any ClassifierPreprint
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.04938 · Cited by 587 · see citing works
A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems
Mohseni, S., Zarei, N., & Ragan, E. D. (2021) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3387166 · Cited by 785 · see citing works
Explainable AI for Healthcare 5.0: Opportunities and Challenges
Saraswat, D., Bhattacharya, P., Verma, A., Prasad, V. K., Tanwar, S., Sharma, G., Bokoro, P. N., & Sharma, R. (2022) · IEEE Access · doi:10.1109/access.2022.3197671 · Cited by 578 · see citing works
SoK: Security and Privacy in Machine Learning
Papernot, N., McDaniel, P., Sinha, A., & Wellman, M. P. (2018) · 2018 IEEE European Symposium on Security and Privacy (EuroS&P) · doi:10.1109/eurosp.2018.00035 · Cited by 502 · see citing works
Society-in-the-loop: programming the algorithmic social contract
Rahwan, I. (2018) · Ethics and Information Technology · doi:10.1007/s10676-017-9430-8 · Cited by 553 · see citing works
SoK: Towards the Science of security and privacy in machine learning
Papernot, N., McDaniel, P. D., Sinha, A., & Wellman, M. P. (2018) · Singapore Management University Institutional Knowledge (InK) (Singapore Management University) · Cited by 427 · see citing works
Towards better analysis of machine learning models: A visual analytics perspective
Liu, S., Wang, X., Liu, M., & Zhu, J. (2017) · Visual Informatics · doi:10.1016/j.visinf.2017.01.006 · Cited by 397 · see citing works
Analysis of Explainers of Black Box Deep Neural Networks for Computer Vision: A Survey
Buhrmester, V., Münch, D., & Arens, M. (2021) · Machine Learning and Knowledge Extraction · doi:10.3390/make3040048 · Cited by 403 · see citing works
A Review of Challenges and Opportunities in Machine Learning for Health
Ghassemi, M., Naumann, T., Schulam, P., Beam, A. L., Chen, I. Y., & Ranganath, R. (2018) · PubMed Central · doi:10.48550/arxiv.1806.00388 · Cited by 323 · see citing works
Islam, M. R., Ahmed, M. U., Barua, S., & Begum, S. (2022) · Applied Sciences · doi:10.3390/app12031353 · Cited by 339 · see citing works
Benchmarking and survey of explanation methods for black box models
Bodria, F., Giannotti, F., Guidotti, R., Naretto, F., Pedreschi, D., & Rinzivillo, S. (2023) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-023-00933-9 · Cited by 353 · see citing works
Faithful and Customizable Explanations of Black Box Models
Lakkaraju, H., Kamar, E. S., Caruana, R., & Leskovec, J. (2019) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3306618.3314229 · Cited by 261 · see citing works
Moustafa, N., Koroniotis, N., Keshk, M., Zomaya, A. Y., & Tari, Z. (2023) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2023.3280465 · Cited by 266 · see citing works
Rulematrix: Visualizing and understanding classifiers with rules
Ming, Y., Qu, H., & Bertini, E. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2864812 · Cited by 243 · see citing works
de Laat, P. B. (2018) · Philosophy & Technology · doi:10.1007/s13347-017-0293-z · Cited by 251 · see citing works
Four principles of explainable artificial intelligence
Phillips, P. J., Hahn, C. A., Fontana, P. C., Yates, A. N., Greene, K., Broniatowski, D. A., & Przybocki, M. A. (2021) · doi:10.6028/nist.ir.8312 · Cited by 262 · see citing works
Computer‐Assisted Keyword and Document Set Discovery from Unstructured Text
King, G., Lam, P., & Roberts, M. E. (2017) · American Journal of Political Science · doi:10.1111/ajps.12291 · Cited by 216 · see citing works
A survey of visual analytics for Explainable Artificial Intelligence methods
Alicioglu, G., & Sun, B. (2022) · Computers & Graphics · doi:10.1016/j.cag.2021.09.002 · Cited by 221 · see citing works
Explainability in human–agent systems
Rosenfeld, A., & Richardson, A. (2019) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-019-09408-y · Cited by 228 · see citing works
"How do I fool you?": Manipulating User Trust via Misleading Black Box Explanations
Lakkaraju, H., & Bastani, O. (2020) · Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3375627.3375833 · Cited by 241 · see citing works
Human factors in model interpretability: Industry practices, challenges, and needs
Hong, S. R., Hullman, J., & Bertini, E. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3392878 · Cited by 219 · see citing works
The Disagreement Problem in Explainable Machine Learning: A Practitioner's PerspectivePreprint
Krishna, S., Han, T., Gu, A., Jabbari, S., Wu, Z. S., & Lakkaraju, H. (2023) · Research Square · doi:10.21203/rs.3.rs-2963888/v1 · Cited by 279 · see citing works
Modern views of machine learning for precision psychiatry
Chen, Z. S., Kulkarni, P., Galatzer‐Levy, I. R., Bigio, B., Nasca, C., & Zhang, Y. (2022) · Patterns · doi:10.1016/j.patter.2022.100602 · Cited by 225 · see citing works
A Review on Explainable Artificial Intelligence for Healthcare: Why, How, and When?
Bharati, S., Mondal, M. R. H., & Podder, P. (2023) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2023.3266418 · Cited by 235 · see citing works
Nasarian, E., Alizadehsani, R., Acharya, U. R., & Tsui, K.-L. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102412 · Cited by 207 · see citing works
Post-hoc vs ante-hoc explanations: xAI design guidelines for data scientists
Retzlaff, C. O., Angerschmid, A., Saranti, A., Schneeberger, D. M., Röttger, R., Müller, H., & Holzinger, A. (2024) · Cognitive Systems Research · doi:10.1016/j.cogsys.2024.101243 · Cited by 206 · see citing works
A Survey of Methods for Explaining Black Box Models
Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., & Pedreschi, D. (2019) · ACM Computing Surveys · doi:10.1145/3236009 · Cited by 629 · see citing works
Explainable AI for Bioinformatics: Methods, Tools and Applications
Karim, M. R., Islam, T., Shajalal, M., Beyan, O., Lange, C., Cochez, M., Rebholz‐Schuhmann, D., & Decker, S. (2023) · Briefings in Bioinformatics · doi:10.1093/bib/bbad236 · Cited by 208 · see citing works
DARPA 's explainable AI ( XAI ) program: A retrospective
Gunning, D., Vorm, E. S., Wang, Y., & Turek, M. (2021) · Applied AI Letters · doi:10.1002/ail2.61 · Cited by 262 · see citing works
Explainable decision forest: Transforming a decision forest into an interpretable tree
Sagi, O., & Rokach, L. (2020) · Information Fusion · doi:10.1016/j.inffus.2020.03.013 · Cited by 189 · see citing works
The Bayesian Case Model: A Generative Approach for Case-Based Reasoning and Prototype Classification
Kim, B., Rudin, C. D., & Shah, J. (2014) · Cited by 185 · see citing works
Self-citationDing, W., Abdel‐Basset, M., Hawash, H., & Ali, A. M. A. (2022) · Information Sciences · doi:10.1016/j.ins.2022.10.013 · Cited by 196 · see citing works
Towards Reverse-Engineering Black-Box Neural Networks
Oh, S. J., Schiele, B., & Fritz, M. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-28954-6_7 · Cited by 291 · see citing works
Comparing sets of patterns with the Jaccard index
Fletcher, S., & Islam, M. Z. (2018) · AJIS. Australasian journal of information systems/AJIS. Australian journal of information systems/Australian journal of information systems · doi:10.3127/ajis.v22i0.1538 · Cited by 172 · see citing works
Hendricks, L. A., Hu, R., Darrell, T., & Akata, Z. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01216-8_17 · Cited by 171 · see citing works
Wani, N. A., Kumar, R., Mamta, ., Bedi, J., & Rida, I. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102472 · Cited by 171 · see citing works
Souillard‐Mandar, W., Davis, R., Rudin, C. D., Au, R., Libon, D. J., Swenson, R. A., Price, C. C., Lamar, M., & Penney, D. L. (2015) · Machine Learning · doi:10.1007/s10994-015-5529-5 · Cited by 163 · see citing works
Self-citationClassification of Explainable Artificial Intelligence Methods through Their Output Formats
Vilone, G., & Longo, L. (2021) · Machine Learning and Knowledge Extraction · doi:10.3390/make3030032 · Cited by 157 · see citing works
Degas, A., Islam, M. R., Hurter, C., Barua, S., Rahman, H., Poudel, M., Ruscio, D., Ahmed, M. U., Begum, S., Rahman, M. A., et al. (2022) · Applied Sciences · doi:10.3390/app12031295 · Cited by 168 · see citing works
Explaining deep neural networks: A survey on the global interpretation methods
Saleem, R., Yuan, B., Kurugöllü, F., Anjum, A., & Liu, L. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.09.129 · Cited by 150 · see citing works
The role of explainable AI in the context of the AI Act
Panigutti, C., Hamon, R., Hupont, I., Llorca, D. F., Yela, D. F., Junklewitz, H., Scalzo, S., Mazzini, G., Sánchez, I., Garrido, J. S., et al. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594069 · Cited by 171 · see citing works
Coordinating Human and Machine Learning for Effective Organizational Learning
Sturm, T., Gerlach, J. P., Pumplun, L., Mesbah, N., Peters, F., Tauchert, C., Nan, N., & Buxmann, P. (2021) · MIS Quarterly · doi:10.25300/misq/2021/16543 · Cited by 158 · see citing works
Toward Explainable Artificial Intelligence for Precision Pathology
Klauschen, F., Dippel, J., Keyl, P., Jurmeister, P. S., Bockmayr, M. L., Möck, A., Buchstab, O., Alber, M., Ruff, L., Montavon, G., et al. (2023) · Annual Review of Pathology Mechanisms of Disease · doi:10.1146/annurev-pathmechdis-051222-113147 · Cited by 144 · see citing works
Kliegr, T., Bahník, Š., & Fürnkranz, J. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103458 · Cited by 139 · see citing works
Scalable Bayesian Rule ListsPreprint
Yang, H., Rudin, C. D., & Seltzer, M. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.08610 · Cited by 138 · see citing works
Self-citationFour Principles of Explainable Artificial Intelligence
Phillips, P. J., Hahn, C. A., Fontana, P. C., Broniatowski, D. A., & Przybocki, M. A. (2020) · doi:10.6028/nist.ir.8312-draft · Cited by 136 · see citing works
Explainable Artificial Intelligence for Tabular Data: A Survey
Sahakyan, M., Aung, Z., & Rahwan, T. (2021) · IEEE Access · doi:10.1109/access.2021.3116481 · Cited by 135 · see citing works
“[173] use Bayesian Rule Lists (BRL) which discretize a high-dimensional, multivariate feature space into a series of interpretable decision rule lists.”
A survey on XAI and natural language explanations
Cambria, E., Malandri, L., Mercorio, F., Mezzanzanica, M., & Nobani, N. (2023) · Information Processing & Management · doi:10.1016/j.ipm.2022.103111 · Cited by 131 · see citing works
Sheu, R.-K., & Pardeshi, M. S. (2022) · Sensors · doi:10.3390/s22208068 · Cited by 129 · see citing works
Ethical machines: The human-centric use of artificial intelligence
Lepri, B., Oliver, N., & Pentland, A. (2021) · iScience · doi:10.1016/j.isci.2021.102249 · Cited by 149 · see citing works
Local Interpretable Model-Agnostic Explanations For Music Content Analysis.
Mishra, S., Sturm, B. L. T., & Dixon, S. (2017) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.1417386 · Cited by 128 · see citing works
Interpreting Blackbox Models via Model ExtractionPreprint
Bastani, O., Kim, C., & Bastani, H. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.08504 · Cited by 126 · see citing works
Learning Optimal and Fair Decision Trees for Non-Discriminative Decision-Making
Aghaei, S., Azizi, M. J., & Vayanos, P. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33011418 · Cited by 135 · see citing works
An Evaluation of the Human-Interpretability of Explanation
Lage, I., Chen, E., He, J., Narayanan, M., Kim, B., Gershman, S., & Doshi-Velez, F. (2019) · arXiv · doi:10.48550/arxiv.1902.00006 · Cited by 125 · see citing works
Post-hoc explanation of black-box classifiers using confident itemsets
Moradi, M., & Matthias Samwald, A. (2020) · Expert Systems with Applications · doi:10.1016/j.eswa.2020.113941 · Cited by 119 · see citing works
Optimal Sparse Decision TreesPreprint
Hu, X., Rudin, C. D., & Seltzer, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.12847 · Cited by 119 · see citing works
Self-citationValdés, G., Luna, J. M., Eaton, E., Simone, C. B., Ungar, L., & Solberg, T. D. (2016) · Scientific Reports · doi:10.1038/srep37854 · Cited by 118 · see citing works
Decision Tree Classification with Differential Privacy
Fletcher, S., & Islam, M. Z. (2019) · ACM Computing Surveys · doi:10.1145/3337064 · Cited by 137 · see citing works
Optimized Scoring Systems: Toward Trust in Machine Learning for Healthcare and Criminal Justice
Rudin, C. D., & Ustun, B. (2018) · INFORMS Journal on Applied Analytics · doi:10.1287/inte.2018.0957 · Cited by 114 · see citing works
Self-citationSingh, J., & Anand, A. (2019) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3289600.3290620 · Cited by 114 · see citing works
Learning Optimal Decision Trees with SAT
Narodytska, N., Ignatiev, A., Pereira, F., & Marques-Silva, J. P. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/189 · Cited by 114 · see citing works
Improving the Interpretability of Deep Neural Networks with Knowledge Distillation
Liu, X., Wang, X., & Matwin, S. (2018) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw.2018.00132 · Cited by 114 · see citing works
Explainable AI for Medical Data: Current Methods, Limitations, and Future Directions
Hossain, M. I., Zamzmi, G., Mouton, P. R., Salekin, M. S., Sun, Y., & Goldgof, D. B. (2023) · ACM Computing Surveys · doi:10.1145/3637487 · Cited by 120 · see citing works
Event Prediction in the Big Data Era: A Systematic Survey
Zhao, L. (2022) · ACM Computing Surveys · doi:10.1145/3450287 · Cited by 135 · see citing works
MM-Pred: A Deep Predictive Model for Multi-attribute Event Sequence
Li, L., Wen, L., & Wang, J. (2019) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9781611975673.14 · Cited by 109 · see citing works
Learning Certifiably Optimal Rule Lists for Categorical Data
Angelino, E., Larus-Stone, N., Alabi, D., Seltzer, M., & Rudin, C. (2017) · arXiv · doi:10.48550/arxiv.1704.01701 · Cited by 108 · see citing works
Self-citationUltra-Strong Machine Learning: comprehensibility of programs learned with ILP
Muggleton, S., Schmid, U., Zeller, C., Tamaddoni‐Nezhad, A., & Besold, T. R. (2018) · Machine Learning · doi:10.1007/s10994-018-5707-3 · Cited by 108 · see citing works
Ada-WHIPS: explaining AdaBoost classification with applications in the health sciences
Hatwell, J., Gaber, M. M., & Azad, R. M. A. (2020) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-020-01201-2 · Cited by 104 · see citing works
CHIRPS: Explaining random forest classification
Hatwell, J., Gaber, M. M., & Azad, R. M. A. (2020) · Artificial Intelligence Review · doi:10.1007/s10462-020-09833-6 · Cited by 101 · see citing works
Silva, A., Schrum, M., Hedlund-Botti, E., Gopalan, N., & Gombolay, M. (2023) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2022.2101698 · Cited by 104 · see citing works
Anik, A. I., & Bunt, A. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445736 · Cited by 103 · see citing works
Survey on Explainable AI: Techniques, challenges and open issues
Abusitta, A., Li, M. Q., & Fung, B. C. M. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.124710 · Cited by 98 · see citing works
Evolving Rule-Based Explainable Artificial Intelligence for Unmanned Aerial Vehicles
Keneni, B., Kaur, D., Bataineh, A. S. A., Devabhaktuni, V. K., Javaid, A. Y., Zaientz, J. D., & Marinier, R. (2019) · IEEE Access · doi:10.1109/access.2019.2893141 · Cited by 98 · see citing works
Explainable AI for Industry 5.0: Vision, Architecture, and Potential Directions
Trivedi, C., Bhattacharya, P., Prasad, V. K., Patel, V., Singh, A., Tanwar, S., Sharma, R., Aluvala, S., Pau, G., & Sharma, G. (2024) · IEEE Open Journal of Industry Applications · doi:10.1109/ojia.2024.3399057 · Cited by 106 · see citing works
What can AI do for me?: evaluating machine learning interpretations in cooperative play
Feng, S., & Boyd-Graber, J. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302265 · Cited by 92 · see citing works
Simple Rules for Complex DecisionsPreprint
Jung, J., Concannon, C., Shroff, R., Goel, S., & Goldstein, D. G. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.2919024 · Cited by 91 · see citing works
- Authors
- Letham, B., Rudin, C., McCormick, T. H., & Madigan, D.
- Affiliations
- Massachusetts Institute of Technology (US)
- University of Washington (US)
- Venue
- The Annals of Applied Statistics
- Publisher
- Institute of Mathematical Statistics
- Published
- 2015-09-01
- Volume
- 9
- Issue
- 3
- Pages
- 1350-1371
- ISSN
- 1932-6157 (ISSN-L)
- 1941-7330
- OpenAlex
- W2118022153
- Cited by
- 766
- References
- 75
- Citations by year
- 2026: 36 · 2025: 51 · 2024: 77 · 2023: 91 · 2022: 90 · 2021: 108 · 2020: 76 · 2019: 88 · 2018: 65 · 2017: 42 · 2016: 26 · 2015: 14 · 2014: 2
- FWCI
- 47.96
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- The Annals of Applied Statistics (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Project Euclid (Cornell University) (submittedVersion)
- DSpace@MIT (Massachusetts Institute of Technology) (submittedVersion, cc-by-nc)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1053407, CAREER)
- National Institutes of Health
- National Institute of Allergy and Infectious Diseases
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- interpretable classifiers, decision lists, generative models, sparsity prior, stroke risk prediction, atrial fibrillation, CHADS2 score, personalized medicine, rule-based prediction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
