Interpretable Machine Learning – A Brief History, State-of-the-Art and Challenges

Random Forests

Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538 · see citing works

Regression shrinkage and selection via the lasso

Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482 · see citing works

Estimating the Dimension of a Model

Schwarz, G. (1978) · The Annals of Statistics · doi:10.1214/aos/1176344136 · Cited by 47,033 · see citing works

Greedy function approximation: A gradient boosting machine

Friedman, J. H. (2001) · The Annals of Statistics · doi:10.1214/aos/1013203451 · Cited by 30,731 · see citing works

A Unified Approach to Interpreting Model Predictions

Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707 · see citing works

Regularization and Variable Selection Via the Elastic Net

Zou, H., & Hastie, T. (2005) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2005.00503.x · Cited by 20,957 · see citing works

Deep learning in neural networks: an overview

Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570 · see citing works

Information Theory and an Extension of the Maximum Likelihood Principle

Akaike, H. (1998) · Springer series in statistics · doi:10.1007/978-1-4612-1694-0_15 · Cited by 18,384 · see citing works

"Why Should I Trust You?": Explaining the Predictions of Any Classifier

Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555 · see citing works

Visualizing and understanding convolutional networks

Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839 · see citing works

Data Analysis Using Regression and Multilevel/Hierarchical Models

Gelman, A., & Hill, J. (2006) · doi:10.1017/cbo9780511790942 · Cited by 13,597 · see citing works

On a Method to Measure Supervised Multiclass Model’s Interpretability: Application to Degradation Diagnosis (Short Paper)

Gauriat, C.-M., Pencolé, Y., Ribot, P., et al. (2024) · HAL · doi:10.4230/oasics.dx.2024.27 · Cited by 12,623 · see citing works

A Value for n-person Games

Shapley, L. (1988) · Cambridge University Press eBooks · doi:10.1017/cbo9780511528446.003 · Cited by 10,698 · see citing works

Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)

Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294 · see citing works

The mythos of model interpretability

Lipton, Z. C. (2018) · Communications of the ACM · doi:10.1145/3233231 · Cited by 5,427 · see citing works

Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps

Simonyan, K., Vedaldi, A., & Zisserman, A. (2013) · arXiv · doi:10.48550/arxiv.1312.6034 · Cited by 5,370 · see citing works

Explanation in artificial intelligence: Insights from the social sciences

Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302 · see citing works

Random forests for classification in ecology

Cutler, D. R., Edwards, T. C., Beard, K. H., Cutler, A., Hess, K. T., Gibson, J., & Lawler, J. J. (2007) · Ecology · doi:10.1890/07-0539.1 · Cited by 4,950 · see citing works

The Strength of Weak Learnability

Schapire, R. E. (1990) · Machine Learning · doi:10.1007/bf00116037 · Cited by 4,600 · see citing works

Axiomatic attribution for deep networks

Sundararajan, M., Taly, A., & Yan, Q. (2017) · arXiv · doi:10.48550/arxiv.1703.01365 · Cited by 4,087 · see citing works

Bias in random forest variable importance measures: Illustrations, sources and a solution

Strobl, C., Boulesteix, A., Zeileis, A., & Hothorn, T. (2007) · BMC Bioinformatics · doi:10.1186/1471-2105-8-25 · Cited by 3,755 · see citing works

Conditional variable importance for random forests

Strobl, C., Boulesteix, A., Kneib, T., Augustin, T., & Zeileis, A. (2008) · BMC Bioinformatics · doi:10.1186/1471-2105-9-307 · Cited by 3,299 · see citing works

Towards A Rigorous Science of Interpretable Machine Learning

Doshi-Velez, F., & Kim, B. (2017) · arXiv · doi:10.48550/arxiv.1702.08608 · Cited by 3,121 · see citing works

Permutation importance: a corrected feature importance measure

Altmann, A., Toloşi, L., Sander, O., & Lengauer, T. (2010) · Bioinformatics · doi:10.1093/bioinformatics/btq134 · Cited by 2,903 · see citing works

Explaining prediction models and individual predictions with feature contributions

Štrumbelj, E., & Kononenko, I. (2014) · Knowledge and Information Systems · doi:10.1007/s10115-013-0679-x · Cited by 2,286 · see citing works

Definitions, methods, and applications in interpretable machine learning

Murdoch, W. J., Singh, C., Kumbier, K., Abbasi-Asl, R., & Yu, B. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1900654116 · Cited by 2,167 · see citing works

Peeking Inside the Black Box: Visualizing Statistical Learning With Plots of Individual Conditional Expectation

Goldstein, A., Kapelner, A., Bleich, J., & Pitkin, E. (2015) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.2014.907095 · Cited by 1,861 · see citing works

Machine Learning Interpretability: A Survey on Methods and Metrics

Carvalho, D. V., Pereira, E. M., & Cardoso, J. S. (2019) · Electronics · doi:10.3390/electronics8080832 · Cited by 1,842 · see citing works

Intelligible Models for HealthCare: Predicting Pneumonia Risk and Hospital 30-day Readmission

Caruana, R., Lou, Y., Gehrke, J., Koch, P., Sturm, M., & Elhadad, N. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2788613 · Cited by 1,685 · see citing works

Understanding Black-box Predictions via Influence Functions

Koh, P. W., & Liang, P. (2017) · arXiv · doi:10.48550/arxiv.1703.04730 · Cited by 1,514 · see citing works

Consistent Individualized Feature Attribution for Tree Ensembles

Lundberg, S. M., Erion, G. G., & Lee, S.-I. (2018) · arXiv · doi:10.48550/arxiv.1802.03888 · Cited by 1,430 · see citing works

Multivariate Statistical Modelling Based on Generalized Linear Models

Fahrmeir, L., & Tutz, G. (1994) · Springer series in statistics · doi:10.1007/978-1-4899-0010-4 · Cited by 1,413 · see citing works

The Extent and Consequences of P-Hacking in Science

Head, M. L., Holman, L., Lanfear, R., Kahn, A. T., & Jennions, M. D. (2015) · PLOS Biology · doi:10.1371/journal.pbio.1002106 · Cited by 1,388 · see citing works

Explaining nonlinear classification decisions with deep taylor decomposition

Montavon, G., Lapuschkin, S., Binder, A., Samek, W., & Müller, K.-R. (2017) · Pattern Recognition · doi:10.1016/j.patcog.2016.11.008 · Cited by 1,357 · see citing works

Predictive learning via rule ensembles

Friedman, J. H., & Popescu, B. E. (2008) · The Annals of Applied Statistics · doi:10.1214/07-aoas148 · Cited by 1,315 · see citing works

Sanity Checks for Saliency Maps

Adebayo, J., Gilmer, J., Muelly, M., Goodfellow, I., Hardt, M., & Kim, B. (2018) · arXiv · doi:10.48550/arxiv.1810.03292 · Cited by 1,294 · see citing works

Techniques for interpretable machine learning

Du, M., Liu, N., & Hu, X. (2019) · Communications of the ACM · doi:10.1145/3359786 · Cited by 1,268 · see citing works

Explaining machine learning classifiers through diverse counterfactual explanations

Mothilal, R. K., Sharma, A., & Tan, C. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372850 · Cited by 1,233 · see citing works

Survey and critique of techniques for extracting rules from trained artificial neural networks

Andrews, R., Diederich, J., & Tickle, A. B. (1995) · Knowledge-Based Systems · doi:10.1016/0950-7051(96)81920-4 · Cited by 1,225 · see citing works

All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously

Fisher, A., Rudin, C., & Dominici, F. (2018) · arXiv · doi:10.48550/arxiv.1801.01489 · Cited by 1,165 · see citing works

The History of Statistics: The Measurement of Uncertainty before 1900

Stigler, S. M. (1986) · Medical Entomology and Zoology · Cited by 1,146 · see citing works

Distribution-Free Predictive Inference for Regression

Lei, J., G’Sell, M., Rinaldo, A., Tibshirani, R. J., & Wasserman, L. (2018) · Journal of the American Statistical Association · doi:10.1080/01621459.2017.1307116 · Cited by 1,049 · see citing works

Feature visualization

Olah, C., Mordvintsev, A., & Schubert, L. (2017) · Distill · doi:10.23915/distill.00007 · Cited by 880 · see citing works

Interpretable Convolutional Neural Networks

Zhang, Q., Wu, Y. N., & Zhu, S.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00920 · Cited by 833 · see citing works

A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems

Mohseni, S., Zarei, N., & Ragan, E. D. (2021) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3387166 · Cited by 785 · see citing works

Model-Based Recursive Partitioning

Zeileis, A., Hothorn, T., & Hornik, K. (2008) · Journal of Computational and Graphical Statistics · doi:10.1198/106186008x319331 · Cited by 771 · see citing works

Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model

Letham, B., Rudin, C., McCormick, T. H., & Madigan, D. (2015) · The Annals of Applied Statistics · doi:10.1214/15-aoas848 · Cited by 766 · see citing works

Linear Inversion of Band-Limited Reflection Seismograms

Santosa, F., & Symes, W. W. (1986) · SIAM Journal on Scientific and Statistical Computing · doi:10.1137/0907087 · Cited by 737 · see citing works

Manipulating and Measuring Model Interpretability

Poursabzi-Sangdeh, F., Goldstein, D. G., Hofman, J. M., Wortman Vaughan, J. W., & Wallach, H. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445315 · Cited by 728 · see citing works

iml: An R package for Interpretable Machine Learning

Molnar, C. (2018) · The Journal of Open Source Software · doi:10.21105/joss.00786 · Cited by 720 · see citing works

Not Just a Black Box: Learning Important Features Through Propagating Activation DifferencesPreprint

Shrikumar, A., Greenside, P., Shcherbina, A., & Kundaje, A. B. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.01713 · Cited by 704 · see citing works

Towards Explainable Artificial Intelligence

Samek, W., & Müller, K. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-28954-6_1 · Cited by 650 · see citing works

A Survey of Methods for Explaining Black Box Models

Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., & Pedreschi, D. (2019) · ACM Computing Surveys · doi:10.1145/3236009 · Cited by 629 · see citing works

Extracting Tree-Structured Representations of Trained Networks

Craven, M. W., & SHAVLIK, J. W. (1995) · Neural Information Processing Systems · Cited by 583 · see citing works

Comprehensible classification models

Freitas, A. A. (2014) · ACM SIGKDD Explorations Newsletter · doi:10.1145/2594473.2594475 · Cited by 575 · see citing works

Variable importance analysis: A comprehensive review

Wei, P., Lü, Z., & Song, J. (2015) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2015.05.018 · Cited by 526 · see citing works

Metrics for explainable AI: challenges and prospects

Hoffman, R. R., Mueller, S. T., Klein, G., & Litman, J. (2018) · arXiv · doi:10.48550/arxiv.1812.04608 · Cited by 518 · see citing works

The What-If Tool: Interactive Probing of Machine Learning Models

Wexler, J., Pushkarna, M., Bolukbasi, T., WATTENBERG, M. P., Viégas, F. B., & Wilson, J. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2019.2934619 · Cited by 512 · see citing works

Actionable Recourse in Linear Classification

Ustun, B., Spangher, A., & Liu, Y. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287566 · Cited by 501 · see citing works

Causality for Machine Learning

Schölkopf, B. (2022) · ACM eBooks · doi:10.1145/3501714.3501755 · Cited by 442 · see citing works

High-Dimensional Variable Selection for Survival Data

Ishwaran, H., Kogalur, U. B., Gorodeski, E. Z., Minn, A. J., & Lauer, M. S. (2010) · Journal of the American Statistical Association · doi:10.1198/jasa.2009.tm08622 · Cited by 439 · see citing works

Foundations of Rule Learning

Fürnkranz, J., Gamberger, D., & Lavrač, N. (2012) · Cognitive Technologies · doi:10.1007/978-3-540-75197-7 · Cited by 428 · see citing works

An empirical evaluation of the comprehensibility of decision table, tree and rule based predictive models

Huysmans, J., Dejaeger, K., Mues, C., Vanthienen, J., & Baesens, B. (2011) · Decision Support Systems · doi:10.1016/j.dss.2010.12.003 · Cited by 425 · see citing works

The many Shapley values for model explanationPreprint

Sundararajan, M., & Najmi, A. (2019) · RePEc: Research Papers in Economics · doi:10.48550/arxiv.1908.08474 · Cited by 415 · see citing works

Distilling a neural network into a soft decision tree

Frosst, N., & Hinton, G. (2017) · arXiv · doi:10.48550/arxiv.1711.09784 · Cited by 407 · see citing works

Explainable Agents and Robots: Results from a Systematic Literature Review

Anjomshoae, S., Najjar, A., Calvaresi, D., & Främling, K. (2019) · Adaptive Agents and Multi-Agent Systems · doi:10.65109/kczb5817 · Cited by 402 · see citing works

Variable importance in binary regression trees and forests

Ishwaran, H. (2012) · Cited by 379 · see citing works

THEORIA MOTUS CORPORUM COELESTIUM IN SECTIONIBUS CONICIS SOLEM AMBIENTIUM

Gauss, C. F. (2011) · Cambridge University Press eBooks · doi:10.1017/cbo9781139058285.001 · Cited by 368 · see citing works

Variable importance in binary regression trees and forests

Ishwaran, H. (2007) · Electronic Journal of Statistics · doi:10.1214/07-ejs039 · Cited by 367 · see citing works

DALEX: Explainers for Complex Predictive Models in R

Biecek, P. (2020) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.3670939 · Cited by 344 · see citing works

InterpretML: A Unified Framework for Machine Learning Interpretability

Nori, H., Jenkins, S., Koch, P., & Caruana, R. (2019) · arXiv · doi:10.48550/arxiv.1909.09223 · Cited by 322 · see citing works

Supersparse linear integer models for optimized medical scoring systems

Ustun, B., & Rudin, C. D. (2015) · Machine Learning · doi:10.1007/s10994-015-5528-6 · Cited by 316 · see citing works

Multi-Objective Counterfactual Explanations

Dandl, S., Molnar, C., Binder, M., & Bischl, B. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58112-1_31 · Cited by 271 · see citing works

Explainable Artificial Intelligence: a Systematic Review

Vilone, G., & Longo, L. (2020) · arXiv · doi:10.48550/arxiv.2006.00093 · Cited by 269 · see citing works

Nouvelles méthodes pour la détermination des orbites des comètes

Legendre, A. M. (1970) · doi:10.3931/e-rara-35115 · Cited by 262 · see citing works

Visualizing the Effects of Predictor Variables in Black Box Supervised Learning ModelsPreprint

Apley, D. W., & Zhu, J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.08468 · Cited by 247 · see citing works

Rulematrix: Visualizing and understanding classifiers with rules

Ming, Y., Qu, H., & Bertini, E. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2864812 · Cited by 243 · see citing works

Visualizing the Feature Importance for Black Box Models

Casalicchio, G., Molnar, C., & Bischl, B. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-10925-7_40 · Cited by 236 · see citing works

Explainability in human–agent systems

Rosenfeld, A., & Richardson, A. (2019) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-019-09408-y · Cited by 228 · see citing works

Problems with Shapley-value-based explanations as feature importance measures

Kumar, I. E., Venkatasubramanian, S., Scheidegger, C., & Friedler, S. (2020) · arXiv · doi:10.48550/arxiv.2002.11097 · Cited by 228 · cited 1× · see citing works

Generalized Functional ANOVA Diagnostics for High-Dimensional Functions of Dependent Variables

Hooker, G. (2007) · Journal of Computational and Graphical Statistics · doi:10.1198/106186007x237892 · Cited by 218 · see citing works

Feature relevance quantification in explainable AI: A causal problem

Janzing, D., Minorics, L., & Blöbaum, P. (2019) · arXiv · doi:10.48550/arxiv.1910.13413 · Cited by 207 · see citing works

A new variable importance measure for random forests with missing data

Hapfelmeier, A., Hothorn, T., Ulm, K., & Strobl, C. (2012) · Statistics and Computing · doi:10.1007/s11222-012-9349-1 · Cited by 202 · see citing works

Fast Calibrated Additive Quantile Regression

Fasiolo, M., Wood, S. N., Zaffran, M., Nedellec, R., & Goude, Y. (2020) · Journal of the American Statistical Association · doi:10.1080/01621459.2020.1725521 · Cited by 195 · see citing works

The Dangers of Post-hoc Interpretability: Unjustified Counterfactual Explanations

Laugel, T., Lesot, M.-J., Marsala, C., Renard, X., & Detyniecki, M. (2019) · Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/388 · Cited by 194 · see citing works

Interpretable Predictions of Tree-based Ensembles via Actionable Feature Tweaking

Tolomei, G., Silvestri, F., Haines, A., & Lalmas, M. (2017) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3097983.3098039 · Cited by 192 · see citing works

A Simple and Effective Model-Based Variable Importance MeasurePreprint

Greenwell, B., Boehmke, B. C., & McCarthy, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.04755 · Cited by 186 · see citing works

Scalable Visualization Methods for Modern Generalized Additive Models

Fasiolo, M., Nedellec, R., Goude, Y., & Wood, S. N. (2019) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.2019.1629942 · Cited by 171 · see citing works

Untitled work

(2015) · International Conference on Artificial Intelligence and Statistics · Cited by 162 · see citing works

Explainable AI in Industry

Gade, K., Geyik, S. C., Kenthapadi, K., Mithal, V., & Taly, A. (2019) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3292500.3332281 · Cited by 156 · see citing works

Monotonic Networks

Sill, J. (1997) · neural information processing systems · Cited by 139 · see citing works

vip: Variable Importance Plots

Greenwell, B., & Boehmke, B. (2018) · doi:10.32614/cran.package.vip · Cited by 134 · see citing works

Interpreting Blackbox Models via Model ExtractionPreprint

Bastani, O., Kim, C., & Bastani, H. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.08504 · Cited by 126 · see citing works

Model-agnostic feature importance and effects with dependent features: a conditional subgroup approach

Molnar, C., König, G., Bischl, B., & Casalicchio, G. (2023) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-022-00901-9 · Cited by 125 · see citing works

ALIME: Autoencoder Based Approach for Local Interpretability

Shankaranarayana, S. M., & Runje, D. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-33607-3_49 · Cited by 118 · see citing works

Stakeholders in Explainable AI

Preece, A., Harborne, D., Braines, D., Tomsett, R., & Chakraborty, S. (2018) · arXiv · doi:10.48550/arxiv.1810.00184 · Cited by 117 · see citing works

OptiLIME: Optimized LIME Explanations for Diagnostic Computer AlgorithmsPreprint

Visani, G., Bagli, E., & Chesani, F. (2020) · International Conference on Information and Knowledge Management · doi:10.48550/arxiv.2006.05714 · Cited by 89 · see citing works

Please Stop Permuting Features: An Explanation and Alternatives.Preprint

Hooker, G., & Mentch, L. (2019) · arXiv (Cornell University) · Cited by 86 · see citing works

PALM: Machine Learning Explanations For Iterative Debugging

Krishnan, S., & Wu, E. (2017) · Proceedings of the 2nd Workshop on Human-In-the-Loop Data Analytics · doi:10.1145/3077257.3077271 · Cited by 81 · see citing works

Quantifying Model Complexity via Functional Decomposition for Better Post-hoc Interpretability

Molnar, C., Casalicchio, G., & Bischl, B. (2020) · Communications in computer and information science · doi:10.1007/978-3-030-43823-4_17 · Cited by 70 · see citing works

Enriching Visual with Verbal Explanations for Relational Concepts – Combining LIME with Aleph

Rabold, J., Deininger, H., Siebers, M., & Schmid, U. (2020) · Communications in computer and information science · doi:10.1007/978-3-030-43823-4_16 · Cited by 59 · see citing works

Measuring the stability of results from supervised statistical learning

Philipp, M., Rusch, T., Hornik, K., & Strobl, C. (2018) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.2018.1473779 · Cited by 56 · see citing works

AI Explainability 360: An Extensible Toolkit for Understanding Data and Machine Learning Models

Arya, V., Bellamy, R., Chen, P., Dhurandhar, A., Hind, M., Hoffman, S. C., Houde, S., Liao, Q. V., Luss, R., Mojsilović, A., et al. (2020) · Journal of Machine Learning Research · Cited by 54 · see citing works

Assessing the Local Interpretability of Machine Learning ModelsPreprint

Slack, D., Friedler, S. A., Scheidegger, C., & Roy, C. D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.03501 · Cited by 50 · see citing works

A Human-Grounded Evaluation Benchmark for Local Explanations of Machine LearningPreprint

Mohseni, S., Block, J. E., & Ragan, E. D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.05075 · Cited by 46 · see citing works

Rule extraction from neural networks — A comparative study

Augasta, M. G., & Kathirvalavakumar, T. (2012) · doi:10.1109/icprime.2012.6208380 · Cited by 41 · see citing works

MAGIX: Model Agnostic Globally Interpretable ExplanationsPreprint

Puri, N., Gupta, P., Agarwal, P., Verma, S., & Krishnamurthy, B. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.07160 · Cited by 39 · see citing works

Locally Interpretable Models and Effects based on Supervised Partitioning (LIME-SUP)Preprint

Hu, L., Chen, J., Nair, V. N., & Sudjianto, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.00663 · Cited by 38 · see citing works

Unrestricted Permutation forces Extrapolation: Variable Importance Requires at least One More Model, or There Is No Free Variable ImportancePreprint

Hooker, G., Mentch, L., & Zhou, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.03151 · Cited by 37 · see citing works

Explaining Black-Box Classifiers with ILP – Empowering LIME with Aleph to Approximate Non-linear Decisions with Relational Rules

Rabold, J., Siebers, M., & Schmid, U. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-99960-9_7 · Cited by 36 · see citing works

Efficient nonparametric statistical inference on population feature importance using Shapley values

Williamson, B. D., & Feng, J. (2020) · PubMed · doi:10.48550/arxiv.2006.09481 · Cited by 34 · see citing works

Pitfalls to Avoid when Interpreting Machine Learning ModelsPreprint

Molnar, C., König, G., Herbinger, J., Freiesleben, T., Dandl, S., Scholbeck, C. A., Casalicchio, G., Grosse‐Wentrup, M., & Bischl, B. (2020) · arXiv (Cornell University) · Cited by 33 · see citing works

Pattern Recognition. ICPR International Workshops and Challenges

König, G., Molnar, C., Bischl, B., & Grosse‐Wentrup, M. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-68787-8 · Cited by 30 · see citing works

Recherches sur la population, les naissances, les décès, les prisons, les dépôts de mendicité, etc., dans le royaume des Pays-Bas

Quetelet, A. (1827) · Nouveaux mémoires de l Académie royale des sciences et belles-lettres de Bruxelles · doi:10.3406/marb.1827.3345 · Cited by 29 · see citing works

Computing AIC for black-box models using generalized degrees of freedom: A comparison with cross-validation

Hauenstein, S., Wood, S. N., & Dormann, C. F. (2017) · Communications in Statistics - Simulation and Computation · doi:10.1080/03610918.2017.1315728 · Cited by 21 · see citing works

TIP: Typifying the Interpretability of ProceduresPreprint

Dhurandhar, A., Iyengar, V. S., Luss, R., & Shanmugam, K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.02952 · Cited by 19 · see citing works

Measuring Interpretability for Different Types of Machine Learning Models

Zhou, Q., Liao, F., Mou, C., & Wang, P. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-04503-6_29 · Cited by 18 · see citing works

A study of data and label shift in the LIME frameworkPreprint

Rahnama, A. H. A., & Boström, H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.14421 · Cited by 14 · see citing works

Testing conditional independence in supervised learning algorithms

Watson, D. S., & Wright, M. N. (2021) · Machine Learning · doi:10.1007/s10994-021-06030-6 · Cited by 13 · see citing works

General Pitfalls of Model-Agnostic Interpretation Methods for Machine Learning ModelsPreprint

Molnar, C., König, G., Herbinger, J., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.04131 · Cited by 11 · see citing works

Counterfactual Explanations & Adversarial Examples - Common Grounds, Essential Differences, and Potential Transfers.Preprint

Freiesleben, T. (2020) · arXiv (Cornell University) · Cited by 10 · see citing works

audioLIME: Listenable Explanations Using Source SeparationPreprint

Haunschmid, V., Manilow, E., & Widmer, G. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.00582 · Cited by 7 · see citing works

MeLIME: Meaningful Local Explanation for Machine Learning ModelsPreprint

Botari, T., Hvilshøj, F., Izbicki, R., & de Carvalho, A. C. P. L. F. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.05818 · Cited by 7 · see citing works

On Feature Relevance Uncertainty: A Monte Carlo Dropout Sampling ApproachPreprint

Fabi, K., & Schneider, J. (2020) · arXiv (Cornell University) · doi:10.5281/zenodo.3970395 · Cited by 5 · see citing works

Current Challenges and Future Opportunities for XAI in Machine Learning-Based Clinical Decision Support Systems: A Systematic Review

Antoniadi, A. M., Du, Y., Guendouz, Y., Wei, L., Mazo, C., Becker, B. A., & Mooney, C. (2021) · Applied Sciences · doi:10.3390/app11115088 · Cited by 629 · see citing works

Evaluating the quality of machine learning explanations: a survey on methods and metrics

Zhou, J., Gandomi, A. H., Chen, F., & Holzinger, A. (2021) · Electronics · doi:10.3390/electronics10050593 · Cited by 606 · see citing works

A review of Earth Artificial Intelligence

Sun, Z., Sandoval, L., Crystal-Ornelas, R., Mousavi, S. M., Wang, J., Lin, C., Cristea, N., Tong, D., Carande, W. H., Ma, X., et al. (2022) · Computers & Geosciences · doi:10.1016/j.cageo.2022.105034 · Cited by 271 · see citing works

Explainable intrusion detection for cyber defences in the internet of things: opportunities and solutions

Moustafa, N., Koroniotis, N., Keshk, M., Zomaya, A. Y., & Tari, Z. (2023) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2023.3280465 · Cited by 266 · see citing works

A survey on aspect-based sentiment classification

Brauwers, G., & Frasincar, F. (2023) · ACM Computing Surveys · doi:10.1145/3503044 · Cited by 165 · see citing works

Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks

Räuker, T., Ho, A., Casper, S., & Hadfield-Menell, D. (2023) · 2023 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml54575.2023.00039 · Cited by 122 · see citing works

A machine learning model that outperforms conventional global subseasonal forecast models

Chen, L., Zhong, X., Li, H., Wu, J., Lu, B., Chen, D., Xie, S.-P., Wu, L., Chao, Q., Lin, C., et al. (2024) · Nature Communications · doi:10.1038/s41467-024-50714-1 · Cited by 105 · see citing works

Explainability and causability in digital pathology

Plass, M., Kargl, M., Kiehl, T., Regitnig, P., Geißler, C., Evans, T., Zerbe, N., Carvalho, R., Holzinger, A., & Müller, H. (2023) · The Journal of Pathology Clinical Research · doi:10.1002/cjp2.322 · Cited by 99 · see citing works

Directly Discriminatory Algorithms

Adams‐Prassl, J., Binns, R., & Kelly‐Lyth, A. (2023) · The Modern Law Review · doi:10.1111/1468-2230.12759 · Cited by 67 · see citing works

It is Not “Accuracy vs. Explainability”—We Need Both for Trustworthy AI Systems

Petkovic, D. (2023) · IEEE Transactions on Technology and Society · doi:10.1109/tts.2023.3239921 · Cited by 63 · see citing works

Interpretable generalized additive neural networks

Kraus, M., Tschernutter, D., Weinzierl, S., & Zschech, P. (2024) · European Journal of Operational Research · doi:10.1016/j.ejor.2023.06.032 · Cited by 59 · see citing works

Using explainable machine learning forecasts to discover subseasonal drivers of high summer temperatures in Western and Central Europe

van Straaten, C., Whan, K., Coumou, D., van den Hurk, B., & Schmeits, M. (2022) · Monthly Weather Review · doi:10.1175/mwr-d-21-0201.1 · Cited by 50 · see citing works

FuXi-Extreme: Improving extreme rainfall and wind forecasts with diffusion model

Zhong, X., Chen, L., Liu, J., Lin, C., Qi, Y., & Li, H. (2024) · Science China Earth Sciences · doi:10.1007/s11430-023-1427-x · Cited by 37 · see citing works

Artificial Intelligence in Cardiovascular Imaging: “Unexplainable” Legal and Ethical Challenges?

Lang, M., Bernier, A., & Knoppers, B. M. (2022) · Canadian Journal of Cardiology · doi:10.1016/j.cjca.2021.10.009 · Cited by 37 · see citing works

Interpretable Bias Mitigation for Textual Data: Reducing Genderization in Patient Notes While Maintaining Classification Performance

Minot, J. R., Cheney, N., Maier, M., Elbers, D. C., Danforth, C. M., & Dodds, P. S. (2022) · ACM Transactions on Computing for Healthcare · doi:10.1145/3524887 · Cited by 23 · see citing works

Toward a Theory of Causation for Interpreting Neural Code Models

Nader Palacio, D., Velasco, A., Cooper, N., Rodriguez, A., Moran, K., & Poshyvanyk, D. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3379943 · Cited by 18 · see citing works

Current best practices and future opportunities for reproducible findings using large-scale neuroimaging in psychiatry

Jahanshad, N., Lenzini, P., & Bijsterbosch, J. (2025) · Neuropsychopharmacology · doi:10.1038/s41386-024-01938-8 · Cited by 12 · see citing works

We are the Sensors of Consciousness! A Review and Analysis on How Awakenings During Sleep Influence Dream Recall

Stucky, B. (2025) · Nature and Science of Sleep · doi:10.2147/nss.s506461 · Cited by 7 · see citing works

Active reinforcement learning versus action bias and hysteresis: control with a mixture of experts and nonexperts

Colas, J. T., O’Doherty, J. P., & Grafton, S. T. (2024) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1011950 · Cited by 6 · see citing works

Interpretable machine learning: Fundamental principles and 10 grand challenges

Rudin, C., Chen, C., Chen, Z., Huang, H., Semenova, L., & Zhong, C. (2022) · Statistics Surveys · doi:10.1214/21-ss133 · Cited by 958 · see citing works

MonoNet: Towards Interpretable Models by Learning Monotonic FeaturesPreprint

Nguyen, A.-P., & Martínez, M. R. (2019) · arXiv.org · doi:10.48550/arxiv.1909.13611 · Cited by 17 · see citing works

Interpretability of deep learning models: A survey of results

Chakraborty, S., Tomsett, R., Raghavendra, R., Harborne, D., Alzantot, M., Cerutti, F., Srivastava, M. B., Preece, A. D., Julier, S., Rao, R., et al. (2017) · IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computing, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI) · doi:10.1109/uic-atc.2017.8397411 · Cited by 469 · see citing works

Agnostic Local Explanation for Time Series Classification

Guillemé, M., Masson, V., Rozé, L., & Termier, A. (2019) · IEEE International Conference on Tools with Artificial Intelligence (ICTAI) · doi:10.1109/ictai.2019.00067 · Cited by 48 · see citing works

Interpretable and Differentially Private Predictions

Harder, F., Bauer, M., & Park, M. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.5827 · Cited by 6 · see citing works