General Pitfalls of Model-Agnostic Interpretation Methods for Machine Learning Models
Open accessThe DOI matches its Crossref record; 134 of the 134 references it cites were found in a registry or an index.
Abstract
An increasing number of model-agnostic interpretation techniques for machine learning (ML) models such as partial dependence plots (PDP), permutation feature importance (PFI) and Shapley values provide insightful model interpretations, but can lead to wrong conclusions if applied incorrectly. We highlight many general pitfalls of ML model interpretation, such as using interpretation techniques in the wrong context, interpreting models that do not generalize well, ignoring feature dependencies, interactions, uncertainty estimates and issues in high-dimensional settings, or making unjustified causal interpretations, and illustrate them with examples. We focus on pitfalls for global methods that describe the average model behavior, but many pitfalls also apply to local methods that explain individual predictions. Our paper addresses ML practitioners by raising awareness of pitfalls and identifying solutions for correct model interpretation, but also addresses ML researchers by discussing open issues for further research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
![Fig. 2 Assuming one-fits-all interpretability . A default xgboost regression model that minimizes the mean squared error (MSE) was fitted on 20 independently and uniformly distributed features to predict another independent, uniformly sampled target. In this setting, predicting the (unconditional) mean E [ Y ] in a constant model is optimal. The learner overfits due to a small training data size. Mean marginal SHAP (red, error bars indicate 0 . 05 and 0 . 95 quantiles) exposes all mechanistically used features. In contrast, PFI on test data (blue, error bars indicate 0 . 05 and 0 . 95 quantiles) considers all features to be irrelevant, since no feature contributes to the generalization performance.](https://figures.cochat.ai/paper-figures/d89fa50113fe4204a130eac0040da274/general-pitfalls-of-model-agnostic-interpretation-methods-for-machine-learning-figure-2.png)


![Fig. 5 Confusing linear correlation with dependence . Highly dependent features X 1 and X 2 that have a correlation close to zero. A test ( H 0 : Features are independent) using Pearson correlation is not significant, but for HSIC, the H 0 -hypothesis gets rejected. Data from [80].](https://figures.cochat.ai/paper-figures/90068dfc976d4ead9a6c7cbb19095454/general-pitfalls-of-model-agnostic-interpretation-methods-for-machine-learning-figure-5.png)

![Fig. 7 Misleading effect due to interactions . Simulation example with interactions: Y = 3 X 1 -6 X 2 + 12 X 2 1 ( X 3 ≥ 0) + ϵ with X 1 , X 2 , X 3 i.i.d. ∼ U [ -1 , 1] and ϵ i.i.d. ∼ N (0 , 0 . 3). A random forest with 500 trees is fitted on 1000 observations. Effects are calculated on 200 randomly sampled (training) observations. A, B: PDP (yellow) and ICE curves of X 1 and X 2 ; C: Derivative ICE curves and their standard deviation of X 2 ; D: 2-dimensional PDP of X 2 and X 3 .](https://figures.cochat.ai/paper-figures/acbe517b1cfe4f9b8d1eb0df227240b1/general-pitfalls-of-model-agnostic-interpretation-methods-for-machine-learning-figure-7.png)
![Fig. 8 Ignoring model and approximation uncertainty . PDP for X 1 with Y = 0 · X 1 + ∑ 10 j =2 X j + ϵ i with X 1 , . . . , X 10 ∼ U [0 , 1] and ϵ i ∼ N (0 , 0 . 9). Left: PDP for X 1 of a random forest trained on 100 data points. Middle: Multiple PDPs (10x) for the model from left plots, but with different samples (each n=100) for PDP estimation. Right: Repeated (10x) data samples of n=100 and newly fitted random forest.](https://figures.cochat.ai/paper-figures/bf95f6cae70f49538e3b3bfd16542e68/general-pitfalls-of-model-agnostic-interpretation-methods-for-machine-learning-figure-8.png)


Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“However, these methods can be misleading when correlations exist between input variables 107,109 .”
From Machine learning in concrete science: applications, challenges, and best practices (2022) · cites this paper as [109]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538 · cited 1×
“Global methods include the partial dependence plot (PDP) [30], partial importance (PI) [16], accumulated local affects (ALE) [2], or the permutation feature importance (PFI) [10,28,16].”
MethodScikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016 · cited 2×
“These resampling procedures are readily available in software [58,76], and well-studied in theory as well as practice [3,9,88], although rigorous analysis of cross-validation is still considered an open problem [87].”
BackgroundRegression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482 · cited 1×
“An alternative is to directly use models that implicitly perform feature selection such as LASSO [95] or component-wise boosting [84] as they can produce sparse models with fewer features.”
MethodElements of Information Theory
Cover, T. M., & Thomas, J. A. (2001) · Wiley eBooks · doi:10.1002/0471200611 · Cited by 36,808
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707 · cited 5×
“The variance of the estimates reduces in O ( 1 m ), where m is the number of evaluated orderings [21,67].”
BackgroundMethodInfluentialA Simple Sequentially Rejective Multiple Test Procedure
(1979) · Scandinavian Journal of Statistics · doi:10.2307/4615733 · Cited by 21,526 · cited 1×
“Since MCP is well known in statistics, we refer the practitioner to [24] for an overview and discussion of alternative adjustment methods, such as the Bonferroni-Holm method [44].”
Method"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Elements of Information Theory
Cover, T. M., & Thomas, J. A. (2005) · doi:10.1002/047174882x · Cited by 13,791
Gauriat, C.-M., Pencolé, Y., Ribot, P., & Brouillet, G. (2024) · HAL (Le Centre pour la Communication Scientifique Directe) · doi:10.4230/oasics.dx.2024.27 · Cited by 12,586
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561 · cited 3×
“However, further research is required, as there is no single perfect definition of interpretability, but rather multiple depending on the context [25,82].”
BackgroundMethod
Research works printed but not linked: 6
- Brenning, A.: Transforming feature space to interpret machine learning models. arXiv:2104.04295 (2021) · arXiv
- Freiesleben, T.: Counterfactual explanations & adversarial examples-common grounds, essential differences, and potential transfers. arXiv preprint arXiv:2009.05487 (2020) · arXiv
- Curran Associates, Inc. (2017). https://doi.org/ 10.5555/3295222.3295230 · doi:10.5555/3295222.3295230
- Molnar, C., Freiesleben, T., König, G., Casalicchio, G., Wright, M.N., Bischl, B.: Relating the partial dependence plot and permutation feature importance to the data generating process. arXiv preprint arXiv:2109.01433 (2021) · arXiv
- Semenova, L., Rudin, C., Parr, R.: A study in Rashomon curves and volumes: a new perspective on generalization and model simplicity in machine learning. arXiv preprint arXiv:1908.01755 (2021) 103. · arXiv
- Alzate, C., et al. (eds.) MIDAS/PAP -2018. LNCS (LNAI), vol. 11054, pp. 81-94. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-13463-1 6 · DOI
Cited by
Items 101–200 of 313
Kalashnikov, D. A., Davenport, F. V., Labe, Z. M., Loikith, P. C., Abatzoglou, J. T., & Singh, D. (2024) · Journal of Geophysical Research Atmospheres · doi:10.1029/2024jd042147 · Cited by 14
Vrhovac, V., Orošnjak, M., Ristić, K., Sremčev, N., Jocanović, M., Spajić, J., & Brkljač, N. (2024) · Mathematics · doi:10.3390/math12233794 · Cited by 14
Towards a responsible machine learning approach to identify forced labor in fisheriesPreprint
Joo, R., McDonald, G. G., Miller, N., Kroodsma, D. A., Farthing, C. M., Belhabib, D., & Hochberg, T. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2302.10987 · Cited by 16
Qualitätsdimensionen maschinellen Lernens in der amtlichen Statistik
Saidani, Y., Dumpert, F., Borgs, C., Brand, A., Nickl, A., Rittmann, A., Rohde, J., Salwiczek, C., Storfinger, N., & Straub, S. (2023) · AStA Wirtschafts- und Sozialstatistisches Archiv · doi:10.1007/s11943-023-00329-7 · Cited by 16
The influence of correlated features on neural network attribution methods in geoscience
Krell, E., Mamalakis, A., King, S. A., Tissot, P. E., & Ebert‐Uphoff, I. (2025) · Environmental Data Science · doi:10.1017/eds.2025.19 · Cited by 18
“This type of explanation has been referred to as feature effect (Molnar et al., 2020) or feature relevance (Flora et al., 2024), in contrast to feature importance methods that rank features based on their contribution to global model performance.”
Bobek, S., & Nalepa, G. J. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103357 · Cited by 17
Cottin, A., Zulian, M., Pécuchet, N., Guilloux, A., & Katsahian, S. (2023) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2023.102741 · Cited by 13
Credit Risk Modeling Using Interpreted XGBoost
Hernes, M., Adaszyński, J., & Tutak, P. (2023) · European Management Studies · doi:10.7172/2956-7602.101.3 · Cited by 15
Bai, G., Deng, Y., Chen, M., Zhu, L., Tuo, Y., Nie, M., Jia, Z., & Wang, X. (2024) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2024.175484 · Cited by 12
Good practices and common pitfalls of machine learning in nutrition research
Kirk, D. (2024) · Proceedings of The Nutrition Society · doi:10.1017/s0029665124007638 · Cited by 14
“Different approaches estimate feature importance in different ways, often arriving at different conclusions (112) , though popular approaches with good statistical properties include SHAP (113) , Lime (114) , and permutation-based feature importance (115) , amongst others (111) .”
Lavelle-Hill, R. E., Smith, G., & Murayama, K. (2025) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459251345696 · Cited by 17
“For example, training a model on different imputed data sets will likely result in different hyperparameters and features being selected in each data set (Molnar et al., 2020), and it is not clear whether it makes sense to integrate results from models with different hyperparameters/predictors.”
InfluentialSchulze, M. C., Prakash, N., Gering, K. L., Colclasure, A. M., & Weddle, P. J. (2025) · Journal of Power Sources · doi:10.1016/j.jpowsour.2025.238338 · Cited by 11
Love, P. E. D., Matthews, J., Fang, W., Porter, S. R., Luo, H., & Ding, L. (2023) · IEEE Engineering Management Review · doi:10.1109/emr.2023.3342200 · Cited by 14
“Restrictions apply. construction organization [42],[55], [61].”
Schenk, P. O., & Kern, C. A. (2024) · AStA Wirtschafts- und Sozialstatistisches Archiv · doi:10.1007/s11943-024-00344-2 · Cited by 16
Love, P. E. D., Matthews, J., Fang, W., Porter, S. R., Luo, H., & Ding, L. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2211.06561 · Cited by 13
Elakrouch, M., Mohaddab, M., Rachid, A., & Jijakli, M. H. (2025) · Sustainability · doi:10.3390/su18010064 · Cited by 11
Artificial Intelligence in Concrete Materials
Soltangharaei, V., Li, Z., & Ziehl, P. H. (2022) · CRC Press eBooks · doi:10.1201/9780367823467-8 · Cited by 12
Krell, E., Kamangir, H., Collins, W. G., King, S. A., & Tissot, P. E. (2023) · Environmental Data Science · doi:10.1017/eds.2023.39 · Cited by 10
“Molnar et al. (2020) gives a detailed explanation of SHAP and its advantages and disadvantages.”
BackgroundMethodInfluentialGamma power and beta envelope correlation are potential neural predictors of deep hypnosis
Farahzadi, Y., Alldredge, C. T., & Kekecs, Z. (2024) · Scientific Reports · doi:10.1038/s41598-024-56633-x · Cited by 13
“This is computed by retraining the model for each subset of features that includes or excludes the feature of interest 68 .”
MethodHilburn, K. (2023) · Artificial Intelligence for the Earth Systems · doi:10.1175/aies-d-22-0093.1 · Cited by 10
RHALE: Robust and Heterogeneity-Aware Accumulated Local Effects
Gkolemis, V., Dalamagas, T., Ntoutsi, E., & Diou, C. (2023) · Frontiers in artificial intelligence and applications · doi:10.3233/faia230354 · Cited by 18
Niu, W., Feng, Y., Li, J., Xu, S., Ma, Z., & Wang, Y. Y. (2025) · Translational Psychiatry · doi:10.1038/s41398-025-03360-0 · Cited by 9
“Speci fi cally, for adults with CSA experiences, Steine et al. found that the number of types of childhood trauma and frequency of victimization were positively associated with the incidence of SA [15].”
Waldauf, P., Scales, N. B., Shahin, J., Schmidt, M., van Beinum, A., Hornby, L., Shemie, S. D., Hogue, M., Wind, T. J., van Mook, W., et al. (2023) · Scientific Reports · doi:10.1038/s41598-023-29042-9 · Cited by 9
“6* Peak inspiratory pressure [cmH2O] 22 (18, 27) 24 (18, 30) 22 (18, 26) < 0.”
The Blame Problem in Evaluating Local Explanations and How to Tackle It
Rahnama, A. H. A. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-50396-2_4 · Cited by 9
The Use of Machine Learning Methods in Political Science: An In-Depth Literature Review
de Slegte, J., Droogenbroeck, F. V., Spruyt, B., Verboven, S., & Ginis, V. (2024) · Political Studies Review · doi:10.1177/14789299241265084 · Cited by 16
“…packages are available that are capable of gaining accurate insights from the data, yet bad model generalization for predictions, unjustified interpretations or unwanted bias in practice sometimes challenge the results coming out of these models (Molnar et al., 2022; van Giffen et al., 2022).”
Wang, D., Xia, Y., Pedrycz, W., Li, Z., & Yu, Z. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.127553 · Cited by 10
Personalized survival probabilities for SARS-CoV-2 positive patients by explainable machine learning
Zucco, A. G., Agius, R., Svanberg, R., Moestrup, K. S., Marandi, R. Z., MacPherson, C. R., Lundgren, J. D., Ostrowski, S. R., & Niemann, C. U. (2022) · Scientific Reports · doi:10.1038/s41598-022-17953-y · Cited by 10
TIGTEC: Token Importance Guided TExt Counterfactuals
Bhan, M., Vittaut, J.-N., Chesneau, N., & Lesot, M. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-43418-1_30 · Cited by 19
Towards Quality Measures for xAI algorithms: Explanation Stability
Pawlicki, M. (2023) · IEEE International Conference on Data Science and Advanced Analytics (DSAA) · doi:10.1109/dsaa60987.2023.10302535 · Cited by 13
“In [9], a list of common pitfalls for the use of common xAI methods is presented.”
MethodUwanuakwa, I. D., Amir, I. Y., & Umba, L. N. (2024) · Journal of Road Engineering · doi:10.1016/j.jreng.2024.05.001 · Cited by 8
Deininger, H., Lavelle-Hill, R. E., Parrisius, C., Pieronczyk, I., Colling, L., Meurers, D., Trautwein, U., Nagengast, B., & Kasneci, G. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-36272-9_46 · Cited by 11
Takefuji, Y. (2025) · Journal of Catalysis · doi:10.1016/j.jcat.2025.116098 · Cited by 7
Takefuji, Y. (2025) · Environmental Research · doi:10.1016/j.envres.2025.123125 · Cited by 7
Evaluating the role of information disclosure on bidding behavior in wholesale electricity markets
Brown, D. P., Cajueiro, D. O., Eckert, A., & Silveira, D. (2025) · Energy Economics · doi:10.1016/j.eneco.2025.108505 · Cited by 9
ODExAI: A Comprehensive Object Detection Explainable AI Evaluation
Nguyen, L., Nguyen, T. T. H., & Cao, H. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-02813-6_9 · Cited by 11
Characterizing climate pathways using feature importance on echo state networks
Goode, K., Ries, D., & McClernon, K. (2024) · Statistical Analysis and Data Mining The ASA Data Science Journal · doi:10.1002/sam.11706 · Cited by 9
Multidimensional poverty in Benin
Ramalho, E. A., Barbosa, L., & Bastos, J. A. (2025) · Socio-Economic Planning Sciences · doi:10.1016/j.seps.2025.102181 · Cited by 8
Xin, X., Hooker, G., & Huang, F. (2025) · Insurance: Mathematics and Economics · doi:10.1016/j.insmatheco.2025.103135 · Cited by 6
Hui, J., Wei, X., & Hou, Y. (2025) · Applied Soft Computing · doi:10.1016/j.asoc.2025.113682 · Cited by 7
Don’t Lie to Me: Avoiding Malicious Explanations With STEALTH
Alvarez, L., & Menzies, T. (2023) · IEEE Software · doi:10.1109/ms.2023.3244713 · Cited by 7
Addressing diversity in hiring procedures: a generative adversarial network approach
Marra, T., & Kubiak, E. (2024) · AI and Ethics · doi:10.1007/s43681-024-00445-2 · Cited by 8
“Permutation feature importance—PFI [48], which provides a model-agnostic approach to gauge the impact of each feature on predictive accuracy, and conditional feature impor-tance—CFI [48, 110], are common techniques to counter that problem.”
Spatial Prediction of Organic Matter Quality in German Agricultural Topsoils
Sakhaee, A., Scholten, T., Taghizadeh‐Mehrjardi, R., Ließ, M., & Don, A. (2024) · Agriculture · doi:10.3390/agriculture14081298 · Cited by 7
“The difference between GPFI and the Shapley value is that while the former shows the effect of a predictor on model performance, the latter illustrates its contribution to the prediction [66].”
BackgroundMethodInfluentialAI-Based Electroencephalogram Analysis in Rodent Models of Epilepsy: A Systematic Review
Edoho, M. E., Mooney, C., & Wei, L. (2024) · Applied Sciences · doi:10.3390/app14167398 · Cited by 8
Explainable AI for Intrusion Detection Systems: A Model Development and Experts’ Evaluation
Durojaye, H., & Naiseh, M. (2024) · Lecture notes in networks and systems · doi:10.1007/978-3-031-66428-1_18 · Cited by 8
Unveiling the drivers contributing to global wheat yield shocks through quantile regression
Vishwakarma, S., Zhang, X., & Lyubchich, V. (2025) · Artificial Intelligence in Agriculture · doi:10.1016/j.aiia.2025.03.004 · Cited by 5
Rehman, Z. U., Chaozhe, J., Jamal, A., Almarhabi, Y., Wang, C., Alam, E., & Islam, M. K. (2025) · Transportation Engineering · doi:10.1016/j.treng.2025.100352 · Cited by 5
Metikoš, L., & van Domselaar, I. (2025) · Journal of Human-Technology Relations · doi:10.59490/jhtr.2025.3.8163 · Cited by 8
Bhan, M., Vittaut, J.-N., Chesneau, N., & Lesot, M. (2023) · Workshop on Trustworthy Natural Language Processing (TrustNLP) · doi:10.18653/v1/2023.trustnlp-1.19 · Cited by 8
“develop methods to interpret and explain such model behaviour (Molnar et al., 2021).”
BackgroundMethodHeberle, B. A., Kluwe‐Schiavon, B., Bicca, C., Rothmann, L. M., Grassi‐Oliveira, R., & Viola, T. W. (2023) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2023.11.043 · Cited by 6
“Information regarding the strength and shape of the associations between features has been suggested as an additional tool for understanding ML models (Molnar et al., 2022).”
MethodCounterfactual Explanations for Models of CodePreprint
Cito, J., Dillig, I., Murali, V., & Chandra, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.05711 · Cited by 5
“However, when designing explanatory systems, there are several criteria and pitfalls to consider, some of which have been discussed in the literature [25].”
BackgroundREPID: Regional Effect Plots with implicit Interaction DetectionPreprint
Herbinger, J., Bischl, B., & Casalicchio, G. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2202.07254 · Cited by 32
“Approaches to group ICE curves to reduce feature dependencies instead of feature interactions is introduced in Molnar et al. (2021a) and Grömping (2020).”
BackgroundInfluentialSelf-citationA Review of Evaluation Approaches for Explainable AI With Applications in CardiologyPreprint
Salih, A. M., Galazzo, I. B., Gkontra, P., Rauseo, E., Lee, A. M., Lekadir, K., Radeva, P. I., Petersen, S. E., & Menegaz, G. (2023) · doi:10.36227/techrxiv.24573304.v1 · Cited by 7
Komarchesqui, M., Ruffo, V. G. D. S., Carvalho, L. F., & Proença, M. L. (2026) · Journal of Network and Systems Management · doi:10.1007/s10922-026-10084-z · Cited by 4
Kabiri, S., & O'Rourke, S. M. (2024) · Frontiers in Environmental Science · doi:10.3389/fenvs.2024.1392469 · Cited by 4
Integrating traditional and non-traditional model risk frameworks in credit scoring
du Toit, H. A., Schutte, W. D., & Raubenheimer, H. (2024) · South African Journal of Economic and Management Sciences · doi:10.4102/sajems.v27i1.5786 · Cited by 4
Oka, S., & Takefuji, Y. (2025) · Scripta Materialia · doi:10.1016/j.scriptamat.2025.116857 · Cited by 4
Local Interpretable Model-Agnostic Explanations for Neural Ranking Models
Rahnama, A. H. A., Alfaro, L. G., Wang, Z., & Movin, M. (2024) · Linköping electronic conference proceedings · doi:10.3384/ecp208017 · Cited by 6
“As other studies have shown [8, 19], transforming the data into binary representations comes with a limitation: we are operating in a data space that is different than the original data space, and moreover, there is an information loss.”
BackgroundA SHAP-based controversy analysis through communities on Twitter
Benslimane, S., Παπαστεργίου, Θ., Azé, J., Bringay, S., Servajean, M., & Mollévi, C. (2024) · World Wide Web · doi:10.1007/s11280-024-01278-z · Cited by 6
Algorithm-Agnostic Feature Attributions for Clustering
Scholbeck, C. A., Funk, H., & Casalicchio, G. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-44064-9_13 · Cited by 11
Self-citationOka, S., Yamazaki, T., & Takefuji, Y. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2025.110710 · Cited by 6
Interaction Difference Hypothesis Test for Prediction Models
Welchowski, T., & Edelmann, D. (2024) · Machine Learning and Knowledge Extraction · doi:10.3390/make6020061 · Cited by 3
The Case for Hybrid Multi-Objective Optimisation in High-Stakes Machine Learning Applications
Freitas, A. A. (2024) · ACM SIGKDD Explorations Newsletter · doi:10.1145/3682112.3682116 · Cited by 6
“…fairness), there are usually multiple specific measures of the quality of a predictive model measuring different aspects of that criterion – discussed e.g. in [25], [30], [37] for predictive accuracy measures; [38], [10], [60] for fairness measures; and [6], [41] for interpretability measures.”
Dual feature-based and example-based explanation methods
Konstantinov, A. V., Kozlov, B. N., Kirpichenko, S. R., Utkin, L. V., & Muliukha, V. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1506074 · Cited by 3
Explainable Machine Learning Models for Credit Rating in Colombian Solidarity Sector Entities
Serna, M. A. A., Montealegre, J. J. Q., Móntes-Gómez, L. F., Orozco‐Duque, A., & Clavijo, L. U. (2025) · Journal of risk and financial management · doi:10.3390/jrfm18090489 · Cited by 3
Głuchowska, A., Khan, M. H. U., & Pawełkowicz, M. E. (2026) · Applied Sciences · doi:10.3390/app16147275 · Cited by 4
Takefuji, Y. (2025) · Coordination Chemistry Reviews · doi:10.1016/j.ccr.2025.216584 · Cited by 3
Unravelling the Enigma of Machine Learning Model Interpretability in Enhancing Disease Prediction
Tripathi, R. K. P., & Tiwari, S. (2023) · Advances in systems analysis, software engineering, and high performance computing book series · doi:10.4018/978-1-6684-8531-6.ch007 · Cited by 3
Wolf, D. W., Ulrich, M., & Kapoor, N. (2023) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw60793.2023.00443 · Cited by 3
“Hence, the Shapley value is a local explanation method [23], which describes the feature effect by quantifying the direction and magnitude of the local gradient in the feature space.”
MethodEvaluation Methodology for Interpretation Methods of Predictive Quality Models
Schulze, T., Buschmann, D., & Schmitt, R. H. (2024) · Procedia CIRP · doi:10.1016/j.procir.2024.08.366 · Cited by 3
Glass box machine learning and corporate bond returns
Bell, S., Kakhbod, A., Lettau, M., & Nazemi, A. (2026) · Journal of Financial Economics · doi:10.1016/j.jfineco.2026.104294 · Cited by 10
A Review of Evaluation Approaches for Explainable AI With Applications in CardiologyPreprint
Salih, A. M., Galazzo, I. B., Gkontra, P., Rauseo, E., Lee, A. M., Lekadir, K., Radeva, P. I., Petersen, S. E., & Menegaz, G. (2023) · doi:10.36227/techrxiv.24573304 · Cited by 3
Zhi, H., Qiu, Y., Jiang, H., Xiao, M., Han, L., Lei, S., & Niu, H. (2025) · Scientific Data · doi:10.1038/s41597-025-05268-2 · Cited by 3
Automatic Construction of Clinical Scoring Systems with LLM AgentsPreprint
Estévez, S. R., Chiu, C., & van der Schaar, M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2601.22324 · Cited by 3
Data-driven production forecast for oil and gas reservoirs - Advantages and limitations
Pires, L. O., Lavi, B., Bertini, J. R., & Schiozer, D. J. (2026) · Geoenergy Science and Engineering · doi:10.1016/j.geoen.2026.214569 · Cited by 2
Oberleitner, T., Baklanov, A., Ferreira, T. B., Hoogenboom, G., Jägermeyr, J., Jain, A. K., Lin, T., Mialyk, O., Müller, C., Ruane, A. C., et al. (2026) · Earth s Future · doi:10.1029/2025ef007457 · Cited by 2
“Explanation fidelity: While easy to communicate, global estimators, by definition, do not explain model behavior across the whole input domain and can be ambiguous and misleading, even for relatively simple models (Molnar et al., 2022).”
Zhao, M., Šćepanović, S., Law, S., Obadić, I., Wu, C., & Quercia, D. (2026) · Computers Environment and Urban Systems · doi:10.1016/j.compenvurbsys.2026.102466 · Cited by 2
Mixture of checkpoint experts for explainable seizure detection using wearable devices
Germino, J., Brinkmann, B. H., Chawla, N. V., Cui, J., Fortune, E., Moniz, N., & Faust, L. J. (2025) · Scientific Reports · doi:10.1038/s41598-025-27610-9 · Cited by 2
An Updated Review on Differential Diagnosis of Crohn\'s Disease and Intestinal Tuberculosis
Sharma, R. K., Singh, A., & Dawra, S. (2025) · Journal of Postgraduate Medicine Education and Research · doi:10.5005/jp-journals-10028-1745 · Cited by 2
DQSA: Dynamic Quantized Self-Attention for Multi-Task Encrypted Network Traffic Classification
Chen, Y., Lan, H., Gwee, B., Cao, Q. P., Razul, S. G., & Lin, Z. (2025) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2025.3647211 · Cited by 2
“This highlights the need for models that integrate interpretability into the classification process itself, eliminating dependence on post-hoc analysis and reducing the associated risks [11].”
A Novel Hybrid Approach for Identification of Discriminative Features in Phishing Emails
Mbura, R., Kato, A., & Sinde, R. (2025) · IEEE Access · doi:10.1109/access.2025.3649636 · Cited by 3
Learning to melt: Emulating Greenland surface melt from a polar RCM with machine learning
Schlager, E., Scher, S., Mottram, R. H., & Langen, P. L. (2026) · The Cryosphere · doi:10.5194/tc-20-3313-2026 · Cited by 4
A Tutorial on Best Practices and Pitfalls in Applying Machine Learning to Environmental Research
Yan, Z., Li, J., Zhang, W., Wen, H., Yu, H., Yu, M., Liu, Q. S., & Jiang, G. (2026) · ACS Environmental Au · doi:10.1021/acsenvironau.6c00025 · Cited by 2
Lemorzsolódás előrejelzése személyre szabott értelmezhető gépi tanulási módszerek segítségével
Nagy, M. (2023) · Scientia et Securitas · doi:10.1556/112.2022.00107 · Cited by 2
Quantification of Explainability in Black Box Models using Complexity Measures
Nneke, N., Lloyd, H., & Crockett, K. A. (2023) · International Conference on Innovations in Information Technology · doi:10.1109/iit59782.2023.10366490 · Cited by 2
“The SHAP method is classified into two types: Local explanations and Global explanations [10].”
BackgroundMethodMETA-SMGO-$\Delta$: Similarity as a Prior in Black-Box Optimization
Busetto, R., Breschi, V., & Formentin, S. (2023) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc49753.2023.10383592 · Cited by 3
Chaubey, V. P., Sareen, D., Harshita, C., Sharma, S., Rana, R. P. S., & Batra, I. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-84059-3_3 · Cited by 2
Navigating the informativeness-compression trade-off in XAI
Oldenburg, N., & Søgaard, A. (2025) · AI and Ethics · doi:10.1007/s43681-025-00733-5 · Cited by 4
“However, a rigor uncertainty quantification can help establish user awareness of these pitfalls and prevent good-enough understandings [61, 80].”
What’s Wrong with Your Synthetic Tabular Data? Using Explainable AI to Evaluate Generative Models
Kapar, J., Koenen, N., & Jullum, M. (2025) · Communications in computer and information science · doi:10.1007/978-3-032-08327-2_2 · Cited by 7
Interpretable Machine Learning for Official Statistics
Dandl, S., Bischl, B., & Bothmann, L. (2025) · Society, environment and statistics · doi:10.1007/978-3-032-10004-7_5 · Cited by 3
Self-citationA Principled Approach for Defining and Comparing Variable Importance MeasuresPreprint
Lobo, A. R., Neuvial, P., & Thirion, B. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2507.17306 · Cited by 2
CHILLI : a data context-aware perturbation method for XAIPreprint
Anwar, S., Griffiths, N., Bhalerao, A. H., Popham, T., Bell, M., & Hellman, S. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2407.07521 · Cited by 3
“For example, when perturbing a feature representing a person’s age, the perturbation method must consider that values cannot be negative or unreason-ably large (Molnar et al., 2021b).”
An evolutionary estimation procedure for generalized semilinear regression trees
Vannucci, G., & Gottard, A. (2022) · Computational Statistics · doi:10.1007/s00180-022-01302-8 · Cited by 2
MASALA : Model-Agnostic Surrogate Explanations by Locality Adaptation
Anwar, S., Griffiths, N., Bhalerao, A. H., & Popham, T. (2024) · Warwick Research Archive Portal (University of Warwick) · doi:10.48550/arxiv.2408.10085 · Cited by 1
Schmid, M., Andersson, F. O., & Willems, J. (2025) · Journal of School Choice · doi:10.1080/15582159.2025.2505812 · Cited by 1
“That is, they are prediction-centered and do not yield model coefficients themselves (Molnar et al., 2022; Si, 2022).”
Tracing need for cognition in digital learning
Schumacher, A., von Keyserlingk, L., Deininger, H., Yu, R., Nixon, N., & Bardach, L. (2026) · Computers in Human Behavior · doi:10.1016/j.chb.2026.109021 · Cited by 1
Beyond explainable AI: Enhancing trust and robustness in machine learning for sleep apnea diagnosis
Takefuji, Y. (2025) · Sleep Medicine Reviews · doi:10.1016/j.smrv.2025.102152 · Cited by 1
Identification of Key Feature Interactions via PDP Decomposition
Eryilmaz, S. E., & Triepels, R. (2025) · Communications in computer and information science · doi:10.1007/978-3-032-04339-9_10 · Cited by 1
Does the Model Say What the Data Says? A Simple Heuristic for Model–Data Alignment
Salgado, H., Kendall, M., & Ceberio, M. C. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-20548-3_4 · Cited by 1
“Post-hoc methods, meanwhile, are computationally expensive and typically highlight associations rather than causal relationships, which can mislead users about the true drivers of model outputs [8].”
Plintz, N. B., Vetter, M., & Ifenthaler, D. (2026) · IEEE Access · doi:10.1109/access.2026.3737607 · Cited by 1
Metadata
- Authors
- Molnar, C., König, G., Herbinger, J., Freiesleben, T., Dandl, S., Scholbeck, C. A., Casalicchio, G., Grosse-Wentrup, M., & Bischl, B.
- Affiliations
- Leibniz Institute for Prevention Research and Epidemiology - BIPS (DE)
- University of Vienna (AT)
- Ludwig-Maximilians-Universität München (DE)
- Corresponding author
- Molnar, Christoph
- ORCID
- Venue
- Lecture Notes in Computer Science
- Publisher
- Springer International Publishing
- Published
- 2022
- Pages
- 39-68
- ISSN
- 0302-9743 (ISSN-L)
- 1611-3349
- OpenAlex
- W3194357095
- Cited by
- 211
- References
- 130
- Citations by year
- 2026: 41 · 2025: 56 · 2024: 48 · 2023: 51 · 2022: 11
- FWCI
- 22.95
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor, 106 d)
- Locations
- Lecture notes in computer science (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- Deutsche Forschungsgemeinschaft (437611051)
- Bundesministerium für Bildung und Forschung (01IS18036A)
- Graduate School of Systemic Neurosciences
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Keywords
- model-agnostic interpretability, explainable machine learning, partial dependence plots, permutation importance, Shapley value, feature interaction, uncertainty estimation, high-dimensional data, causal interpretation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.