Cited by 10,551Explainable Artificial Intelligence (XAI)Artificial Intelligence in HealthcareData Mining Algorithms and Applications

The DOI matches its Crossref record; 45 of the 45 references it cites were found in a registry or an index.

From local explanations to global understanding with explainable AI for trees

Open access

Lundberg, Scott M., Erion, Gabriel, Chen, Hugh, DeGrave, Alex, Prutkin, Jordan M., Nair, Bala, Katz, Ronit, Himmelfarb, Jonathan, Bansal, Nisha, Lee, Su-In·2020·Nature Machine Intelligence

DOI 10.1038/s42256-019-0138-9PMC PMC7326367

Open this paper in CoChat — free account

Abstract

Tree-based machine learning models such as random forests, decision trees, and gradient boosted trees are popular non-linear predictive models, yet comparatively little attention has been paid to explaining their predictions. Here, we improve the interpretability of tree-based models through three main contributions: 1) The first polynomial time algorithm to compute optimal explanations based on game theory. 2) A new type of explanation that directly measures local feature interaction effects. 3) A new set of tools for understanding global model structure based on combining many local explanations of each prediction. We apply these tools to three medical machine learning problems and show how combining many high-quality local explanations allows us to represent global structure while retaining local faithfulness to the original model. These tools enable us to i) identify high magnitude but low frequency non-linear mortality risk factors in the US population, ii) highlight distinct population sub-groups with shared risk characteristics, iii) identify non-linear interaction effects among risk factors for chronic kidney disease, and iv) monitor a machine learning model deployed in a hospital by identifying which features are degrading the model's performance over time. Given the popularity of tree-based machine learning models, these improvements to their interpretability have implications across a broad set of domains.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1038/s42256-019-0138-9

Cites

45 of 45 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Random Forests
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
XGBoost: A Scalable Tree Boosting System
Chen, T., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939785 · Cited by 54,569
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707
Self-citation
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
On a Method to Measure Supervised Multiclass Model’s Interpretability: Application to Degradation Diagnosis (Short Paper)
Gauriat, C.-M., Pencolé, Y., Ribot, P., et al. (2024) · HAL · doi:10.4230/oasics.dx.2024.27 · Cited by 12,623
A Value for n-person Games
Shapley, L. (1988) · Cambridge University Press eBooks · doi:10.1017/cbo9780511528446.003 · Cited by 10,698
Heart Disease and Stroke Statistics—2016 Update: A Report From the American Heart Association
Mozaffarian, D., Benjamin, E. J., Go, A. S., Arnett, D. K., Blaha, M. J., Cushman, M., Das, S. R., de Ferranti, S., Després, J.-P., Fullerton, H. J., et al. (2016) · Circulation · doi:10.1161/cir.0000000000000350 · Cited by 6,596
A Randomized Trial of Intensive versus Standard Blood-Pressure Control
The SPRINT Research Group (2015) · New England Journal of Medicine · doi:10.1056/nejmoa1511939 · Cited by 6,325
Heart Disease and Stroke Statistics—2015 Update: A Report From the American Heart Association
Mozaffarian, D., Benjamin, E. J., Go, A. S., Arnett, D. K., Blaha, M. J., Cushman, M., de Ferranti, S., Després, J.-P., Fullerton, H. J., Howard, V. J., et al. (2015) · Circulation · doi:10.1161/cir.0000000000000152 · Cited by 5,797
Repeated observation of breast tumor subtypes in independent gene expression data sets
Sørlie, T., Tibshirani, R. J., Parker, J. S., Hastie, T. J., Marron, J. S., Nobel, A. B., Deng, S., Johnsen, H., Pesich, R., Geisler, S. B., et al. (2003) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0932692100 · Cited by 5,508

Show all 38 references

Sorted by citations

Explainable Artificial Intelligence (XAI): What we know and what is left to attain Trustworthy Artificial Intelligence
Ali, S., Abuhmed, T., El-Sappagh, S., Muhammad, K., Alonso-Moral, J. M., Confalonieri, R., Guidotti, R., Del Ser, J., Díaz-Rodríguez, N., & Herrera, F. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101805 · Cited by 1,762
Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications
Samek, W., Montavon, G., Lapuschkin, S., Anders, C. J., & Muller, K.-R. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3060483 · Cited by 1,477
“sex of the participant (figure is adapted from [115]).”
BackgroundMethodInfluential
Explainable AI (XAI): Core Ideas, Techniques, and Solutions
Dwivedi, R., Dave, D., Naik, H., Singhal, S., Omer, R., Patel, P., Qian, B., Wen, Z., Shah, T., Morgan, G., et al. (2023) · ACM Computing Surveys · doi:10.1145/3561048 · Cited by 1,386
An interpretable mortality prediction model for COVID-19 patients
Li, Y., Zhang, H., Gonçalves, J. M., Xiao, Y., Wang, M., Guo, Y., Sun, C., Tang, X., Jing, L., Zhang, M., et al. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-0180-7 · Cited by 1,328
Deep Neural Networks and Tabular Data: A Survey
Borisov, V., Leemann, T., Seßler, K., Haug, J., Pawelczyk, M., & Kasneci, G. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3229161 · Cited by 1,400
Drug discovery with explainable artificial intelligence
Jiménez-Luna, J., Grisoni, F., & Schneider, G. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-00236-4 · Cited by 1,063
Algorithmic bias in education
Baker, R. S., & Hawn, A. (2022) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-021-00285-9 · Cited by 993
A governance model for the application of AI in health care
Reddy, S., Allan, S., Coghlan, S., & Cooper, P. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz192 · Cited by 979
Deep learning in cancer diagnosis, prognosis and treatment selection
Tran, K. A., Kondrashova, O., Bradley, A., Williams, E. D., Pearson, J. V., & Waddell, N. (2021) · Genome Medicine · doi:10.1186/s13073-021-00968-x · Cited by 932
Practical guide to SHAP analysis: Explaining supervised machine learning model predictions in drug development
Bobadilla, A. V. P., Schmitt, V., Maier, C. S., Mensing, S., & Stodtmann, S. (2024) · Clinical and Translational Science · doi:10.1111/cts.70056 · Cited by 980
Viral epitope profiling of COVID-19 patients reveals cross-reactivity and correlates of severity
Shrock, E., Fujimura, E., Kula, T., Timms, R. T., Lee, I.-H., Leng, Y., Robinson, M. L., Sie, B. M., Li, M. Z., Chen, Y., et al. (2020) · Science · doi:10.1126/science.abd4250 · Cited by 678
Unbox the black-box for the medical explainable AI via multi-modal and multi-centre data fusion: a mini-review, two showcases and beyond
Yang, G., Ye, Q., & Xia, J. (2022) · Information Fusion · doi:10.1016/j.inffus.2021.07.016 · Cited by 717
Interpretation of machine learning models using shapley values: application to compound potency and multi-target activity predictions
Rodríguez-Pérez, R., & Bajorath, J. (2020) · Journal of Computer-Aided Molecular Design · doi:10.1007/s10822-020-00314-0 · Cited by 718
Pan-cancer integrative histology-genomic analysis via multimodal deep learning
Chen, R. J., Lu, M. Y., Williamson, D. F. K., Chen, T. Y., Lipkova, J., Noor, Z., Shaban, M., Shady, M., Williams, M., Joo, B., et al. (2022) · Cancer Cell · doi:10.1016/j.ccell.2022.07.004 · Cited by 660
The Future of Sensitivity Analysis: An essential discipline for systems modeling and policy support
Razavi, S., Jakeman, A. J., Saltelli, A., Prieur, C., Iooss, B., Borgonovo, E., Plischke, E., Piano, S. L., Iwanaga, T., Becker, W., et al. (2020) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2020.104954 · Cited by 703
Understanding the diversity of the metal-organic framework ecosystem
Moosavi, S. M., Nandy, A., Jablonka, K. M., Ongari, D., Janet, J. P., Boyd, P. G., Lee, Y., Smit, B., & Kulik, H. J. (2020) · Nature Communications · doi:10.1038/s41467-020-17755-8 · Cited by 622
Explainable Machine Learning in Credit Risk Management
Bussmann, N., Giudici, P., Marinelli, D., & Papenbrock, J. (2020) · Computational Economics · doi:10.1007/s10614-020-10042-0 · Cited by 635
From explanations to feature selection: assessing SHAP values as feature selection mechanism
Marcilio, W. E., & Eler, D. M. (2020) · 2020 33rd SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI) · doi:10.1109/sibgrapi51738.2020.00053 · Cited by 504
Algorithms to estimate Shapley value feature attributions
Chen, H., Covert, I., Lundberg, S., & Lee, S. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00657-x · Cited by 540
Self-citation
On the Tractability of SHAP Explanations
den Broeck, G. V., Lykov, A., Schleich, M., & Suciu, D. M. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13283 · Cited by 501
Artificial Intelligence for Predictive Maintenance Applications: Key Components, Trustworthiness, and Future Trends
Uçar, A., Karaköse, M., & Kırımça, N. (2024) · Applied Sciences · doi:10.3390/app14020898 · Cited by 596
Interpretable XGBoost-SHAP Machine-Learning Model for Shear Strength Prediction of Squat RC Walls
Feng, D., Wang, W., Mangalathu, S., & Taciroğlu, E. (2021) · Journal of Structural Engineering · doi:10.1061/(asce)st.1943-541x.0003115 · Cited by 457
Explainable AI Methods - A Brief Overview
Holzinger, A., Saranti, A., Molnar, C., Biecek, P., & Samek, W. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-04083-2_2 · Cited by 518
Explainable artificial intelligence model to predict acute critical illness from electronic health records
Lauritsen, S. M., Kristensen, M., Olsen, M. V., Larsen, M. S., Lauritsen, K. M., Jørgensen, M. J., Lange, J., & Thiesson, B. (2020) · Nature Communications · doi:10.1038/s41467-020-17431-x · Cited by 448
CatBoost model and artificial intelligence techniques for corporate failure prediction
Jabeur, S. B., Gharib, C., Mefteh‐Wali, S., & Arfi, W. B. (2021) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2021.120658 · Cited by 420
Deterministic Local Interpretable Model-Agnostic Explanations for Stable Explainability
Zafar, M. R., & Khan, N. (2021) · Machine Learning and Knowledge Extraction · doi:10.3390/make3030027 · Cited by 432
Predictive Performances of Ensemble Machine Learning Algorithms in Landslide Susceptibility Mapping Using Random Forest, Extreme Gradient Boosting (XGBoost) and Natural Gradient Boosting (NGBoost)
Kavzoğlu, T., & Teke, A. (2022) · Arabian Journal for Science and Engineering · doi:10.1007/s13369-022-06560-8 · Cited by 379
On scientific understanding with artificial intelligence
Krenn, M., Pollice, R., Guo, S. Y., Aldeghi, M., Cervera-Lierta, A., Friederich, P., Gomes, G., Häse, F., Jinich, A., Nigam, A., et al. (2022) · Nature Reviews Physics · doi:10.1038/s42254-022-00518-3 · Cited by 444
An open-source, high-performance tool for automated sleep staging
Vallat, R., & Walker, M. P. (2021) · eLife · doi:10.7554/elife.70092 · Cited by 366
Proteomic aging clock predicts mortality and risk of common age-related diseases in diverse populations
Argentieri, M. A., Xiao, S., Bennett, D., Winchester, L. M., Nevado‐Holgado, A. J., Ghose, U., Albukhari, A., Yao, P., Mazidi, M., Lv, J., et al. (2024) · Nature Medicine · doi:10.1038/s41591-024-03164-7 · Cited by 421
Interpretable prediction of 3-year all-cause mortality in patients with heart failure caused by coronary heart disease based on machine learning and SHAP
Wang, K., Tian, J., Zheng, C., Yang, H., Ren, J., Liu, Y., Han, Q., & Zhang, Y. (2021) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2021.104813 · Cited by 361
Explanations in Autonomous Driving: A Survey
Omeiza, D., Webb, H., Jirotka, M., & Kunze, L. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3122865 · Cited by 355
Evaluating compressive strength of concrete made with recycled concrete aggregates using machine learning approach
Tran, V. Q., Dang, V. Q., & Ho, L. S. (2022) · Construction and Building Materials · doi:10.1016/j.conbuildmat.2022.126578 · Cited by 367
Explainable AI and Mass Surveillance System-Based Healthcare Framework to Combat COVID-I9 Like Pandemics
Hossain, M. S., Muhammad, G., & Guizani, N. (2020) · IEEE Network · doi:10.1109/mnet.011.2000458 · Cited by 347
Shapley values for feature selection: the good, the bad, and the axioms
Fryer, D., Strumke, I., & Nguyen, H. (2021) · IEEE Access · doi:10.1109/access.2021.3119110 · Cited by 347
Explainable machine learning can outperform Cox regression predictions and provide insights in breast cancer survival
Moncada‐Torres, A., van Maaren, M. C., Hendriks, M. P., Siesling, S., & Geleijnse, G. (2021) · Scientific Reports · doi:10.1038/s41598-021-86327-7 · Cited by 358
Quantifying and comparing the effects of key risk factors on various types of roadway segment crashes with LightGBM and SHAP
Wen, X. L., Xie, Y., Wu, L., & Jiang, L. (2021) · Accident Analysis & Prevention · doi:10.1016/j.aap.2021.106261 · Cited by 327
Using Explainable Artificial Intelligence to Improve Process Quality: Evidence from Semiconductor Manufacturing
Senoner, J., Netland, T., & Feuerriegel, S. (2022) · Management Science · doi:10.1287/mnsc.2021.4190 · Cited by 317
A Systematic Review of Explainable Artificial Intelligence in Terms of Different Application Domains and Tasks
Islam, M. R., Ahmed, M. U., Barua, S., & Begum, S. (2022) · Applied Sciences · doi:10.3390/app12031353 · Cited by 366
Towards better process management in wastewater treatment plants: Process analytics based on SHAP values for tree-based machine learning methods
Wang, D., Thunéll, S., Lindberg, U., Jiang, L., Trygg, J., & Tysklind, M. (2021) · Journal of Environmental Management · doi:10.1016/j.jenvman.2021.113941 · Cited by 309
Improving performance of deep learning models with axiomatic attribution priors and expected gradients
Erion, G., Janizek, J. D., Sturmfels, P., Lundberg, S. M., & Lee, S.-I. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00343-w · Cited by 300
Self-citation
Development and Validation of the Quick COVID-19 Severity Index: A Prognostic Tool for Early Clinical Decompensation
Haimovich, A. D., Ravindra, N. G., Stoytchev, S., Young, H. P., Wilson, F. P., van Dijk, D., Schulz, W. L., & Taylor, R. A. (2020) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2020.07.022 · Cited by 294
Revealing Drivers of Haze Pollution by Explainable Machine Learning
Hou, L., Dai, Q., Song, C., Liu, B., Guo, F., Dai, T., Li, L., Liu, B., Bi, X., Zhang, Y., et al. (2022) · Environmental Science & Technology Letters · doi:10.1021/acs.estlett.1c00865 · Cited by 304
DeepSTARR predicts enhancer activity from DNA sequence and enables the de novo design of enhancers
de Almeida, B. P., Reiter, F., Pagani, M., & Stark, A. (2021) · doi:10.1101/2021.10.05.463203 · Cited by 284
Counterfactuals and causability in explainable artificial intelligence: Theory, algorithms, and applications
Chou, Y.-L., Moreira, C., Bruza, P., Ouyang, C., & Jorge, J. (2022) · Information Fusion · doi:10.1016/j.inffus.2021.11.003 · Cited by 302
Explaining a series of models by propagating Shapley values
Chen, H., Lundberg, S., & Lee, S. (2022) · Nature Communications · doi:10.1038/s41467-022-31384-3 · Cited by 303
Self-citation
DeepSTARR predicts enhancer activity from DNA sequence and enables the de novo design of synthetic enhancers
de Almeida, B. P., Reiter, F., Pagani, M., & Stark, A. (2022) · Nature Genetics · doi:10.1038/s41588-022-01048-5 · Cited by 283
A microbiome-dependent gut–brain pathway regulates motivation for exercise
Dohnalová, L., Lundgren, P., Carty, J. R. E., Goldstein, N., Wenski, S. L., Nanudorn, P., Thiengmag, S., Huang, K.-P., Litichevskiy, L., Descamps, H. C., et al. (2022) · Nature · doi:10.1038/s41586-022-05525-z · Cited by 308
Nonlinear and synergistic effects of TOD on urban vibrancy: Applying local explanations for gradient boosting decision tree
Xiao, L., Lo, S., Liu, J., Zhou, J., & Li, Q. (2021) · Sustainable Cities and Society · doi:10.1016/j.scs.2021.103063 · Cited by 286
Predicting the efficiency of prime editing guide RNAs in human cells
Kim, H. K., Yu, G., Park, J., Min, S., Lee, S., Yoon, S., & Kim, S. (2020) · Nature Biotechnology · doi:10.1038/s41587-020-0677-y · Cited by 282
Self-citation
Advancements and challenges of digital twins in industry
Tao, F., Zhang, H., & Zhang, C. (2024) · Nature Computational Science · doi:10.1038/s43588-024-00603-w · Cited by 303
Interpretable machine learning with an ensemble of gradient boosting machines
Konstantinov, A. V., & Utkin, L. V. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.106993 · Cited by 299
Interpretable tree-based ensemble model for predicting beach water quality
Li, L., Qiao, J., Yu, G., Wang, L., Li, H.-Y., Liao, C., & Zhu, Z. (2022) · Water Research · doi:10.1016/j.watres.2022.118078 · Cited by 273
A Holistic Approach to the Design of Artificial Intelligence (AI) Education for K-12 Schools
Chiu, T. K. F. (2021) · TechTrends · doi:10.1007/s11528-021-00637-1 · Cited by 271
Unveiling the black box: A systematic review of Explainable Artificial Intelligence in medical image analysis
Muhammad, D., & Bendechache, M. (2024) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2024.08.005 · Cited by 325
Artificial intelligence-powered electronic skin
Xu, C., Solomon, S. A., & Gao, W. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00760-z · Cited by 259
Fine-scale modeling of the urban heat island: A comparison of multiple linear regression and random forest approaches
Oukawa, G. Y., Krecl, P., & Targino, A. C. (2022) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2021.152836 · Cited by 257
Machine learning models to accelerate the design of polymeric long-acting injectables
Bannigan, P., Bao, Z., Hickman, R. J., Aldeghi, M., Häse, F., Aspuru‐Guzik, A., & Allen, C. J. (2023) · Nature Communications · doi:10.1038/s41467-022-35343-w · Cited by 246
Machine learning enables interpretable discovery of innovative polymers for gas separation membranes
Yang, J., Tao, L., He, J., McCutcheon, J. R., & Li, Y. (2022) · Science Advances · doi:10.1126/sciadv.abn9545 · Cited by 251
Decision trees: from efficient prediction to responsible AI
Blockeel, H., Devos, L., Frénay‬, B., Nanfack, G., & Nijssen, S. (2023) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2023.1124553 · Cited by 301
Machine learning model to predict mental health crises from electronic health records
Garriga, R., Mas, J., Abraha, S., Nolan, J., Harrison, O., Tadros, G., & Matic, A. (2022) · Nature Medicine · doi:10.1038/s41591-022-01811-5 · Cited by 229
Predictive modeling of depression and anxiety using electronic health records and a novel machine learning approach with artificial intelligence
Nemesure, M. D., Heinz, M. V., Huang, R., & Jacobson, N. C. (2021) · Scientific Reports · doi:10.1038/s41598-021-81368-4 · Cited by 258
Spatial heterogeneity modeling of water quality based on random forest regression and model interpretation
Wang, F., Wang, Y., Zhang, K., Hu, M., Weng, Q., & Zhang, H. J. (2021) · Environmental Research · doi:10.1016/j.envres.2021.111660 · Cited by 237
Machine Learning to Predict Mortality and Critical Events in a Cohort of Patients With COVID-19 in New York City: Model Development and Validation
Vaid, A., Somani, S. S., Russak, A. J., Freitas, J. K. D., Chaudhry, F. F., Paranjpe, I., Johnson, K. W., Lee, S. J., Miotto, R., Zhao, S., et al. (2020) · Journal of Medical Internet Research · doi:10.2196/24018 · Cited by 238
Self-citation
Electricity price forecasting on the day-ahead market using machine learning
Tschora, L., Pierre, E., Plantevit, M., & Robardet, C. (2022) · Applied Energy · doi:10.1016/j.apenergy.2022.118752 · Cited by 249
A novel construction cost prediction model using hybrid natural and light gradient boosting
Chakraborty, D., Elhegazy, H. M., Elzarka, H. M., & Gutierrez, L. (2020) · Advanced Engineering Informatics · doi:10.1016/j.aei.2020.101201 · Cited by 227
Initial whole-genome sequencing and analysis of the host genetic contribution to COVID-19 severity and susceptibility
Wang, F., Huang, S., Gao, R., Zhou, Y., Lai, C., Li, Z., Xian, W., Qian, X., Li, Z., Huang, Y., et al. (2020) · Cell Discovery · doi:10.1038/s41421-020-00231-4 · Cited by 228
Resource–diversity relationships in bacterial communities reflect the network structure of microbial metabolism
Bello, M. D., Lee, H., Goyal, A., & Gore, J. (2021) · Nature Ecology & Evolution · doi:10.1038/s41559-021-01535-8 · Cited by 228
Dynamics of particle network in composite battery cathodes
Li, J., Sharma, N., Jiang, Z., Yang, Y., Monaco, F., Xu, Z., Hou, D., Ratner, D. F., Pianetta, P., Cloetens, P., et al. (2022) · Science · doi:10.1126/science.abm8962 · Cited by 223
Shapley value: from cooperative game to explainable artificial intelligence
Li, M., Sun, H., Huang, Y., & Chen, H. (2024) · Autonomous Intelligent Systems · doi:10.1007/s43684-023-00060-8 · Cited by 264
Self-citation
Explainable AI for clinical and remote health applications: A survey on tabular and time series data
Di Martino, F., & Delmastro, F. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-022-10304-3 · Cited by 216
Gut microbiota, inflammation, and molecular signatures of host response to infection
Gou, W., Fu, Y., Yue, L., Chen, G., Cai, X., Shuai, M., Xu, F., Yi, X., Chen, H., Zhu, Y., et al. (2021) · Journal of genetics and genomics/Journal of Genetics and Genomics · doi:10.1016/j.jgg.2021.04.002 · Cited by 220
Self-citation
Understanding Global Feature Contributions With Additive Importance MeasuresPreprint
Covert, I., Lundberg, S., & Lee, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.00668 · Cited by 256
Self-citation
Simple Behavioral Analysis (SimBA) – an open source toolkit for computer classification of complex social behaviors in experimental animalsPreprint
Nilsson, S. R. O., Goodwin, N. L., Choong, J. J., Hwang, S., Wright, H. R., Norville, Z. C., Tong, X. F., Lin, D., Bentzley, B. S., Eshel, N., et al. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.04.19.049452 · Cited by 233
General Pitfalls of Model-Agnostic Interpretation Methods for Machine Learning Models
Molnar, C., König, G., Herbinger, J., Freiesleben, T., Dandl, S., Scholbeck, C. A., Casalicchio, G., Grosse-Wentrup, M., & Bischl, B. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-04083-2_4 · Cited by 211
Prediction of the sequence-specific cleavage activity of Cas9 variants
Kim, N., Kim, H. K., Lee, S., Seo, J. H., Choi, J. W., Park, J., Min, S., Yoon, S., Cho, S.-R., & Kim, H. H. (2020) · Nature Biotechnology · doi:10.1038/s41587-020-0537-9 · Cited by 198
Self-citation
Explainable AI in Clinical Decision Support Systems: A Meta-Analysis of Methods, Applications, and Usability Challenges
Abbas, Q., Jeong, W., & Lee, S. W. (2025) · Healthcare · doi:10.3390/healthcare13172154 · Cited by 236
“In contrast, model-specific approaches such as Grad-CAM, attention mechanisms, and Integrated Gradients are tailored to deep neural network architectures and require access to gradients or attention weights [106].”
InfluentialSelf-citation
A Review on Explainable Artificial Intelligence for Healthcare: Why, How, and When?
Bharati, S., Mondal, M. R. H., & Podder, P. (2023) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2023.3266418 · Cited by 251
Understanding deep learning in land use classification based on Sentinel-2 time series
Campos‐Taberner, M., García-Haro, F. J., Martínez, B., Izquierdo‐Verdiguier, E., Atzberger, C., Camps‐Valls, G., & Navarro, M. A. G. (2020) · Scientific Reports · doi:10.1038/s41598-020-74215-5 · Cited by 230
Using Eye-Tracking Data to Predict Situation Awareness in Real Time During Takeover Transitions in Conditionally Automated Driving
Zhou, F., Yang, X. J., & de Winter, J. (2021) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3069776 · Cited by 206
An interpretable model for the susceptibility of rainfall-induced shallow landslides based on SHAP and XGBoost
Zhou, X., Wen, H., Li, Z., Zhang, H., & Zhang, W. (2022) · Geocarto International · doi:10.1080/10106049.2022.2076928 · Cited by 204
Post-hoc vs ante-hoc explanations: xAI design guidelines for data scientists
Retzlaff, C. O., Angerschmid, A., Saranti, A., Schneeberger, D. M., Röttger, R., Müller, H., & Holzinger, A. (2024) · Cognitive Systems Research · doi:10.1016/j.cogsys.2024.101243 · Cited by 229
Simple Behavioral Analysis (SimBA) as a platform for explainable machine learning in behavioral neuroscience
Goodwin, N. L., Choong, J. J., Hwang, S., Pitts, K. M., Bloom, L., Islam, A., Zhang, Y. Y., Szelenyi, E. R., Tong, X., Newman, E. L., et al. (2024) · Nature Neuroscience · doi:10.1038/s41593-024-01649-9 · Cited by 211
Creativity and artificial intelligence—a student perspective
Marrone, R., Taddeo, V., & Hill, G. (2022) · Journal of Intelligence · doi:10.3390/jintelligence10030065 · Cited by 208
Guidelines and evaluation of clinical explainable AI in medical image analysis
Jin, W., Li, X., Fatehi, M., & Hamarneh, G. (2023) · Medical Image Analysis · doi:10.1016/j.media.2022.102684 · Cited by 199
Machine Learning Solutions for Osteoporosis—A Review
Smets, J., Shevroja, E., Hügle, T., Leslie, W. D., & Hans, D. B. (2020) · Journal of Bone and Mineral Research · doi:10.1002/jbmr.4292 · Cited by 207
Operational Research and Artificial Intelligence Methods in Banking
Doumpos, M., Zopounidis, C., Gounopoulos, D., Platanakis, E., & Zhang, W. (2022) · European Journal of Operational Research · doi:10.1016/j.ejor.2022.04.027 · Cited by 256
Prevalence and drivers of abrupt vegetation shifts in global drylands
Berdugo, M., Gaitán, J., Delgado‐Baquerizo, M., Crowther, T. W., & Dakos, V. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2123393119 · Cited by 199
Integration of machine learning and genome-scale metabolic modeling identifies multi-omics biomarkers for radiation resistance
Lewis, J. E., & Kemp, M. L. (2021) · Nature Communications · doi:10.1038/s41467-021-22989-1 · Cited by 187
A novel feature engineered-CatBoost-based supervised machine learning framework for electricity theft detection
Hussain, S., Mustafa, M. W., Jumani, T. A., Baloch, S. K., Alotaibi, H., Khan, I., & Khan, A. (2021) · Energy Reports · doi:10.1016/j.egyr.2021.07.008 · Cited by 190
Interpretable machine learning for imbalanced credit scoring datasets
Chen, Y., Calabrese, R., & Martin-Barragan, B. (2024) · European Journal of Operational Research · doi:10.1016/j.ejor.2023.06.036 · Cited by 190
Interpretable deep learning for automatic diagnosis of 12-lead electrocardiogram
Zhang, D., Yang, S., Yuan, X., & Zhang, P. (2021) · iScience · doi:10.1016/j.isci.2021.102373 · Cited by 201
Recent Applications of Explainable AI (XAI): A Systematic Literature Review
Saarela, M., & Podgorelec, V. (2024) · Applied Sciences · doi:10.3390/app14198884 · Cited by 247
COVID-19 mortality risk assessment: An international multi-center study
Bertsimas, D., Lukin, G., Mingardi, L., Nohadani, O., Orfanoudaki, A., Stellato, B., Wiberg, H., González-García, S., Parra-Calderón, C. L., Robinson, K. J., et al. (2020) · PLoS ONE · doi:10.1371/journal.pone.0243262 · Cited by 187
An analytic framework using deep learning for prediction of traffic accident injury severity based on contributing factors
Ma, Z., Mei, G., & Cuomo, S. (2021) · Accident Analysis & Prevention · doi:10.1016/j.aap.2021.106322 · Cited by 201
Increased probability of hot and dry weather extremes during the growing season threatens global crop yields
Heino, M., Kinnunen, P., Anderson, W., Ray, D. K., Puma, M. J., Varis, O., Siebert, S., & Kummu, M. (2023) · Scientific Reports · doi:10.1038/s41598-023-29378-2 · Cited by 213
XGBoost-SHAP-based interpretable diagnostic framework for alzheimer’s disease
Yi, F., Yang, H., Chen, D., Qin, Y., Han, H., Cui, J., Bai, W., Ma, Y., Zhang, R., & Yu, H. (2023) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-023-02238-9 · Cited by 205
An interpretable probabilistic model for short-term solar power forecasting using natural gradient boosting
Mitrentsis, G., & Lens, H. (2022) · Applied Energy · doi:10.1016/j.apenergy.2021.118473 · Cited by 192
Comprehensive comparative study of multi-label classification methods
Bogatinovski, J., Todorovski, L., Džeroski, S., & Kocev, D. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.117215 · Cited by 219
How Interpretable Machine Learning Can Benefit Process Understanding in the Geosciences
Jiang, S., Sweet, L., Blougouras, G., Brenning, A., Li, W., Reichstein, M., Denzler, J., Shangguan, W., Yu, G., Huang, F., et al. (2024) · Earth s Future · doi:10.1029/2024ef004540 · Cited by 209

Authors: Lundberg, S. M., Erion, G., Chen, H., DeGrave, A., Prutkin, J. M., Nair, B., Katz, R., Himmelfarb, J., Bansal, N., & Lee, S.-I.
Affiliations: Microsoft (United States) (US)
University of Washington (US)
Corresponding author: Lee, Su-In
ORCID: Lundberg, Scott M.
Erion, Gabriel
Chen, Hugh
DeGrave, Alex
Prutkin, Jordan M.
Nair, Bala
Katz, Ronit
Himmelfarb, Jonathan
Bansal, Nisha
Lee, Su-In
Venue: Nature Machine Intelligence
Publisher: Springer Science and Business Media LLC
Published: 2020-01-17
Volume: 2
Issue: 1
Pages: 56-67
ISSN: 2522-5839 (ISSN-L)
DOI: 10.1038/s42256-019-0138-9
PMC: PMC7326367
OpenAlex: W2999615587
Cited by: 10,551
References: 38
Citations by year: 2026: 3607 · 2025: 2330 · 2024: 1650 · 2023: 1197 · 2022: 959 · 2021: 614 · 2020: 187
FWCI: 291.51
Type: journal article
Language: en
Open access: green
www.ncbi.nlm.nih.gov
Licence: www.springernature.com/gp/researchers/text-and-data-mining (tdm)
www.springernature.com/gp/researchers/text-and-data-mining (vor)
Locations: Nature Machine Intelligence (publishedVersion)
PubMed (publishedVersion)
PubMed Central (acceptedVersion)
Funders: National Science Foundation (DGE-1256082, DBI-1759487, DBI-1552309, DBI-1355899)
American Cancer Society (127332-RSG-15-097-01-TBG)
U.S. Department of Health & Human Services | National Institutes of Health (R35 GM 128638, R01 NIA AG 061132)
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
Physical Sciences › Computer Science › Information Systems › Data Mining Algorithms and Applications
Keywords: explainable AI, tree-based models, local explanations, mortality risk, chronic kidney disease, model monitoring, machine learning in medicine

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

From local explanations to global understanding with explainable AI for trees

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks