The DOI matches its Crossref record; 45 of the 45 references it cites were found in a registry or an index.
From local explanations to global understanding with explainable AI for trees
Open accessAbstract
Tree-based machine learning models such as random forests, decision trees, and gradient boosted trees are popular non-linear predictive models, yet comparatively little attention has been paid to explaining their predictions. Here, we improve the interpretability of tree-based models through three main contributions: 1) The first polynomial time algorithm to compute optimal explanations based on game theory. 2) A new type of explanation that directly measures local feature interaction effects. 3) A new set of tools for understanding global model structure based on combining many local explanations of each prediction. We apply these tools to three medical machine learning problems and show how combining many high-quality local explanations allows us to represent global structure while retaining local faithfulness to the original model. These tools enable us to i) identify high magnitude but low frequency non-linear mortality risk factors in the US population, ii) highlight distinct population sub-groups with shared risk characteristics, iii) identify non-linear interaction effects among risk factors for chronic kidney disease, and iv) monitor a machine learning model deployed in a hospital by identifying which features are degrading the model's performance over time. Given the popularity of tree-based machine learning models, these improvements to their interpretability have implications across a broad set of domains.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“SHAP [6] popularized this for machine learning feature importance, with extensions to tree models [5].”
From AgentSHAP: Interpreting LLM Agent Tool Importance with Monte Carlo Shapley Value Estimation (2025) · cites this paper as [5]
In the citing paper
“The algorithm provides interpretable explanations suitable for regression and classification of models with tree structure applied to tabular data [22,39].”
From Explainable AI (XAI): Core Ideas, Techniques, and Solutions (2023) · cites this paper as [39]
In the citing paper
“ML and AI techniques, such as social network analysis and agent-based modeling, can model the complex interactions between individuals and their social environments to identify key leverage points for interventions that promote health and well-being across communities [40, 47].”
From Integrating machine learning and artificial intelligence in life-course epidemiology: pathways to innovative public health solutions (2024) · cites this paper as [40]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
XGBoost: A Scalable Tree Boosting System
Chen, T., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939785 · Cited by 54,569
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707
Self-citation"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Gauriat, C.-M., Pencolé, Y., Ribot, P., et al. (2024) · HAL · doi:10.4230/oasics.dx.2024.27 · Cited by 12,623
Shapley, L. (1988) · Cambridge University Press eBooks · doi:10.1017/cbo9780511528446.003 · Cited by 10,698
Heart Disease and Stroke Statistics—2016 Update: A Report From the American Heart Association
Mozaffarian, D., Benjamin, E. J., Go, A. S., Arnett, D. K., Blaha, M. J., Cushman, M., Das, S. R., de Ferranti, S., Després, J.-P., Fullerton, H. J., et al. (2016) · Circulation · doi:10.1161/cir.0000000000000350 · Cited by 6,596
A Randomized Trial of Intensive versus Standard Blood-Pressure Control
The SPRINT Research Group (2015) · New England Journal of Medicine · doi:10.1056/nejmoa1511939 · Cited by 6,325
Heart Disease and Stroke Statistics—2015 Update: A Report From the American Heart Association
Mozaffarian, D., Benjamin, E. J., Go, A. S., Arnett, D. K., Blaha, M. J., Cushman, M., de Ferranti, S., Després, J.-P., Fullerton, H. J., Howard, V. J., et al. (2015) · Circulation · doi:10.1161/cir.0000000000000152 · Cited by 5,797
Repeated observation of breast tumor subtypes in independent gene expression data sets
Sørlie, T., Tibshirani, R. J., Parker, J. S., Hastie, T. J., Marron, J. S., Nobel, A. B., Deng, S., Johnsen, H., Pesich, R., Geisler, S. B., et al. (2003) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0932692100 · Cited by 5,508
Cited by
Sorted by citations
Items 1,001–1,032 of 1,032
Zhu, H., Zhang, P., Wang, N., Zhang, F., Ma, W., Wen, F., Li, M., Wang, Y., Fan, X., Hou, K., et al. (2024) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2024.142306 · Cited by 37
Park, S., Zhang, T., & Kang, S. (2023) · International Journal of Molecular Sciences · doi:10.3390/ijms24119533 · Cited by 35
Rodrigo, H., Beukes, E. W., Andersson, G., & Manchaiah, V. (2021) · doi:10.2196/preprints.28999 · Cited by 33
Wang, H., Zhao, Q.-Y., Luo, J.-C., Liu, K., Yu, S.-J., Ma, J.-F., Luo, M.-H., Hao, G.-W., Su, Y., Zhang, Y.-J., et al. (2022) · BMC Pulmonary Medicine · doi:10.1186/s12890-022-02096-7 · Cited by 32
Powell, W., Song, X., Mohamed, Y., Walsh, D., Parks, E. J., McMahon, T. M., Khan, M., & Waitman, L. R. (2023) · Obesity · doi:10.1002/oby.23859 · Cited by 32
Chmiel, F. P., Burns, D. K., Azor, M., Borca, F., Boniface, M. J., Zlatev, Z. D., White, N. M., Daniels, T. W. V., & Kiuber, M. (2020) · doi:10.1101/2020.12.02.20239194 · Cited by 30
Resource Reservation in Sliced Networks: An Explainable Artificial Intelligence (XAI) Approach
Barnard, P., Macaluso, I., Marchetti, N., & DaSilva, L. A. (2022) · ICC 2022 - IEEE International Conference on Communications · doi:10.1109/icc45855.2022.9838766 · Cited by 29
Interpretable video-based tracking and quantification of Parkinsonism clinical motor states
Deng, D., Ostrem, J. L., Nguyen, V., Cummins, D. D., Sun, J., Pathak, A., Little, S., & Abbasi-Asl, R. (2024) · npj Parkinson's Disease · doi:10.1038/s41531-024-00742-x · Cited by 25
Children's Mental Models of AI Reasoning: Implications for AI Literacy Education
Dangol, A., Wolfe, R., Zhao, R., Kim, J., Ramanan, T., Davis, K., & Kientz, J. A. (2025) · Proceedings of the 24th Interaction Design and Children · doi:10.1145/3713043.3728856 · Cited by 25
Regulating the safety of health–related artificial intelligence.
Silva, M., Flood, C., Goldenberg, A., & Singh, D. (2022) · Healthcare Policy | Politiques de Santé · doi:10.12927/hcpol.2022.26824 · Cited by 23
Factors Associated With Semaglutide Initiation Among Adults With Obesity
Podolsky, M. I., Raquib, R., Shafer, P. R., Hempstead, K., Ellis, R. P., & Stokes, A. C. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.55222 · Cited by 26
Coupland, H., Scheidwasser, N., Katsiferis, A., Davies, M., Flaxman, S., Hulvej Rod, N., Mishra, S., Bhatt, S., & Unwin, H. J. T. (2025) · BMC Public Health · doi:10.1186/s12889-025-22705-4 · Cited by 26
Rinderknecht, M. D., & Klopfenstein, Y. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00482-9 · Cited by 23
Predicting self-intercepted medication ordering errors using machine learning
King, C. R., Abraham, J., Fritz, B. A., Cui, Z., Galanter, W., Chen, Y., & Kannampallil, T. (2021) · PLOS ONE · doi:10.1371/journal.pone.0254358 · Cited by 22
Fox, S., & Rey, V. F. (2024) · Machine Learning and Knowledge Extraction · doi:10.3390/make6030071 · Cited by 22
“Application of (SHAP) values can involve a wide variety of diagrams, including bar charts that have both positive and negative SHAP values for multiple variables represented by horizontal bars, which can be colored to indicate the extent to which each variable has a high or low feature value [74].”
BackgroundDeep learning and CRISPR-Cas13d ortholog discovery for optimized RNA targeting
Wei, J., Lotfy, P., Faizi, K., Baungaard, S., Gibson, E., Wang, E., Slabodkin, H., Kinnaman, E., Chandrasekaran, S., Kitano, H., et al. (2021) · doi:10.1101/2021.09.14.460134 · Cited by 67
Explainable AI-Aided Feature Selection and Model Reduction for DRL-Based V2X Resource Allocation
Khan, N., Abdallah, A., Celik, A., Eltawil, A. M., & Coleri, S. (2025) · IEEE Transactions on Communications · doi:10.1109/tcomm.2025.3554655 · Cited by 20
Aguirre, U., & Urrechaga, E. (2023) · Clinical Chemistry and Laboratory Medicine (CCLM) · doi:10.1515/cclm-2022-0713 · Cited by 17
Explainable AI in Cybersecurity Operations: Lessons Learned from xAI Tool Deployment
Nyre-Yu, M., Morris, E., Smith, M., Moss, B., & Smutz, C. (2022) · Proceedings 2022 Symposium on Usable Security · doi:10.14722/usec.2022.23014 · Cited by 15
An Empirical Comparison of Interpretable Models to Post-Hoc Explanations
Mahya, P., & Fürnkranz, J. (2023) · AI · doi:10.3390/ai4020023 · Cited by 17
“Recently, several approaches have been investigated which try to construct global models from local explanations (e.g., [4,5]).”
BackgroundAntoniadi, A. M., Galvin, M., Heverin, M., Hardiman, O., & Mooney, C. (2021) · Proceedings of the 36th Annual ACM Symposium on Applied Computing · doi:10.1145/3412841.3441940 · Cited by 12
Interpretable AI/ML for High-stakes Tasks with Human-in-the-loop: Critical Review and Future Trends
Kovalerchuk, B. (2024) · doi:10.21203/rs.3.rs-3989807/v1 · Cited by 9
Rodrigo, H., Beukes, E. W., Andersson, G., & Manchaiah, V. (2022) · American Journal of Audiology · doi:10.1044/2022_aja-21-00270 · Cited by 9
Kiefer, S. M., Lerch, S., Ludwig, P., & Pinto, J. G. (2023) · Artificial Intelligence for the Earth Systems · doi:10.1175/aies-d-23-0020.1 · Cited by 9
Study on the Helpfulness of Explainable Artificial Intelligence
Labarta, T., Kulicheva, E., Froelian, R., Geißler, C., Melman, X., & von Klitzing, J. (2024) · Communications in Computer and Information Science · doi:10.1007/978-3-031-63803-9_16 · Cited by 8
“…Layer-wise relevance propagation ( LRP ) [2,31], Gradient class activation mapping ( GradCAM ) [7,10], Local In-terpretable Model-Agnostic Explanations ( LIME ) [15], SHapley Additive exPla-nations ( SHAP ) [28,29,37,40], Integrated Gradients ( Integrated Gradients ) [41] and Confidence Scores .”
BackgroundA damage accumulation model identifies distinct aging regimes across species
Raz, N., Yang, Y., Pridham, G., Shenhar, B., Frenkel, S., Levy, T., Tlusty, T., Mayo, A., & Alon, U. (2026) · Nature Aging · doi:10.1038/s43587-026-01138-7 · Cited by 7
Celepli, S., Bigat, I., Karakas, B., Tezcan, H. M., Yar, M. D., Celepli, P., Aksahin, M. F., Hancerliogullari, O., Yilmaz, Y. F., & Erogul, O. (2025) · Diagnostics · doi:10.3390/diagnostics15162065 · Cited by 6
Camastra, C., Pelagi, A., Quattrone, A., & Sarica, A. (2025) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-82487-6_17 · Cited by 1
StoryScope: Investigating idiosyncrasies in AI fiction
Russell, J., Rajendhran, R., Pham, C. M., Iyyer, M., & Wieting, J. (2026) · arXiv · doi:10.48550/arxiv.2604.03136 · Cited by 13
“XGBoost consistently outperformed the alternatives on both binary and multiclass tasks, and integrates cleanly with SHAP-based interpretability analyses (Lundberg & Lee, 2017; Lundberg et al., 2020), so we adopt it as the default classifier throughout.”
AgentSHAP: Interpreting LLM Agent Tool Importance with Monte Carlo Shapley Value Estimation
Horovicz, M. (2025) · arXiv · doi:10.48550/arxiv.2512.12597
“SHAP [6] popularized this for machine learning feature importance, with extensions to tree models [5].”
Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey
Das, A., & Rad, P. (2020) · arXiv · doi:10.48550/arxiv.2006.11371 · Cited by 504
Human-Centered Explainable AI (XAI): From Algorithms to User Experiences
Liao, Q. V., & Varshney, K. R. (2021) · arXiv · doi:10.48550/arxiv.2110.10790 · Cited by 120
Metadata
- Authors
- Lundberg, S. M., Erion, G., Chen, H., DeGrave, A., Prutkin, J. M., Nair, B., Katz, R., Himmelfarb, J., Bansal, N., & Lee, S.-I.
- Affiliations
- Microsoft (United States) (US)
- University of Washington (US)
- Corresponding author
- Lee, Su-In
- ORCID
- Venue
- Nature Machine Intelligence
- Published
- 2020-01-17
- Volume
- 2
- Issue
- 1
- Pages
- 56-67
- ISSN
- 2522-5839 (ISSN-L)
- PMC
- PMC7326367
- OpenAlex
- W2999615587
- Cited by
- 10,551
- References
- 38
- Citations by year
- 2026: 3607 · 2025: 2330 · 2024: 1650 · 2023: 1197 · 2022: 959 · 2021: 614 · 2020: 187
- FWCI
- 291.51
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Licence
- Locations
- Nature Machine Intelligence (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Science Foundation (DGE-1256082, DBI-1759487, DBI-1552309, DBI-1355899)
- American Cancer Society (127332-RSG-15-097-01-TBG)
- U.S. Department of Health & Human Services | National Institutes of Health (R35 GM 128638, R01 NIA AG 061132)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Physical Sciences › Computer Science › Information Systems › Data Mining Algorithms and Applications
- Keywords
- explainable AI, tree-based models, local explanations, mortality risk, chronic kidney disease, model monitoring, machine learning in medicine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
