iForest: Interpreting Random Forests via Visual Analytics
The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.
Abstract
As an ensemble model that consists of many independent decision trees, random forests generate predictions by feeding the input to internal trees and summarizing their outputs. The ensemble nature of the model helps random forests outperform any individual decision tree. However, it also leads to a poor model interpretability, which significantly hinders the model from being used in fields that require transparent and explainable predictions, such as medical diagnosis and financial fraud detection. The interpretation challenges stem from the variety and complexity of the contained decision trees. Each decision tree has its unique structure and properties, such as the features used in the tree and the feature threshold in each tree node. Thus, a data input may lead to a variety of decision paths. To understand how a final prediction is achieved, it is desired to understand and compare all decision paths in the context of all tree structures, which is a huge challenge for any users. In this paper, we propose a visual analytic system aiming at interpreting random forest models and predictions. In addition to providing users with all the tree information, we summarize the decision paths in random forests, which eventually reflects the working mechanism of the model and reduces users' mental burden of interpretation. To demonstrate the effectiveness of our system, two usage scenarios and a qualitative user study are conducted.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538 · cited 6×
“The feature importance is often evaluated based on either the Mean Decrease Accuracy (MDA) [10] or the Mean Decrease Impurity (MDI) [13].”
BackgroundInfluentialGreedy function approximation: A gradient boosting machine
Friedman, J. H. (2001) · The Annals of Statistics · doi:10.1214/aos/1013203451 · Cited by 30,731 · cited 2×
“Apart from feature importance, another powerful tool for illustrating the relationships between features and predictions is partial dependence plots (PDPs) [21, 22], which depict how feature value changes affect predictions.”
BackgroundMethodThe Elements of Statistical Learning
Hastie, T., Friedman, J., & Tibshirani, R. (2001) · Springer Series in Statistics · doi:10.1007/978-0-387-21606-5 · Cited by 24,162
An introduction to ROC analysis
Fawcett, T. (2006) · Pattern Recognition Letters · doi:10.1016/j.patrec.2005.10.010 · Cited by 21,879 · cited 1×
“From the evaluation perspective, we can further enable users to switch between different model evaluation measures, such as ROC-AUC [19] or training loss [46].”
Method"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Newell, A. (1972) · Cited by 6,222
Lipton, Z. C. (2018) · Queue · doi:10.1145/3236386.3241340 · Cited by 5,427 · cited 6×
“To make random forests transparent, users first need to understand what the model has learned in general and be able to evaluate the model’s predictions [34].”
BackgroundMethodInfluentialStatistical Modeling: The Two Cultures (with comments and a rejoinder by the author)
Breiman, L. (2001) · Statistical Science · doi:10.1214/ss/1009213726 · Cited by 4,340
Kolodner, J. L. (2005) · The Cambridge Handbook of the Learning Sciences · doi:10.1017/cbo9780511816833.015 · Cited by 4,247
(2013) · Information Visualization · doi:10.1016/b978-0-12-381464-7.00018-1 · Cited by 3,742
Gene selection and classification of microarray data using random forest
Díaz-Uriarte, R., & Alvarez de Andrés, S. (2006) · BMC Bioinformatics · doi:10.1186/1471-2105-7-3 · Cited by 2,929 · cited 1×
“Random forests have been extensively studied in the machine learning and data mining areas for many years [10] and proved to perform well in many domains, such as biomedical engineering [17] and traffic planning [25].”
BackgroundVariable selection using random forests
Genuer, R., Poggi, J.-M., & Tuleau-Malot, C. (2010) · Pattern Recognition Letters · doi:10.1016/j.patrec.2010.03.014 · Cited by 2,619
Graph-based approach for airborne light detection and ranging segmentation
Vilariño, D. L., Cabaleiro, J. C., Martínez, J., Rivera, F. F., & Pena, T. F. (2017) · Journal of Applied Remote Sensing · doi:10.1117/1.jrs.11.015020 · Cited by 1,721
Visualization Analysis and Design
Munzner, T. (2014) · A K Peters/CRC Press eBooks · doi:10.1201/b17511 · Cited by 1,697
Intelligible Models for HealthCare: Predicting Pneumonia Risk and Hospital 30-day Readmission
Caruana, R., Lou, Y., Gehrke, J., Koch, P., Sturm, M., & Elhadad, N. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2788613 · Cited by 1,685 · cited 3×
“Based on the idea of solving problems by analogy [34], case-based reasoning facilitates users to determine their confidence in the prediction of a given data input by providing similar data examples as references [15, 28].”
BackgroundUnderstanding variable importances in forests of randomized trees
Louppe, G., Wehenkel, L. A., Sutera, A., & Geurts, P. (2013) · ORBi (University of Liège) · Cited by 1,364
Tree visualization with tree-maps: 2-d space-filling approach
Shneiderman, B. (1992) · ACM Transactions on Graphics · doi:10.1145/102377.115768 · Cited by 1,328 · cited 2×
“Decision trees can be visualized using different visualization techniques, such as node-link diagram [41, 56, 59], icicle plot [6, 36, 54], and Treemap [38, 49].”
MethodStatistical modeling: The two cultures
Breiman, L. (2001) · Cited by 1,314
How Many Trees in a Random Forest?
Oshiro, T. M., Perez, P. S., & Baranauskas, J. A. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-31537-4_13 · Cited by 912
Case-based reasoning: A review
Watson, I., & Marir, F. (1994) · The Knowledge Engineering Review · doi:10.1017/s0269888900007098 · Cited by 680 · cited 1×
“Popular case-based reasoning methods used in general machine learning models are usually based on model proximity [14] or feature similarity [57].”
MethodA Survey of Methods for Explaining Black Box Models
Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., & Pedreschi, D. (2019) · ACM Computing Surveys · doi:10.1145/3236009 · Cited by 629 · cited 1×
“These structures can provide deeper insights into the underlying working mechanisms of random forest (G2) [26, 34].”
BackgroundInteracting with Predictions: Visual Inspection of Black-box Machine Learning Models
Krause, J., Perer, A., & Ng, K. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858529 · Cited by 390 · cited 2×
“[30] recently introduces another heatmap-based design to represent PDPs, in which the color encodes prediction probabilities.”
BackgroundThe Bayesian Case Model: A Generative Approach for Case-Based Reasoning and Prototype Classification
Kim, B., Rudin, C. D., & Shah, J. (2014) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 340
Icicle Plots: Better Displays for Hierarchical Clustering
Kruskal, J. B., & Landwehr, J. M. (1983) · The American Statistician · doi:10.1080/00031305.1983.10482733 · Cited by 264 · cited 1×
“A more space-efficient way to visualize decision trees is icicle plots [31], in which the nodes are encoded by multiple stacked bars.”
Interpreting tree ensembles with inTrees
Deng, H. (2019) · International Journal of Data Science and Analytics · doi:10.1007/s41060-018-0144-8 · Cited by 253 · cited 2×
“Surrogate models are often simple and interpretable, such as decision trees [48, 60], decision rules [16, 23], and decision sets [32].”
BackgroundCase-based reasoning in the health sciences: What's next?
Bichindaritz, I., & Marling, C. (2006) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2005.10.008 · Cited by 218 · cited 1×
“This aligns with the analogical decision making process of humans [29, 40], which has been widely adopted in various real-world applications, such as the medical [8] and the financial [33] domains.”
MethodInteractive machine learning: letting users build classifiers
WARE, M., FRANK, E., HOLMES, G., HALL, M., & WITTEN, I. H. (2001) · International Journal of Human-Computer Studies · doi:10.1006/ijhc.2001.0499 · Cited by 207 · cited 1×
“Decision trees can be visualized using different visualization techniques, such as node-link diagram [41, 56, 59], icicle plot [6, 36, 54], and Treemap [38, 49].”
MethodLearning Driver Behavior Models from Traffic Observations for Decision Making and Planning
Gindele, T., Brechtel, S., & Dillmann, R. (2015) · IEEE Intelligent Transportation Systems Magazine · doi:10.1109/mits.2014.2357038 · Cited by 199 · cited 1×
“Random forests have been extensively studied in the machine learning and data mining areas for many years [10] and proved to perform well in many domains, such as biomedical engineering [17] and traffic planning [25].”
BackgroundTowards an effective cooperation of the user and the computer for classification
Ankerst, M., Ester, M., & Kriegel, H. (2000) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/347090.347124 · Cited by 180 · cited 1×
“[7] adopt a pixel-based design that uses pixels to represent data items and pixel colors to represent the data labels.”
MethodInterpreting Random Forest Classification Models Using a Feature Contribution Method
Palczewska, A., Palczewski, J., Robinson, R. M., & Neagu, D. C. (2014) · Advances in intelligent systems and computing · doi:10.1007/978-3-319-04717-1_9 · Cited by 176
BaobabView: Interactive construction and analysis of decision trees
van den Elzen, S., & van Wijk, J. J. (2011) · 2011 IEEE Conference on Visual Analytics Science and Technology (VAST) · doi:10.1109/vast.2011.6102453 · Cited by 172 · cited 1×
“In addition to tree structures, BaobabView [53] further encodes more information of models, such as the split point in each node and the training data volume that passes through each branch.”
BackgroundRanking-order case-based reasoning for financial distress prediction
Li, H., & Sun, J. (2008) · Knowledge-Based Systems · doi:10.1016/j.knosys.2008.03.047 · Cited by 161 · cited 1×
“This aligns with the analogical decision making process of humans [29, 40], which has been widely adopted in various real-world applications, such as the medical [8] and the financial [33] domains.”
Global refinement of random forest
Ren, S., Cao, X., Wei, Y., & Sun, J. Z. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298672 · Cited by 126 · cited 1×
“From the evaluation perspective, we can further enable users to switch between different model evaluation measures, such as ROC-AUC [19] or training loss [46].”
MethodThe Bayesian Case Model: A Generative Approach for Case-Based Reasoning and Prototype Classification
Kim, B., Rudin, C. D., & Shah, J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1503.01161 · Cited by 117 · cited 1×
“Based on the idea of solving problems by analogy [34], case-based reasoning facilitates users to determine their confidence in the prediction of a given data input by providing similar data examples as references [15, 28].”
Visual Diagnosis of Tree Boosting Methods
Liu, S., Xiao, J., Liu, J., Wang, X., Wu, J., & Zhu, J. (2017) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2017.2744378 · Cited by 111 · cited 1×
“[35] propose an interactive visual diagnosis tool to analyze the performance of boosting trees, which are a tree ensemble model similar to random forests.”
MethodCase-based explanation of non-case-based learning methods.
Caruana, R., Kangarloo, H., Dionisio, J. D. N., Sinha, U., & Johnson, D. (1999) · PubMed · Cited by 94
(2017) · arXiv (Cornell University) · Cited by 86
Human-Centric Justification of Machine Learning Predictions
Biran, O., & McKeown, K. (2017) · Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/202 · Cited by 84 · cited 1×
“Users should be able to audit the decision process of a prediction and make sure they agree before making a decision [9].”
BackgroundConfident Interpretation of Bayesian Decision Tree Ensembles for Clinical Applications
Schetinin, V., Fieldsend, J. E., Partridge, D., Coats, T. J., Krzanowski, W. J., Everson, R. M., Bailey, T. C., & Hernandez, A. (2007) · IEEE Transactions on Information Technology in Biomedicine · doi:10.1109/titb.2006.880553 · Cited by 64 · cited 2×
“Surrogate models are often simple and interpretable, such as decision trees [48, 60], decision rules [16, 23], and decision sets [32].”
BackgroundMethodTreePOD: Sensitivity-Aware Selection of Pareto-Optimal Decision Trees
Mühlbacher, T., Linhardt, L., Möller, T. B., & Piringer, H. (2017) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2017.2745158 · Cited by 61 · cited 3×
“Decision trees can be visualized using different visualization techniques, such as node-link diagram [41, 56, 59], icicle plot [6, 36, 54], and Treemap [38, 49].”
MethodSocial class and survival on the S.S. Titanic
Hall, W. (1986) · Social Science & Medicine · doi:10.1016/0277-9536(86)90041-9 · Cited by 43 · cited 1×
“This suggests that the model’s learned pattern aligns with the fact that female passengers indeed are more likely to survive in the Titanic shipwreck [27].”
BackgroundShort-term storage capacity for visual objects depends on expertise
Sørensen, T. A., & Kyllingsbæk, S. (2012) · Acta Psychologica · doi:10.1016/j.actpsy.2012.04.004 · Cited by 40 · cited 1×
“We support to visualize six to ten features in the same window depending on the display size, which is usually sufficient since previous research indicates that human has a limited visual capacity that around three to seven objects [50].”
MethodInterpreting Models via Single Tree Approximation
Zhou, Y., & Hooker, G. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.09036 · Cited by 39 · cited 1×
“Surrogate models are often simple and interpretable, such as decision trees [48, 60], decision rules [16, 23], and decision sets [32].”
BackgroundDesign and evaluation of visualization support to facilitate decision trees classification
Liu, Y., & Salvendy, G. (2006) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2006.07.005 · Cited by 39 · cited 1×
“Decision trees can be visualized using different visualization techniques, such as node-link diagram [41, 56, 59], icicle plot [6, 36, 54], and Treemap [38, 49].”
MethodTree Space Prototypes: Another Look at Making Tree Ensembles Interpretable
Tan, S., Soloviev, M., Hooker, G., & Wells, M. T. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.07115 · Cited by 36
Carrilio, T. (2006) · The Social Policy Journal · doi:10.1300/j185v04n03_01 · Cited by 34
Interpretable & Explorable Approximations of Black Box Models
Lakkaraju, H., Kamar, E., Caruana, R., & Leskovec, J. (2017) · arXiv · doi:10.48550/arxiv.1707.01154 · Cited by 23 · cited 1×
“Surrogate models are often simple and interpretable, such as decision trees [48, 60], decision rules [16, 23], and decision sets [32].”
BackgroundVisualizing Random Forest with Self-Organising Map
Płoński, P., & Zaremba, K. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-07176-3_6 · Cited by 19 · cited 1×
“[45] use Self-Organizing Map (SOM) instead of MDS to achieve higher accuracy.”
MethodVisualizing Random Forest with Self-Organising Map
Płoński, P., & Zaremba, K. (n.d.) · Cited by 17
Concept tree based clustering visualization with shaded similarity matrices
Wang, J., Yu, B., & Gasser, L. (2003) · Proceedings · doi:10.1109/icdm.2002.1184032 · Cited by 10 · cited 1×
“Decision trees can be visualized using different visualization techniques, such as node-link diagram [41, 56, 59], icicle plot [6, 36, 54], and Treemap [38, 49].”
MethodGallego-Ortiz, C., & Martel, A. L. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.08288 · Cited by 9 · cited 1×
“Surrogate models are often simple and interpretable, such as decision trees [48, 60], decision rules [16, 23], and decision sets [32].”
Background(2014) · doi:10.5555/2627435.2697065 · cited 1×
“One exhaustive study [20] evaluates the performances of 179 classifiers stemming from different families on the UCI classification database, and shows that random forest classifiers overall outperform other classifier families, such as neural networks and support vector machines (SVM).”
Background
Cited by
A Survey on the Explainability of Supervised Machine Learning
Burkart, N., & Huber, M. F. (2021) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.12228 · Cited by 1,008
The What-If Tool: Interactive Probing of Machine Learning Models
Wexler, J., Pushkarna, M., Bolukbasi, T., Wattenberg, M., Viegas, F., & Wilson, J. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2019.2934619 · Cited by 505
El-Sappagh, S., Alonso, J. M., Islam, S. M. R., Sultan, A. M., & Kwak, K. S. (2021) · Scientific Reports · doi:10.1038/s41598-021-82098-3 · Cited by 363
A survey of visual analytics techniques for machine learning
Yuan, J., Chen, C., Yang, W., Liu, M., Xia, J., & Liu, S. (2021) · Computational Visual Media · doi:10.1007/s41095-020-0191-7 · Cited by 258
explAIner: A Visual Analytics Framework for Interactive and Explainable Machine Learning
Spinner, T., Schlegel, U., Schafer, H., & El-Assady, M. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2019.2934629 · Cited by 254
Metadata
- Authors
- Zhao, X., Wu, Y., Lee, D. L., & Cui, W.
- Venue
- IEEE Transactions on Visualization and Computer Graphics
- Published
- 2019-01
- Volume
- 25
- Issue
- 1
- Pages
- 407-416
- ISSN
- 1077-2626 (ISSN-L)
- 1941-0506
- 2160-9306
- OpenAlex
- W2891130433
- Cited by
- 203
- References
- 52
- FWCI
- 9.19
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.