The DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index; 1 of the 5 sources its claims cite were checked for support.
An Empirical Comparison of Interpretable Models to Post-Hoc Explanations
Open accessAbstract
Recently, some effort went into explaining intransparent and black-box models, such as deep neural networks or random forests. So-called model-agnostic methods typically approximate the prediction of the intransparent black-box model with an interpretable model, without considering any specifics of the black-box model itself. It is a valid question whether direct learning of interpretable white-box models should not be preferred over post-hoc approximations of intransparent and black-box models. In this paper, we report the results of an empirical study, which compares post-hoc explanations and interpretable models on several datasets for rule-based and feature-based interpretable models. The results seem to underline that often directly learned interpretable models approximate the black-box models at least as well as their post-hoc surrogates, even though the former do not have direct access to the black-box model.
Overview
CoChat Summary
TL;DR
This paper empirically compares directly learned interpretable models against post-hoc surrogate explanations for rule-based and feature-based settings. It finds that directly learned interpretable models often approximate black-box models at least as well as post-hoc surrogates in accuracy and fidelity, even without access to the black-box model.
Key findings
- For rule-based models on the adult dataset, JRIP achieved accuracy 0.839, fidelity 0.957, and 28 rules, while GLOCALX’s best fidelity setting (α = 10) achieved accuracy 0.762, fidelity 0.942, and 10 rules. Across datasets, JRIP had slightly higher accuracy and fidelity than GLOCALX: adult 0.839/0.957 vs. 0.762/0.942; German 0.712/0.809 vs. 0.700/0.764; NHANES I 0.823/0.838 vs. 0.788/0.819; compas-two-year 0.782/0.836 vs. 0.756/0.804.
- For feature-based models, the paper states GA2MS had higher accuracy and fidelity than MAPLE on all datasets; Table 4 shows, for example, Adult GA2MS 0.851/0.901 vs. MAPLE 0.723/0.734, and NHANES I GA2MS 0.826/0.867 vs. MAPLE 0.745/0.788. One table entry is an accuracy exception to the text’s “all datasets” claim: credit card fraud MAPLE accuracy 0.947 vs. GA2MS 0.939, though GA2MS fidelity is higher (0.989 vs. 0.955).
- The paper reports Rashomon-effect differences: MAPLE and GA2MS give different feature importances on adult, e.g., age 0.187 vs. 0.665, marital status 0.002 vs. 0.629, capital gain 0.640 vs. 0.551, and relationship −0.050 vs. 0.487.
- GLOCALX generally generated shorter rules with fewer conditions than JRIP, but JRIP obtained higher accuracy while generating more complex rules.
Methods
- Binary classification experiments used datasets including adult, compas-two-year, German, NHANES I, credit card fraud, Bank, and Heart Disease, mostly from the UCI repository.
- Rule-based comparison paired GLOCALX (post-hoc, using random forests as black-box and LORE for local explanations) against JRIP (direct rule learner). Data were split into 60% for black-box training, 20% for GLOCALX training, and 20% for validation; JRIP used the first 80% as training.
- Feature-based comparison paired MAPLE (post-hoc) against GA2MS (interpretable GAM). Evaluation used accuracy, fidelity, and number of rules for rule-based models; feature-based fidelity used normalized feature-importance weights, a linear scoring rule, ROC-based threshold tuning, and AUC.
Limitations
- The paper does not state a dedicated limitations section or explicit limitations.
- Evident from the text: experiments are limited to binary classification datasets, and the rule-based setup uses one black-box model (random forests) and one local explainer (LORE) for GLOCALX.
- Evident from the text: the feature-based fidelity comparison uses a proxy linear model with normalized weights and threshold tuning rather than native MAPLE/GA2MS prediction outputs, and no statistical tests are reported.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
1 Valid · 0 Not valid · 4 Not checked · 16 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As can be seen, for the adult dataset, GLOCALX obtained its best results in terms of fidelity and the number of rules with α = 10.”
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“By comparing the rules provided by GLOCALX and JRIP in terms of the rule length and number of conditions for each rule, we see that in general GLOCALX generates shorter rules with smaller number of conditions than JRIP.”
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“According to the experiment's results, the performance of interpretable models in terms of accuracy and fidelity is as good as post-hoc methods.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Thus, as a simple inference, GLOCALX can be more understandable for human since it generates shorter and simpler rules.”
plainstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, here we see this phenomenon from a novel angle: Very different rules may provide different explanations with the same fidelity to an underlying black-box model.”
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
2 more claims
Create a free account to see all 21 claims and every source verdict.
- Probably not a substantial claim. 7
- It may state two separate claims as one. 11
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ 4.2. Results on Feature-Based Interpretability Methods
“The results confirm that in all the datasets, GA 2 MS has higher accuracy and fidelity compared to MAPLE”
§ 4.2. Results on Feature-Based Interpretability Methods
“| Credit card fraud | 0.939 | 0.989 | 0.947 | 0.955 |”
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ 4.2. Results on Feature-Based Interpretability Methods
“The results confirm that in all the datasets, GA 2 MS has higher accuracy and fidelity compared to MAPLE”
§ 4.2. Results on Feature-Based Interpretability Methods
“| Bank | 0.731 | 0.807 | 0.713 | 0.825 |”
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561 · cited 2×
“For these and other reasons, it has been argued that more efforts should be devoted to learning more accurate interpretable models [7].”
BackgroundMethodFrom local explanations to global understanding with explainable AI for trees
Lundberg, S. M., Erion, G., Chen, H., DeGrave, A., Prutkin, J. M., Nair, B., Katz, R., Himmelfarb, J., Bansal, N., & Lee, S.-I. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-019-0138-9 · Cited by 10,551 · cited 1×
“Recently, several approaches have been investigated which try to construct global models from local explanations (e.g., [4,5]).”
Background“Why Should I Trust You?”: Explaining the Predictions of Any Classifier
Ribeiro, M., Singh, S., & Guestrin, C. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations · doi:10.18653/v1/n16-3020 · Cited by 6,682
Statistical Modeling: The Two Cultures (with comments and a rejoinder by the author)
Breiman, L. (2001) · Statistical Science · doi:10.1214/ss/1009213726 · Cited by 4,340 · cited 2×
“Furthermore, different ways of explaining a model might exist according to the so-called Rashomon effect [20], which, in a nutshell, states that in particular with structured models such as trees or rules, there are often multiple different models which explain the data equally well.”
BackgroundCohen, W. W. (1995) · Machine Learning Proceedings 1995 · doi:10.1016/b978-1-55860-377-6.50023-2 · Cited by 3,787
Explainable AI: Interpreting, Explaining and Visualizing Deep Learning
(2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-28954-6 · Cited by 1,377
Interpretable Machine Learning
Molnar, C. (2020) · Cited by 1,104
Layer-Wise Relevance Propagation: An Overview
Montavon, G., Binder, A., Lapuschkin, S., Samek, W., & Müller, K.-R. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-28954-6_10 · Cited by 750
A Survey of Methods for Explaining Black Box Models
Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., & Pedreschi, D. (2019) · ACM Computing Surveys · doi:10.1145/3236009 · Cited by 629 · cited 1×
“The necessity of having some explanations to understand how the model works led to substantial research on explaining learned models [1].”
BackgroundAccurate intelligible models with pairwise interactions
Lou, Y., Caruana, R., Gehrke, J., & Hooker, G. (2013) · Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2487575.2487579 · Cited by 564 · cited 3×
“Among various feature-based methods on interpretability, GA 2 M S [13] is selected as a glass-box model, intelligible algorithm, and MAPLE is selected as a post-hoc method.”
BackgroundMethodInfluentialFürnkranz, J., Gamberger, D., & Lavrač, N. (2012) · Cognitive Technologies · doi:10.1007/978-3-540-75197-7 · Cited by 428 · cited 2×
“Both methods generate explanations in the form of rulesets, which are our preference as they produce more compact models and are very close to human reasoning language [11].”
BackgroundMethodSelf-citationFactual and Counterfactual Explanations for Black Box Decision Making
Guidotti, R., Monreale, A., Giannotti, F., Pedreschi, D., Ruggieri, S., & Turini, F. (2019) · IEEE Intelligent Systems · doi:10.1109/mis.2019.2957223 · Cited by 349 · cited 2×
“LOcal Rule-based Explanations (LORE) is a variant of LIME, which is specifically tailored to rules as local surrogate models [14].”
BackgroundMethodExplainable AI for Trees: From Local Explanations to Global Understanding
Lundberg, S., Erion, G., Chen, H., DeGrave, A. J., Prutkin, J. M., Nair, B. G., Katz, R., Himmelfarb, J., Bansal, N., & Lee, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.04610 · Cited by 249
Explainable and Interpretable Models in Computer Vision and Machine Learning
(2018) · The Springer Series on Challenges in Machine Learning · doi:10.1007/978-3-319-98131-4 · Cited by 199 · cited 1×
“We refer to [8–10] for general surveys and only briefly recapitulate the most relevant works for our study.”
Model Agnostic Supervised Local Explanations
Plumb, G., Molitor, D., & Talwalkar, A. (2018) · arXiv · doi:10.48550/arxiv.1807.02910 · Cited by 149
GLocalX—From Local to Global Explanations of Black Box AI Models
Setzu, M., Guidotti, R., Monreale, A., Turini, F., Pedreschi, D., & Giannotti, F. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103457 · Cited by 140 · cited 4×
“GL OCAL X [4] generates global explanations for a black-box model using local explanations created by a local surrogate model such as LORE [14], and the predicted labels from a black-box model.”
BackgroundMethodInfluentialGlobal Model Interpretation Via Recursive Partitioning
Yang, C., Rangarajan, A., & Ranka, S. (2018) · 2018 IEEE 20th International Conference on High Performance Computing and Communications; IEEE 16th International Conference on Smart City; IEEE 4th International Conference on Data Science and Systems (HPCC/SmartCity/DSS) · doi:10.1109/hpcc/smartcity/dss.2018.00256 · Cited by 93 · cited 1×
“[18] proposed Global Interpretation via Recursive Partitioning (GIRP), a method to build a global interpretation tree for a wide range of machine learning models based on their local explanations.”
MethodBoulicaut, J.-F., Siebes, A., & Morik, K. (2005) · Lecture notes in computer science · doi:10.1007/b137601 · Cited by 51
Global Aggregations of Local Explanations for Black Box models
van der Linden, I., Haned, H., & Kanoulas, E. (2019) · arXiv · doi:10.48550/arxiv.1907.03039 · Cited by 35
From Local to Global Patterns: Evaluation Issues in Rule Learning Algorithms
Fürnkranz, J. (2005) · Lecture notes in computer science · doi:10.1007/11504245_2 · Cited by 22
Self-citation
Research works printed but not linked: 6
- Krishnapuram, B., Shah, M., Smola, A.J., Aggarwal, C.C., Shen, D., and Rastogi, R. (2016). Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), ACM.
- Guyon, I., von Luxburg, U., Bengio, S., Wallach, H.M., Fergus, R., Vishwanathan, S.V.N., and Garnett, R. (2017). Advances in Neural Information Processing Systems 30, Proceedings of the NIPS 2017, Long Beach, CA, USA, 4–9 December 2017, Curran Associates Inc.
- Samek, W., Montavon, G., Vedaldi, A., Hansen, L.K., and Müller, K. (2019). Explainable AI: Interpreting, Explaining and Visualizing Deep Learning, Springer. · DOI · Paper page
- Prieditis, A., and Russell, S. (1995). Machine Learning Proceedings 1995, Proceedings of the Twelfth International Conference on Machine Learning, Tahoe City, CA, USA, 9–12 July 1995, Morgan Kaufmann.
- Bengio, S., Wallach, H.M., Larochelle, H., Grauman, K., Cesa-Bianchi, N., and Garnett, R. (2018). Advances in Neural Information Processing Systems 31, Proceedings of the NeurIPS 2018, Montreal, QC, Canada, 3–8 December 2018, Curran Associates Inc.
- Knobbe, A.J. (2008). From Local Patterns to Global Models: Proceedings of the ECML/PKDD-08 Workshop (LeGo-08), University and State Library Darmstadt.
Cited by
Qian, X., Wang, B., Chen, J., Fan, Y., Mo, R., Xu, C., Liu, W., Liu, J., & Zhong, P.-A. (2025) · Journal of Hydrology · doi:10.1016/j.jhydrol.2025.133968 · Cited by 29
Gurmessa, D. K., & Jimma, W. (2023) · Cogent Engineering · doi:10.1080/23311916.2023.2273088 · Cited by 12
“One proposed solution is to sidestep the need for post-hoc explanations altogether by employing interpretable models from the outset, as suggested by (Chaddad et al., 2023; Mahya & Fürnkranz, 2023b).”
MethodInterpretable machine learning for precision cognitive aging
Mahamadou, A. J. D., Rodrigues, E. A., Vakorin, V. A., Antoine, V., & Moreno, S. (2025) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2025.1560064 · Cited by 6
Khan, H., AlQahtani, F., Jiang, W., Ali, M. H., Amin, S. U., & Koo, J. (2026) · Computer Science Review · doi:10.1016/j.cosrev.2026.101037 · Cited by 5
Alves, A. H. D. S., Neto, G. A. C., Gomes, M. D. S., et al. (2025) · Applied Sciences · doi:10.3390/app15052608 · Cited by 2
Metadata
- Authors
- Mahya, P., & Fürnkranz, J.
- Affiliations
- Johannes Kepler University of Linz (AT)
- Corresponding author
- Mahya, Parisa
- Venue
- AI
- Publisher
- MDPI AG
- Published
- 2023-05-19
- Volume
- 4
- Issue
- 2
- Pages
- 426-436
- ISSN
- 2673-2688 (ISSN-L)
- OpenAlex
- W4377115400
- Semantic Scholar
- e962ff891497ccdb9fa1fca3ba64a24bd54e027c
- Cited by
- 17
- References
- 14
- Citations by year
- 2026: 5 · 2025: 9 · 2024: 2 · 2023: 1
- FWCI
- 1.21
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- AI (publishedVersion, cc-by)
- University Library Linz repository (Johannes Kepler Universitat Linz) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Keywords
- interpretable models, post-hoc explanation, model-agnostic explanation, black-box models, white-box models, rule-based models, feature-based models, explainable machine learning, model approximation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-02, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-02 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.