The DOI matches its Crossref record; 100 of the 103 references it cites were found in a registry or an index, and 1 are not checked yet.
An Empirical Study of Model-Agnostic Techniques for Defect Prediction Models
Abstract
Software analytics have empowered software organisations to support a wide range of improved decision-making and policy-making. However, such predictions made by software analytics to date have not been explained and justified. Specifically, current defect prediction models still fail to explain why models make such a prediction and fail to uphold the privacy laws in terms of the requirement to explain any decision made by an algorithm. In this paper, we empirically evaluate three model-agnostic techniques, i.e., two state-of-the-art Local Interpretability Model-agnostic Explanations technique (LIME) and BreakDown techniques, and our improvement of LIME with Hyper Parameter Optimisation (LIME-HPO). Through a case study of 32 highly-curated defect datasets that span across 9 open-source software systems, we conclude that (1) model-agnostic techniques are needed to explain individual predictions of defect models; (2) instance explanations generated by model-agnostic techniques are mostly overlapping (but not exactly the same) with the global explanation of defect models and reliable when they are re-generated; (3) model-agnostic techniques take less than a minute to generate instance explanations; and (4) more than half of the practitioners perceive that the contrastive explanations are necessary and useful to understand the predictions of defect models. Since the implementation of the studied model-agnostic techniques is available in both Python and R, we recommend model-agnostic techniques be used in the future.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Post hoc interpretability approaches such as Local Interpretable Model-agnostic Explanation [164] and Shapley Additive Explanations [116], widely adopted in defect prediction [81, 82], performance analysis [173], code analysis [177, 188], and code generation [112], explain the predictions after the model training and without altering the models.”
From A 2030 Roadmap for Software Engineering (2025) · cites this paper as [82]
Citations
Cites
Storn, R., & Price, K. (1997) · Journal of Global Optimization · doi:10.1023/a:1008202821328 · Cited by 29,569
The Elements of Statistical Learning
Hastie, T., Friedman, J., & Tibshirani, R. (2001) · Springer Series in Statistics · doi:10.1007/978-0-387-21606-5 · Cited by 24,162
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707
(1992) · Cited by 23,431
The meaning and use of the area under a receiver operating characteristic (ROC) curve.
Hanley, J. A., & McNeil, B. J. (1982) · Radiology · doi:10.1148/radiology.143.1.7063747 · Cited by 21,856 · cited 1×
“The values of AUC range between 0 (worst), 0.5 (no better than random guessing), and 1 (best) [33].”
BackgroundInfluentialAn Introduction to the Bootstrap
Efron, B., & Tibshirani, R. J. (1993) · doi:10.1007/978-1-4899-4541-9 · Cited by 21,230
The WEKA data mining software: an update
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I. H. (2009) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1656274.1656278 · Cited by 17,750
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Harrell (2015) · Springer Series in Statistics · doi:10.1007/978-3-319-19425-7 · Cited by 6,878
Lipton, Z. C. (2018) · Queue · doi:10.1145/3236386.3241340 · Cited by 5,427
Research works printed but not linked: 6
- Bas, 1980, The Scientific Image
- French, 2014, Decisionanalysis
- 10.2307/2699986 · doi:10.2307/2699986
- Lipton, The Mythos of model interpretability, 96
- 10.1145/1540438.1540448 · DOI · Paper page
- Regulation, 2016, Regulation (EU) 2016/679 of the European parliament and of the council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing directive 95/46, Official J. Eur. Union, 59, 1
Cited by
Semi-Supervised Log-Based Anomaly Detection via Probabilistic Label Estimation
Yang, L., Chen, J., Wang, Z., Wang, W., Jiang, J., Dong, X., & Zhang, W. (2021) · 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE) · doi:10.1109/icse43902.2021.00130 · Cited by 244
Code Prediction by Feeding Trees to Transformers
Kim, S., Zhao, J., Tian, Y., & Chandra, S. (2021) · 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE) · doi:10.1109/icse43902.2021.00026 · Cited by 209
VulRepair: a T5-based automated software vulnerability repair
Fu, M., Tantithamthavorn, C., Le, T., Nguyen, V., & Phung, D. (2022) · Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3540250.3549098 · Cited by 207
Self-citationGiray, G. (2021) · Journal of Systems and Software · doi:10.1016/j.jss.2021.111031 · Cited by 187
JITLine: A Simpler, Better, Faster, Finer-grained Just-In-Time Defect Prediction
Pornprasit, C., & Tantithamthavorn, C. K. (2021) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr52588.2021.00049 · Cited by 185
Self-citation
Metadata
- Authors
- Jiarpakdee, J., Tantithamthavorn, C. K., Dam, H. K., & Grundy, J.
- Affiliations
- Monash University (AU)
- University of Wollongong (AU)
- Venue
- IEEE Transactions on Software Engineering
- Published
- 2022-01-01
- Volume
- 48
- Issue
- 1
- Pages
- 166-185
- ISSN
- 0098-5589 (ISSN-L)
- 1939-3520
- 2326-3881
- OpenAlex
- W3013745307
- Cited by
- 148
- References
- 103
- Citations by year
- 2026: 12 · 2025: 23 · 2024: 32 · 2023: 27 · 2022: 21
- FWCI
- 19.05
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Software Engineering (publishedVersion)
- Research Online (University of Wollongong) (submittedVersion)
- Funders
- Australian Research Council (DE200100941)
- ARC Laureate Fellowship (FL190100035)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Keywords
- fault prediction, model-agnostic explanation, lime, SHAP, breakdown, explainable AI, privacy, hyperparameter optimization, contrastive explanations, software analytics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: An Empirical Study of Model-Agnostic Techniques for Defect Prediction Models. https://app.cochat.ai/papers/10.1109/tse.2020.2982385
- The paper
- Jiarpakdee, J., Tantithamthavorn, C. K., Dam, H. K., & Grundy, J. (2022). An Empirical Study of Model-Agnostic Techniques for Defect Prediction Models. IEEE Transactions on Software Engineering. https://doi.org/10.1109/tse.2020.2982385
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.