Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 100 of the 103 references it cites were found in a registry or an index, and 1 are not checked yet.

An Empirical Study of Model-Agnostic Techniques for Defect Prediction Models

Jiarpakdee, Jirayus, Tantithamthavorn, Chakkrit Kla, Dam, Hoa Khanh, Grundy, John · 2022 · IEEE Transactions on Software Engineering

Abstract

Software analytics have empowered software organisations to support a wide range of improved decision-making and policy-making. However, such predictions made by software analytics to date have not been explained and justified. Specifically, current defect prediction models still fail to explain why models make such a prediction and fail to uphold the privacy laws in terms of the requirement to explain any decision made by an algorithm. In this paper, we empirically evaluate three model-agnostic techniques, i.e., two state-of-the-art Local Interpretability Model-agnostic Explanations technique (LIME) and BreakDown techniques, and our improvement of LIME with Hyper Parameter Optimisation (LIME-HPO). Through a case study of 32 highly-curated defect datasets that span across 9 open-source software systems, we conclude that (1) model-agnostic techniques are needed to explain individual predictions of defect models; (2) instance explanations generated by model-agnostic techniques are mostly overlapping (but not exactly the same) with the global explanation of defect models and reliable when they are re-generated; (3) model-agnostic techniques take less than a minute to generate instance explanations; and (4) more than half of the practitioners perceive that the contrastive explanations are necessary and useful to understand the predictions of defect models. Since the implementation of the studied model-agnostic techniques is available in both Python and R, we recommend model-agnostic techniques be used in the future.

Overview

Integrity signals

Registry

Crossref · 10.1109/tse.2020.2982385

Cites

100 of 101 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 103 references

Research works printed but not linked: 6

  1. Bas, 1980, The Scientific Image
  2. French, 2014, Decisionanalysis
  3. 10.2307/2699986 · doi:10.2307/2699986
  4. Lipton, The Mythos of model interpretability, 96
  5. 10.1145/1540438.1540448 · DOI · Paper page
  6. Regulation, 2016, Regulation (EU) 2016/679 of the European parliament and of the council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing directive 95/46, Official J. Eur. Union, 59, 1

Cited by

See all 183

Metadata

Authors
Jiarpakdee, J., Tantithamthavorn, C. K., Dam, H. K., & Grundy, J.
Affiliations
  • Monash University (AU)
  • University of Wollongong (AU)
Venue
IEEE Transactions on Software Engineering
Published
2022-01-01
Volume
48
Issue
1
Pages
166-185
ISSN
  • 0098-5589 (ISSN-L)
  • 1939-3520
  • 2326-3881
OpenAlex
W3013745307
Cited by
148
References
103
Citations by year
2026: 12 · 2025: 23 · 2024: 32 · 2023: 27 · 2022: 21
FWCI
19.05
Type
journal article
Language
en
Open access
closed
Locations
  • IEEE Transactions on Software Engineering (publishedVersion)
  • Research Online (University of Wollongong) (submittedVersion)
Funders
  • Australian Research Council (DE200100941)
  • ARC Laureate Fellowship (FL190100035)
Topics
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Research
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
Keywords
fault prediction, model-agnostic explanation, lime, SHAP, breakdown, explainable AI, privacy, hyperparameter optimization, contrastive explanations, software analytics

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: An Empirical Study of Model-Agnostic Techniques for Defect Prediction Models. https://app.cochat.ai/papers/10.1109/tse.2020.2982385
The paper
Jiarpakdee, J., Tantithamthavorn, C. K., Dam, H. K., & Grundy, J. (2022). An Empirical Study of Model-Agnostic Techniques for Defect Prediction Models. IEEE Transactions on Software Engineering. https://doi.org/10.1109/tse.2020.2982385

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.