An Empirical Study of Model-Agnostic Techniques for Defect Prediction Models
The DOI matches its Crossref record; 101 of the 103 references it cites were found in a registry or an index.
Abstract
Software analytics have empowered software organisations to support a wide range of improved decision-making and policy-making. However, such predictions made by software analytics to date have not been explained and justified. Specifically, current defect prediction models still fail to explain why models make such a prediction and fail to uphold the privacy laws in terms of the requirement to explain any decision made by an algorithm. In this paper, we empirically evaluate three model-agnostic techniques, i.e., two state-of-the-art Local Interpretability Model-agnostic Explanations technique (LIME) and BreakDown techniques, and our improvement of LIME with Hyper Parameter Optimisation (LIME-HPO). Through a case study of 32 highly-curated defect datasets that span across 9 open-source software systems, we conclude that (1) model-agnostic techniques are needed to explain individual predictions of defect models; (2) instance explanations generated by model-agnostic techniques are mostly overlapping (but not exactly the same) with the global explanation of defect models and reliable when they are re-generated; (3) model-agnostic techniques take less than a minute to generate instance explanations; and (4) more than half of the practitioners perceive that the contrastive explanations are necessary and useful to understand the predictions of defect models. Since the implementation of the studied model-agnostic techniques is available in both Python and R, we recommend model-agnostic techniques be used in the future.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Post hoc interpretability approaches such as Local Interpretable Model-agnostic Explanation [164] and Shapley Additive Explanations [116], widely adopted in defect prediction [81, 82], performance analysis [173], code analysis [177, 188], and code generation [112], explain the predictions after the model training and without altering the models.”
From A 2030 Roadmap for Software Engineering (2025) · cites this paper as [82]
Citations
Cites
Storn, R., & Price, K. (1997) · Journal of Global Optimization · doi:10.1023/a:1008202821328 · Cited by 29,569
The Elements of Statistical Learning
Hastie, T., Friedman, J., & Tibshirani, R. (2001) · Springer Series in Statistics · doi:10.1007/978-0-387-21606-5 · Cited by 24,162
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707
(1992) · Cited by 23,431
The meaning and use of the area under a receiver operating characteristic (ROC) curve.
Hanley, J. A., & McNeil, B. J. (1982) · Radiology · doi:10.1148/radiology.143.1.7063747 · Cited by 21,856 · cited 1×
“The values of AUC range between 0 (worst), 0.5 (no better than random guessing), and 1 (best) [33].”
BackgroundInfluentialAn Introduction to the Bootstrap
Efron, B., & Tibshirani, R. J. (1993) · doi:10.1007/978-1-4899-4541-9 · Cited by 21,230
The WEKA data mining software: an update
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I. H. (2009) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1656274.1656278 · Cited by 17,750
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Harrell (2015) · Springer Series in Statistics · doi:10.1007/978-3-319-19425-7 · Cited by 6,878
Lipton, Z. C. (2018) · Queue · doi:10.1145/3236386.3241340 · Cited by 5,427 · cited 1×
“The explainability of AI/ML algorithms can be achieved by (1) making the entire decision-making process transparent and compre-hensible and (2) explicitly providing an explanation for each decision [54] (since an explanation is not likely applicable to all decisions [50]).”
MethodExplanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
Gradient boosting machines, a tutorial
Natekin, A., & Knoll, A. (2013) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2013.00021 · Cited by 3,887 · cited 1×
“We use the relative influence of metrics derived from boosted trees to generate the global explanation of GBM and xGBTree [24], [71].”
MethodConditional variable importance for random forests
Strobl, C., Boulesteix, A.-L., Kneib, T., Augustin, T., & Zeileis, A. (2008) · BMC Bioinformatics · doi:10.1186/1471-2105-9-307 · Cited by 3,254
Individual Comparisons by Ranking Methods
Wilcoxon, F. (1992) · Springer Series in Statistics · doi:10.1007/978-1-4612-4380-9_16 · Cited by 2,915
Quinlan, J. R. (1987) · International Journal of Man-Machine Studies · doi:10.1016/s0020-7373(87)80053-6 · Cited by 2,553
Scientific Explanation and the Causal Structure of the World
Salmon, W. C. (1985) · Princeton University Press eBooks · doi:10.1515/9780691221489 · Cited by 2,459
Krosnick, J. A. (1999) · Annual Review of Psychology · doi:10.1146/annurev.psych.50.1.537 · Cited by 1,852
How to Measure Survey Reliability and Validity
Litwin, M. (1995) · Sage eBooks · doi:10.4135/9781483348957 · Cited by 1,459
Gevrey, M., Dimopoulos, I., & Lek, S. (2003) · Ecological Modelling · doi:10.1016/s0304-3800(02)00257-0 · Cited by 1,248
Lessmann, S., Baesens, B., Mues, C., & Pietsch, S. (2008) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2008.35 · Cited by 1,226
Data Mining Static Code Attributes to Learn Defect Predictors
Menzies, T., Greenwald, J., & Frank, A. (2007) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2007.256941 · Cited by 1,215
A Systematic Literature Review on Fault Prediction Performance in Software Engineering
Hall, T., Beecham, S., Bowes, D., Gray, D., & Counsell, S. (2012) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2011.103 · Cited by 1,154 · cited 2×
“Shihab [88] and Hall et al. [32] show that logistic regression and random forests are the two most-popularly-used classi-fication techniques in the literature of software defect prediction, since they are explainable and have built-in model explanation techniques (i.e., the ANOVA analysis for the…”
BackgroundMethodAnvik, J., Hiew, L., & Murphy, G. C. (2006) · Proceedings of the 28th international conference on Software engineering · doi:10.1145/1134285.1134336 · Cited by 924
Mining metrics to predict component failures
Nagappan, N., Ball, T., & Zeller, A. (2006) · Proceedings of the 28th international conference on Software engineering · doi:10.1145/1134285.1134349 · Cited by 782
Wiley StatsRef: Statistics Reference Online
(2014) · Wiley eBooks · doi:10.1002/9781118445112 · Cited by 780
A large-scale empirical study of just-in-time quality assurance
Kamei, Y., Shihab, E., Adams, B., Hassan, A. E., Mockus, A., Sinha, A., & Ubayashi, N. (2013) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2012.70 · Cited by 715
Model-Agnostic Interpretability of Machine Learning
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · arXiv · doi:10.48550/arxiv.1606.05386 · Cited by 699
Cross-project defect prediction: a large scale experiment on data vs. domain vs. process
Zimmermann, T., Nagappan, N., Gall, H. C., Giger, E., & Murphy, B. (2009) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-25785 · Cited by 694
DEoptim: An R Package for Global Optimization by Differential Evolution
Mullen, K., Ardia, D., Gil, D., Windover, D., & Cline, J. (2011) · Journal of Statistical Software · doi:10.18637/jss.v040.i06 · Cited by 687
The structure and function of explanations
Lombrozo, T. (2006) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2006.08.004 · Cited by 639
An extensive comparison of bug prediction approaches
D'Ambros, M., Lanza, M., & Robbes, R. (2010) · 2010 7th IEEE Working Conference on Mining Software Repositories (MSR 2010) · doi:10.1109/msr.2010.5463279 · Cited by 637
A Survey of Methods for Explaining Black Box Models
Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., & Pedreschi, D. (2019) · ACM Computing Surveys · doi:10.1145/3236009 · Cited by 629 · cited 2×
“According to the literature survey of model-agnostic techniques [30], we selected and discussed two state-of-the-art model-agnostic techniques (i.e., LIME and BreakDown).”
BackgroundMethodEvaluating defect prediction approaches: a benchmark and an extensive comparison
D’Ambros, M., Lanza, M., & Robbes, R. (2012) · Empirical Software Engineering · doi:10.1007/s10664-011-9173-9 · Cited by 626
An Empirical Comparison of Model Validation Techniques for Defect Prediction Models
Tantithamthavorn, C., McIntosh, S., Hassan, A. E., & Matsumoto, K. (2017) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2016.2584050 · Cited by 574 · cited 4×
“We use the out-of-sample bootstrap validation technique (1) to ensure that the generated training samples are representative to the original dataset and (2) to ensure that the produced estimates are the least bias and most reliable [101].”
MethodInfluentialSelf-citationPredicting defect-prone software modules using support vector machines
Elish, K. O., & Elish, M. O. (2008) · Journal of Systems and Software · doi:10.1016/j.jss.2007.07.040 · Cited by 525
Kitchenham, B. A., & Pfleeger, S. L. (2008) · Guide to Advanced Empirical Software Engineering · doi:10.1007/978-1-84800-044-5_3 · Cited by 484
Arisholm, E., Briand, L. C., & Johannessen, E. B. (2010) · Journal of Systems and Software · doi:10.1016/j.jss.2009.06.055 · Cited by 455
The Impact of Automated Parameter Optimization on Defect Prediction Models
Tantithamthavorn, C., McIntosh, S., Hassan, A. E., & Matsumoto, K. (2019) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2018.2794977 · Cited by 415
Self-citationRevisiting the Impact of Classification Techniques on the Performance of Defect Prediction Models
Ghotra, B., McIntosh, S., & Hassan, A. E. (2015) · 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering · doi:10.1109/icse.2015.91 · Cited by 364
McIntosh, S., Kamei, Y., Adams, B., & Hassan, A. E. (2014) · Proceedings of the 11th Working Conference on Mining Software Repositories · doi:10.1145/2597073.2597076 · Cited by 348
Don't touch my code!: examining the effects of ownership on software quality
Bird, C., Nagappan, N., Murphy, B., Gall, H., & Devanbu, P. (2011) · Proceedings of the 19th ACM SIGSOFT symposium and the 13th European conference on Foundations of software engineering · doi:10.1145/2025113.2025119 · Cited by 340
Automated parameter optimization of classification techniques for defect prediction models
Tantithamthavorn, C., McIntosh, S., Hassan, A. E., & Matsumoto, K. (2016) · Proceedings of the 38th International Conference on Software Engineering · doi:10.1145/2884781.2884857 · Cited by 331
Self-citationSoftware defect prediction using Bayesian networks
Okutan, A., & Yıldız, O. T. (2014) · Empirical Software Engineering · doi:10.1007/s10664-012-9218-8 · Cited by 330
Tantithamthavorn, C., Hassan, A. E., & Matsumoto, K. (2020) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2018.2876537 · Cited by 327
Self-citationDeep Semantic Feature Learning for Software Defect Prediction
Wang, S., Liu, T., Nam, J., & Tan, L. (2020) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2018.2877612 · Cited by 320
Lipton, P. (1990) · Royal Institute of Philosophy Supplement · doi:10.1017/s1358246100005130 · Cited by 281
Heterogeneous Defect Prediction
Nam, J., Fu, W., Kim, S., Menzies, T., & Tan, L. (2018) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2017.2720603 · Cited by 277
Local versus Global Lessons for Defect Prediction and Effort Estimation
Menzies, T., Butcher, A., Cok, D., Marcus, A., Layman, L., Shull, F., Turhan, B., & Zimmermann, T. (2013) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2012.83 · Cited by 261
Cross-project defect prediction: a large scale experiment on data vs. domain vs. process
Zimmermann, T., Nagappan, N., Gall, H. C., Giger, E., & Murphy, T. B. (2009) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/1595696.1595713 · Cited by 257
The Psychology of Counterfactual Thinking
(2007) · Routledge eBooks · doi:10.4324/9780203963784 · Cited by 255
How, and why, process metrics are better
Rahman, F., & Devanbu, P. (2013) · 2013 35th International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2013.6606589 · Cited by 248
Reducing Features to Improve Code Change-Based Bug Prediction
Shivaji, S., Whitehead, E. J., Akella, R., & Kim, S. (2013) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2012.43 · Cited by 247
Yang, Y., Zhou, Y., Liu, J., Zhao, Y., Lu, H., Xu, L., Xu, B., & Leung, H. (2016) · Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering · doi:10.1145/2950290.2950353 · Cited by 229
Online Defect Prediction for Imbalanced Data
Tan, M., Tan, L., Dara, S., & Mayeux, C. (2015) · 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering · doi:10.1109/icse.2015.139 · Cited by 229
Thongtanunam, P., Tantithamthavorn, C., Kula, R. G., Yoshida, N., Iida, H., & Matsumoto, K.-I. (2015) · 2015 IEEE 22nd International Conference on Software Analysis, Evolution, and Reengineering (SANER) · doi:10.1109/saner.2015.7081824 · Cited by 228
Self-citationTuning for software analytics: Is it really necessary?
Fu, W., Menzies, T., & Shen, X. (2016) · Information and Software Technology · doi:10.1016/j.infsof.2016.04.017 · Cited by 210
Ownership, experience and defects: a fine-grained study of authorship
Rahman, F., & Devanbu, P. (2011) · Proceedings of the 33rd International Conference on Software Engineering · doi:10.1145/1985793.1985860 · Cited by 203
Revisiting the evaluation of defect prediction models
Mende, T., & Koschke, R. (2009) · Proceedings of the 5th International Conference on Predictor Models in Software Engineering · doi:10.1145/1540438.1540448 · Cited by 179
Multi-objective Cross-Project Defect Prediction
Canfora, G., De Lucia, A., Di Penta, M., Oliveto, R., Panichella, A., & Panichella, S. (2013) · 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation · doi:10.1109/icst.2013.38 · Cited by 171
Fine-grained just-in-time defect prediction
Pascarella, L., Palomba, F., & Bacchelli, A. (2018) · Journal of Systems and Software · doi:10.1016/j.jss.2018.12.001 · Cited by 161
Agrawal, A., & Menzies, T. (2018) · Proceedings of the 40th International Conference on Software Engineering · doi:10.1145/3180155.3180197 · Cited by 159
Multiple kernel ensemble learning for software defect prediction
Wang, T., Zhang, Z., Jing, X., & Zhang, L. (2015) · Automated Software Engineering · doi:10.1007/s10515-015-0179-1 · Cited by 157
Mining Software Defects: Should We Consider Affected Releases?
Yatish, S., Jiarpakdee, J., Thongtanunam, P., & Tantithamthavorn, C. K. (2019) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2019.00075 · Cited by 156
Self-citationDoes bug prediction support human developers? Findings from a Google case study
Lewis, C., Lin, Z., Sadowski, C., Zhu, X., Ou, R., & Whitehead, E. J. (2013) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2013.6606583 · Cited by 148
The Impact of Correlated Metrics on the Interpretation of Defect Models
Jiarpakdee, J., Tantithamthavorn, C. K., & Hassan, A. E. (2019) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2019.2891758 · Cited by 146
Self-citationA Large-Scale Study of the Impact of Feature Selection Techniques on Defect Classification Models
Ghotra, B., McIntosh, S., & Hassan, A. E. (2017) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr.2017.18 · Cited by 142
Menzies, T., & Zimmermann, T. (2013) · IEEE Software · doi:10.1109/ms.2013.86 · Cited by 141
The Impact of Mislabelling on the Performance and Interpretation of Defect Prediction Models
Tantithamthavorn, C., McIntosh, S., Hassan, A. E., Ihara, A., & Matsumoto, K. (2015) · 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering · doi:10.1109/icse.2015.93 · Cited by 140
Self-citationThongtanunam, P., McIntosh, S., Hassan, A. E., & Iida, H. (2016) · Proceedings of the 38th International Conference on Software Engineering · doi:10.1145/2884781.2884852 · Cited by 138
Explanations of Model Predictions with Live and breakDown Packages
Staniak, M., & Biecek, P. (2018) · Lincoln (University of Nebraska) · Cited by 128
Supervised vs Unsupervised Models: A Holistic Look at Effort-Aware Just-in-Time Defect Prediction
Huang, Q., Xia, X., & Lo, D. (2017) · 2017 IEEE International Conference on Software Maintenance and Evolution (ICSME) · doi:10.1109/icsme.2017.51 · Cited by 124
Special issue on repeatable results in software engineering prediction
Menzies, T., & Shepperd, M. (2012) · Empirical Software Engineering · doi:10.1007/s10664-011-9193-5 · Cited by 122
An experience report on defect modelling in practice: pitfalls and challenges
Tantithamthavorn, C., & Hassan, A. E. (2018) · Proceedings of the 40th International Conference on Software Engineering: Software Engineering in Practice · doi:10.1145/3183519.3183547 · Cited by 122
Self-citationExplainable software analytics
Dam, H. K., Tran, T., & Ghose, A. (2018) · Proceedings of the 40th International Conference on Software Engineering: New Ideas and Emerging Results · doi:10.1145/3183399.3183424 · Cited by 120
Self-citationThink locally, act globally: Improving defect and effort prediction models
Bettenburg, N., Nagappan, M., & Hassan, A. E. (2012) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr.2012.6224300 · Cited by 111
Effort Estimation in Agile Software Development using Story Points
Coelho, E., & Basu, A. (2012) · International Journal of Applied Information Systems · doi:10.5120/ijais12-450574 · Cited by 109
Can data transformation help in the detection of fault-prone modules?
Jiang, Y., Čukić, B., & Menzies, T. (2008) · doi:10.1145/1390817.1390822 · Cited by 104
MANAGEMENT HANDBOOK FOR THE ESTIMATION OF COMPUTER PROGRAMMING COSTS
Nelson, E. A. (1967) · doi:10.21236/ad0648750 · Cited by 95
Explanations of Model Predictions with live and breakDown Packages
Staniak, M., & Biecek, P. (2019) · The R Journal · doi:10.32614/rj-2018-072 · Cited by 94
An investigation of the effect of module size on defect prediction using static measures
Koru, A. G., & Liu, H. (2005) · ACM SIGSOFT Software Engineering Notes · doi:10.1145/1082983.1083172 · Cited by 90
Remote Causes, Bad Explanations?
BOUWEL, J. V., & Weber, E. (2002) · Journal for the Theory of Social Behaviour · doi:10.1111/1468-5914.00197 · Cited by 80
The Impact of Using Regression Models to Build Defect Classifiers
Rajbahadur, G. K., Wang, S., Kamei, Y., & Hassan, A. E. (2017) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr.2017.4 · Cited by 80
Comments on “Researcher Bias: The Use of Machine Learning in Software Defect Prediction”
Tantithamthavorn, C., McIntosh, S., Hassan, A. E., & Matsumoto, K. (2016) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2016.2553030 · Cited by 76
Self-citationExplanation as a Cognitive Process
Horne, Z., Muradoglu, M., & Cimpian, A. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2018.12.004 · Cited by 76
Application of Random Forest in Predicting Fault-Prone Classes
Kaur, A., & Malhotra, R. (2008) · 2008 International Conference on Advanced Computer Theory and Engineering · doi:10.1109/icacte.2008.204 · Cited by 70
Reducing false alarms in software defect prediction by decision threshold optimization
Tosun, A., & Bener, A. (2009) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem.2009.5316006 · Cited by 67
AutoSpearman: Automatically Mitigating Correlated Software Metrics for Interpreting Defect Models
Jiarpakdee, J., Tantithamthavorn, C., & Treude, C. (2018) · 2018 IEEE International Conference on Software Maintenance and Evolution (ICSME) · doi:10.1109/icsme.2018.00018 · Cited by 65
Self-citationSearching for rules to detect defective modules: A subgroup discovery approach
Rodríguez, D., Ruiz, R., Riquelme, J. C., & Aguilar–Ruiz, J. S. (2012) · Information Sciences · doi:10.1016/j.ins.2011.01.039 · Cited by 62
Petković, D., Sosnick, M. H., Okada, K., Todtenhoefer, R., Huang, S., Miglani, N., & Vigil, A. (2016) · IEEE Frontiers in Education Conference (FIE) · doi:10.1109/fie.2016.7757406 · Cited by 60
Replication of defect prediction studies: problems, pitfalls and recommendations
Mende, T. (2010) · Proceedings of the 6th International Conference on Predictive Models in Software Engineering · doi:10.1145/1868328.1868336 · Cited by 57
Agile software development: Human values and culture
Miller, K., & Larson, D. K. (2005) · IEEE Technology and Society Magazine · doi:10.1109/mtas.2005.1563500 · Cited by 50
An Exploration of Challenges Limiting Pragmatic Software Defect Prediction
Shihab, E. (2012) · QSpace (Queen's University Library) · Cited by 48
Applications of psychological science for actionable analytics
Chen, D., Fu, W., Krishna, R., & Menzies, T. (2018) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3236024.3236050 · Cited by 47
Better software analytics via “DUO”: Data mining algorithms using/used-by optimizers
Agrawal, A., Menzies, T., Minku, L. L., Wagner, M., & Yu, Z. (2020) · Empirical Software Engineering · doi:10.1007/s10664-020-09808-9 · Cited by 40
A Study of Redundant Metrics in Defect Prediction Datasets
Jiarpakdee, J., Tantithamthavorn, C. K., Ihara, A., & Matsumoto, K. (2016) · IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW) · doi:10.1109/issrew.2016.30 · Cited by 37
Self-citationPredicting Fault-Prone Software Modules with Rank Sum Classification
Cahill, J., Hogan, J. M., & Thomas, R. N. (2013) · Australasian Software Engineering Conference (ASWEC) · doi:10.1109/aswec.2013.33 · Cited by 28
Tantithamthavorn, C. K. (2016) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/2889160.2889256 · Cited by 28
Self-citationLeake, D. B. (2014) · Psychology Press eBooks · doi:10.4324/9781315807072 · Cited by 27
The course of events: counterfactuals, causal sequences, and explanation
Hilton, D., & McClure, J. (2005) · ResearchOnline - JCU (James Cook University) · doi:10.4324/9780203963784-11 · Cited by 22
Simpler Transfer Learning (Using "Bellwethers").Preprint
Krishna, R., & Menzies, T. (2017) · arXiv (Cornell University) · Cited by 4
Artefact: An R Implementation of the AutoSpearman Function
Jiarpakdee, J., Tantithamthavorn, C. K., & Treude, C. (2018) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme.2018.00083 · Cited by 3
Self-citation
Research works printed but not linked: 6
- Bas, 1980, The Scientific Image
- French, 2014, Decisionanalysis
- 10.2307/2699986 · doi:10.2307/2699986
- Lipton, The Mythos of model interpretability, 96
- 10.1145/1540438.1540448 · DOI · Paper page
- Regulation, 2016, Regulation (EU) 2016/679 of the European parliament and of the council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing directive 95/46, Official J. Eur. Union, 59, 1
Cited by
Semi-Supervised Log-Based Anomaly Detection via Probabilistic Label Estimation
Yang, L., Chen, J., Wang, Z., Wang, W., Jiang, J., Dong, X., & Zhang, W. (2021) · 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE) · doi:10.1109/icse43902.2021.00130 · Cited by 244
Code Prediction by Feeding Trees to Transformers
Kim, S., Zhao, J., Tian, Y., & Chandra, S. (2021) · 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE) · doi:10.1109/icse43902.2021.00026 · Cited by 209
VulRepair: a T5-based automated software vulnerability repair
Fu, M., Tantithamthavorn, C., Le, T., Nguyen, V., & Phung, D. (2022) · Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3540250.3549098 · Cited by 207
Self-citationGiray, G. (2021) · Journal of Systems and Software · doi:10.1016/j.jss.2021.111031 · Cited by 187
JITLine: A Simpler, Better, Faster, Finer-grained Just-In-Time Defect Prediction
Pornprasit, C., & Tantithamthavorn, C. K. (2021) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr52588.2021.00049 · Cited by 185
Self-citation
Metadata
- Authors
- Jiarpakdee, J., Tantithamthavorn, C. K., Dam, H. K., & Grundy, J.
- Affiliations
- Monash University (AU)
- University of Wollongong (AU)
- Venue
- IEEE Transactions on Software Engineering
- Published
- 2022-01-01
- Volume
- 48
- Issue
- 1
- Pages
- 166-185
- ISSN
- 0098-5589 (ISSN-L)
- 1939-3520
- 2326-3881
- OpenAlex
- W3013745307
- Cited by
- 148
- References
- 103
- Citations by year
- 2026: 12 · 2025: 23 · 2024: 32 · 2023: 27 · 2022: 21
- FWCI
- 19.05
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Software Engineering (publishedVersion)
- Research Online (University of Wollongong) (submittedVersion)
- Funders
- Australian Research Council (DE200100941)
- ARC Laureate Fellowship (FL190100035)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Keywords
- fault prediction, model-agnostic explanation, lime, SHAP, breakdown, explainable AI, privacy, hyperparameter optimization, contrastive explanations, software analytics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.