CoChat
Sign Up
Cited by 24Software Testing and Debugging TechniquesSoftware Engineering ResearchSoftware Reliability and Analysis Research

Towards Better Graph Neural Network-Based Fault Localization through Enhanced Code Representation

Open access
Rafi, Md Nakhla, Kim, Dong Jae, Chen, An Ran, Chen, Tse-Hsun (Peter), Wang, Shaowei·2024·Proceedings of the ACM on Software Engineering
DOI 10.1145/3660793
Open this paper in CoChat — free account

Abstract

Automatic software fault localization plays an important role in software quality assurance by pinpointing faulty locations for easier debugging. Coverage-based fault localization is a commonly used technique, which applies statistics on coverage spectra to rank faulty code based on suspiciousness scores. However, statisticsbased approaches based on formulae are often rigid, which calls for learning-based techniques. Amongst all, Grace , a graph-neural network (GNN) based technique has achieved state-of-the-art due to its capacity to preserve coverage spectra, i.e., test-to-source coverage relationships, as precise abstract syntax-enhanced graph representation, mitigating the limitation of other learning-based technique which compresses the feature representation. However, such representation is not scalable due to the increasing complexity of software, correlating with increasing coverage spectra and AST graph, making it challenging to extend, let alone train the graph neural network in practice. In this work, we proposed a new graph representation, DepGraph , that reduces the complexity of the graph representation by 70 % in nodes and edges by integrating the interprocedural call graph in the graph representation of the code. Moreover, we integrate additional features—code change information—into the graph as attributes so the model can leverage rich historical project data. We evaluate DepGraph using Defects4j 2.0.0, and it outperforms Grace by locating 20 % more faults in Top-1 and improving the Mean First Rank (MFR) and the Mean Average Rank (MAR) by over 50 % while decreasing GPU memory usage by 44 % and training/inference time by 85 % . Additionally, in cross-project settings, DepGraph surpasses the state-of-the-art baseline with a 42 % higher Top-1 accuracy, and 68 % and 65 % improvement in MFR and MAR, respectively. Our study demonstrates DepGraph ’s robustness, achieving state-of-the-art accuracy and scalability for future extension and adoption.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3660793

Cites

5 of 5 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • DepGraph's standalone Top-K results are 299 (Top-1), 382 (Top-3), 415 (Top-5), and 449 (Top-10).

    “For instance, DepGraph 's Top-K results are higher: 299 (Top-1), 382 (Top-3), 415 (Top-5), and 449 (Top-10) compared to the results from LLM-based methods.”

    From Order Matters! An Empirical Study on Large Language Models' Input Order Bias in Software Fault Localization (2024) · cites this paper as [26]

    Section 5.1 (p. 13) reports 336 Top-1 faults for DepGraph without code-change information and says full DepGraph identifies 23 more, giving 359—not 299. The same section reports 588 Top-10 faults for the version without code-change information, already above the claimed 449. The cited paper therefore does not support these figures as DepGraph’s standalone results. (LLM, conf=0.88)

    Checked by a reasoning model — may be wrong.

    Experimental — accuracy not yet measured

  • DepGraph is the state-of-the-art supervised FL technique based on a graph neural network model.

    “Finally, for Learning-based ordering, we use DepGraph , which is the state-of-the-art supervised FL technique based on a graph neural network model [26] that transforms the rich static and dynamic code information into a graph structure.”

    From Order Matters! An Empirical Study on Large Language Models' Input Order Bias in Software Fault Localization (2024) · cites this paper as [26]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 49

  1. [1][1] Rui Abreu, Peter Zoeteweij, and Arjan JC Van Gemund. 2006. An evaluation of similarity coefficients for software fault localization. In 2006 12th Pacific Rim International Symposium on Dependable Computing (PRDC'06) . IEEE, 39-46. -
  2. [2][2] Rui Abreu, Peter Zoeteweij, and Arjan JC Van Gemund. 2009. Spectrum-based multiple fault localization. In 2009 IEEE/ACM International Conference on Automated Software Engineering . IEEE, 88-99. -
  3. [3][3] Saad Albawi, Tareq Abed Mohammed, and Saad Al-Zawi. 2017. Understanding of a convolutional neural network. In 2017 international conference on engineering and technology (ICET) . Ieee, 1-6. -
  4. [5][5] Tien-Duy B. Le, David Lo, Claire Le Goues, and Lars Grunske. 2016. A learning-to-rank based fault localization approach using likely invariants. In Proceedings of the 25th international symposium on software testing and analysis . 177-188. -
  5. [6][6] Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. 2016. Layer normalization. arXiv preprint arXiv:1607.06450 (2016). - · arXiv
  6. [7][7] Samuel Benton, Xia Li, Yiling Lou, and Lingming Zhang. 2020. On the effectiveness of unified debugging: An extensive study on 16 program repair systems. In Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering . 907-918. -
  7. [9][9] An Ran Chen, Tse-Hsun Chen, and Junjie Chen. 2022. How Useful is Code Change Information for Fault Localization in Continuous Integration?. In Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering . 1-12. -
  8. [10][10] Zhanqi Cui, Minghua Jia, Xiang Chen, Liwei Zheng, and Xiulei Liu. 2020. Improving software fault localization by combining spectrum and mutation. IEEE Access 8 (2020), 172296-172307. -
  9. [11][11] Rahul Dey and Fathi M Salem. 2017. Gate-variants of gated recurrent unit (GRU) neural networks. In 2017 IEEE 60th international midwest symposium on circuits and systems (MWSCAS) . IEEE, 1597-1600. -
  10. [12][12] Arpita Dutta and Sangharatna Godboley. 2021. Msfl: A model for fault localization using mutation-spectra technique. In Lean and Agile Software Development: 5th International Conference, LASD 2021, Virtual Event, January 23, 2021, Proceedings 5 . Springer, 156-173. -
  11. [17][17] James A Jones, Mary Jean Harrold, and John Stasko. 2002. Visualization of test information to assist fault localization. In Proceedings of the 24th international conference on Software engineering . 467-477. -
  12. [19][19] Pavneet Singh Kochhar, Xin Xia, David Lo, and Shanping Li. 2016. Practitioners' expectations on automated fault localization. In Proceedings of the 25th international symposium on software testing and analysis . 165-176. -
  13. [20][20] Tien-Duy B Le, Richard J Oentaryo, and David Lo. 2015. Information retrieval and spectrum based bug localization: Better together. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering . 579-590. -
  14. [21][21] Tien-Duy B Le, Ferdian Thung, and David Lo. 2013. Theory and practice, do they match? a case with spectrum-based fault localization. In 2013 IEEE International Conference on Software Maintenance . IEEE, 380-383. -
  15. [23][23] Xia Li and Lingming Zhang. 2017. Transforming programs and tests in tandem for fault localization. Proceedings of the ACM on Programming Languages 1, OOPSLA (2017), 1-30. -
  16. [24][24] Yujia Li, Daniel Tarlow, Marc Brockschmidt, and Richard Zemel. 2015. Gated graph sequence neural networks. arXiv preprint arXiv:1511.05493 (2015). - · arXiv
  17. [25][25] Yi Li, Shaohua Wang, and Tien Nguyen. 2021. Fault localization with code coverage representation learning. In 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE) . IEEE, 661-673. -
  18. [26][26] Zhenhao Li, Heng Li, Tse-Hsun Chen, and Weiyi Shang. 2021. DeepLV: Suggesting log levels using ordinal based neural networks. In 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE) . IEEE, 1461-1472. -
  19. [27][27] Jiahao Liu, Jun Zeng, Xiang Wang, Kaihang Ji, and Zhenkai Liang. 2022. Tell: log level suggestions via modeling multi-level code block information. In Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis . 27-38. -
  20. [28][28] Yiling Lou, Ali Ghanbari, Xia Li, Lingming Zhang, Haotian Zhang, Dan Hao, and Lu Zhang. 2020. Can automated program repair refine fault localization? a unified debugging approach. In Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis . 75-87. -
  21. [29][29] Yiling Lou, Qihao Zhu, Jinhao Dong, Xia Li, Zeyu Sun, Dan Hao, Lu Zhang, and Lingming Zhang. 2021. Boosting coverage-based fault localization via graph-based representation learning. In Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering . 664-676. -
  22. [31][31] Seokhyeon Moon, Yunho Kim, Moonzoo Kim, and Shin Yoo. 2014. Ask the mutants: Mutating faulty programs for fault localization. In 2014 IEEE Seventh International Conference on Software Testing, Verification and Validation . IEEE, 153-162. -
  23. [32][32] Nachiappan Nagappan and Thomas Ball. 2005. Use of relative code churn measures to predict system defect density. In Proceedings of the 27th international conference on Software engineering . 284-292. -
  24. [33][33] Thanh-Dat Nguyen, Thanh Le-Cong, Duc-Minh Luong, Van-Hai Duong, Xuan-Bach D Le, David Lo, and Quyet-Thang Huynh. 2022. FFL: Fine-grained Fault Localization for Student Programs via Syntactic and Semantic Reasoning. In 2022 IEEE International Conference on Software Maintenance and Evolution (ICSME) . IEEE, 151-162. -
  25. [34][34] Suphakit Niwattanakul, Jatsada Singthongchai, Ekkachai Naenudorn, and Supachanun Wanapu. 2013. Using of Jaccard coefficient for keywords similarity. In Proceedings of the international multiconference of engineers and computer scientists , Vol. 1. 380-384. -
  26. [35][35] Mike Papadakis and Yves Le Traon. 2015. Metallaxis-FL: mutation-based fault localization. Software Testing, Verification and Reliability 25, 5-7 (2015), 605-628. -
  27. [36][36] Chris Parnin and Alessandro Orso. 2011. Are automated debugging techniques actually helping programmers?. In Proceedings of the 2011 international symposium on software testing and analysis . 199-209. -
  28. [37][37] Jie Qian, Xiaolin Ju, and Xiang Chen. 2023. GNet4FL: effective fault localization via graph convolutional neural network. Automated Software Engineering 30, 2 (2023), 16. -
  29. [38][38] Jie Qian, Xiaolin Ju, Xiang Chen, Hao Shen, and Yiheng Shen. 2021. AGFL: a graph convolutional neural network-based method for fault localization. In 2021 IEEE 21st International Conference on Software Quality, Reliability and Security (QRS) . IEEE, 672-680. -
  30. [39][39] Jeongju Sohn and Shin Yoo. 2017. Fluccs: Using code and change metrics to improve fault localization. In Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis . 273-283. -
  31. [42][42] Béla Vancsics, Ferenc Horváth, Attila Szatmári, and Arpád Beszédes. 2021. Call frequency-based fault localization. In 2021 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) . IEEE, 365-376. -
  32. [43][43] Wenhan Wang, Ge Li, Bo Ma, Xin Xia, and Zhi Jin. 2020. Detecting code clones with graph neural network and flow-augmented abstract syntax tree. In 2020 IEEE 27th International Conference on Software Analysis, Evolution and Reengineering (SANER) . IEEE, 261-271. -
  33. [44][44] Ming Wen, Junjie Chen, Yongqiang Tian, Rongxin Wu, Dan Hao, Shi Han, and Shing-Chi Cheung. 2019. Historical spectrum based fault localization. IEEE Transactions on Software Engineering 47, 11 (2019), 2348-2368. -
  34. [45][45] W Eric Wong, Vidroha Debroy, Ruizhi Gao, and Yihao Li. 2013. The DStar method for effective software fault localization. IEEE Transactions on Reliability 63, 1 (2013), 290-308. -
  35. [46][46] WEric Wong, Vidroha Debroy, Richard Golden, Xiaofeng Xu, and Bhavani Thuraisingham. 2011. Effective software fault localization using an RBF neural network. IEEE Transactions on Reliability 61, 1 (2011), 149-169. -
  36. [47][47] WEric Wong, Ruizhi Gao, Yihao Li, Rui Abreu, and Franz Wotawa. 2016. A survey on software fault localization. IEEE Transactions on Software Engineering 42, 8 (2016), 707-740. -
  37. [48][48] WEric Wong and Yu Qi. 2009. BP neural network-based effective fault localization. International Journal of Software Engineering and Knowledge Engineering 19, 04 (2009), 573-597. -
  38. [49][49] Shumei Wu, Zheng Li, Yong Liu, Xiang Chen, and Mingyu Li. 2023. GMBFL: Optimizing Mutation-Based Fault Localization via Graph Representation. In 2023 IEEE International Conference on Software Maintenance and Evolution (ICSME) . IEEE, 245-257. -
  39. [50][50] Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip. 2020. A comprehensive survey on graph neural networks. IEEE transactions on neural networks and learning systems 32, 1 (2020), 4-24. -
  40. [51][51] Xiaoyuan Xie, Tsong Yueh Chen, Fei-Ching Kuo, and Baowen Xu. 2013. A theoretical analysis of the risk evaluation formulas for spectrum-based fault localization. ACM Transactions on software engineering and methodology (TOSEM) 22, 4 (2013), 1-40. -
  41. [52][52] Xiaoyuan Xie, Zicong Liu, Shuo Song, Zhenyu Chen, Jifeng Xuan, and Baowen Xu. 2016. Revisit of automatic debugging via human focus-tracking analysis. In Proceedings of the 38th International Conference on Software Engineering . 808-819. -
  42. [53][53] Jiaxi Xu, Fei Wang, and Jun Ai. 2020. Defect prediction with semantics and context features of codes based on graph representation learning. IEEE Transactions on Reliability 70, 2 (2020), 613-625. -
  43. [54][54] Xuezheng Xu, Changwei Zou, and Jingling Xue. 2020. Every mutation should be rewarded: Boosting fault localization with mutated predicates. In 2020 IEEE International Conference on Software Maintenance and Evolution (ICSME) . IEEE, 196-207. -
  44. [55][55] Mengshi Zhang, Xia Li, Lingming Zhang, and Sarfraz Khurshid. 2017. Boosting spectrum-based fault localization using pagerank. In Proceedings of the 26th ACM SIGSOFT international symposium on software testing and analysis . 261-272. -
  45. [56][56] Mengshi Zhang, Yaoxian Li, Xia Li, Lingchao Chen, Yuqun Zhang, Lingming Zhang, and Sarfraz Khurshid. 2019. An empirical study of boosting spectrum-based fault localization via pagerank. IEEE Transactions on Software Engineering 47, 6 (2019), 1089-1113. -
  46. [57][57] Zhuo Zhang, Yan Lei, Xiaoguang Mao, and Panpan Li. 2019. CNN-FL: An effective approach for localizing faults using convolutional neural networks. In 2019 IEEE 26th International Conference on Software Analysis, Evolution and Reengineering (SANER) . IEEE, 445-455. -
  47. [60][60] Zhe Zhou, Cong Li, Xuechao Wei, Xiaoyang Wang, and Guangyu Sun. 2022. Gnnear: Accelerating full-batch training of graph neural networks with near-memory processing. In Proceedings of the International Conference on Parallel Architectures and Compilation Techniques . 54-68. -
  48. [61][61] Thomas Zimmermann, Rahul Premraj, and Andreas Zeller. 2007. Predicting defects for eclipse. In Third International Workshop on Predictor Models in Software Engineering (PROMISE'07: ICSE Workshops 2007) . IEEE, 9-9. -
  49. [62][62] Daming Zou, Jingjing Liang, Yingfei Xiong, Michael D Ernst, and Lu Zhang. 2019. An empirical study of fault localization families and their combinations. IEEE Transactions on Software Engineering 47, 2 (2019), 332-347. Received 20 February 2007; revised 12 March 2009; accepted 5 June 2009

Cited by

Sorted by citations

Authors
Rafi, M. N., Kim, D. J., Chen, A. R., Chen, T.-H. (., & Wang, S.
Affiliations
  • Concordia University (CA)
  • DePaul University (US)
  • University of Alberta (CA)
  • University of Manitoba (CA)
ORCID
Venue
Proceedings of the ACM on Software Engineering
Publisher
Association for Computing Machinery (ACM)
Published
2024-07-12
Volume
1
Issue
FSE
Pages
1937-1959
ISSN
2994-970X (ISSN-L)
DOI
10.1145/3660793
OpenAlex
W4400582478
Cited by
24
References
5
Citations by year
2026: 12 · 2025: 11 · 2024: 1
FWCI
24.61
Type
journal article
Language
en
Open access
Licence
Locations
  • Proceedings of the ACM on software engineering. (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Research
  • Physical Sciences › Computer Science › Software › Software Reliability and Analysis Research
Keywords
software fault localization, graph neural networks, abstract syntax tree, cross-project defect prediction, scalability, Defects4J

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review