A Survey on Software Fault Localization
The DOI matches its Crossref record; 436 of the 451 references it cites were found in a registry or an index.
Abstract
Software fault localization, the act of identifying the locations of faults in a program, is widely recognized to be one of the most tedious, time consuming, and expensive - yet equally critical - activities in program debugging. Due to the increasing scale and complexity of software today, manually locating faults when failures occur is rapidly becoming infeasible, and consequently, there is a strong demand for techniques that can guide software developers to the locations of faults in a program with minimal human intervention. This demand in turn has fueled the proposal and development of a broad spectrum of fault localization techniques, each of which aims to streamline the fault localization process and make it more effective by attacking the problem in a unique way. In this article, we catalog and provide a comprehensive overview of such techniques and discuss key issues and concerns that are pertinent to software fault localization as a whole.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Also, Steinder et al. [21], Wong et al. [24], and Solé et al. [25] described the process of root-cause identification but did not focus on the microservices architecture.”
From Anomaly detection and root-cause identification in microservices: a survey (2026) · cites this paper as [24]
Citations
Cites
Cousot, P., & Cousot, R. (1977) · Proceedings of the 4th ACM SIGACT-SIGPLAN symposium on Principles of programming languages - POPL '77 · doi:10.1145/512950.512973 · Cited by 6,279 · cited 1×
“lem, Mayer and Stumptner [246] propose an abstractionbased model in which abstract interpretation [55], [78] is applied to handle loops, recursive procedures, and heap data structures.”
BackgroundBasic concepts and taxonomy of dependable and secure computing
Avizienis, A., Laprie, J.-C., Randell, B., & Landwehr, C. (2004) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2004.2 · Cited by 5,223 · cited 1×
“The following terms appear repeatedly throughout this article, and thus for convenience, we provide definitions for them here per the taxonomy provided in [37]: A failure is when a service deviates from its correct behavior.”
BackgroundCAUSALITY: MODELS, REASONING AND INFERENCE
Hitchcock, C. (2001) · The Philosophical Review · doi:10.1215/00318108-110-4-639 · Cited by 4,741
Weiser, M. (1984) · IEEE Transactions on Software Engineering · doi:10.1109/tse.1984.5010248 · Cited by 3,204
A theory of diagnosis from first principles
Reiter, R. (1987) · Artificial Intelligence · doi:10.1016/0004-3702(87)90062-2 · Cited by 3,031 · cited 1×
“While using model-based diagnosis [301], it is assumed that a correct model of each program being diagnosed is available.”
MethodFundamentals of neural networks: architectures, algorithms, and applications
Fausett, L. V. (1994) · CERN Document Server (European Organization for Nuclear Research) · Cited by 3,025
Causality: models, reasoning, and inference
(2000) · Choice Reviews Online · doi:10.5860/choice.38-1494a · Cited by 2,873
Theory of the backpropagation neural network
Hecht-Nielsen (1989) · International Joint Conference on Neural Networks · doi:10.1109/ijcnn.1989.118638 · Cited by 2,222
de Kleer, J., & Williams, B. C. (1987) · Artificial Intelligence · doi:10.1016/0004-3702(87)90063-4 · Cited by 2,039
Lewis, D. (1973) · The Journal of Philosophy · doi:10.2307/2025310 · Cited by 1,914
Eraser: a dynamic data race detector for multithreaded programs
Savage, S., Burrows, M., Nelson, G., Sobalvarro, P., & Anderson, T. (1997) · ACM Transactions on Computer Systems · doi:10.1145/265924.265927 · Cited by 1,572
Empirical evaluation of the tarantula automatic fault-localization technique
Jones, J. A., & Harrold, M. J. (2005) · Proceedings of the 20th IEEE/ACM International Conference on Automated Software Engineering · doi:10.1145/1101908.1101949 · Cited by 1,238 · cited 10×
“A study on the Siemens suite [185] shows that Tarantula inspects less code before the first faulty state-”
BackgroundMethodInfluentialAdvanced methods in neural computing
Wasserman, P. D. (1989) · Medical Entomology and Zoology · Cited by 1,105
Shapiro, E. Y. (1983) · doi:10.7551/mitpress/1192.001.0001 · Cited by 1,085 · cited 1×
“Algorithmic debugging (also called declarative debugging), first discussed in Shapiro’s dissertation [325] with more details in [328], [402], decomposes a complex computation into a series of sub-computations to help locate program bugs.”
MethodSimplifying and isolating failure-inducing input
Zeller, A., & Hildebrandt, R. (2002) · IEEE Transactions on Software Engineering · doi:10.1109/32.988498 · Cited by 1,079 · cited 2×
“and Hildebrandt propose a technique, delta debugging [400], [401], by contrasting program states between executions of a successful test and a failed test via their memory graphs which are described in [426].”
BackgroundMethodDynamically discovering likely program invariants to support program evolution
Ernst, M. D., Cockrell, J., Griswold, W. G., & Notkin, D. (2001) · IEEE Transactions on Software Engineering · doi:10.1109/32.908957 · Cited by 1,050
A Survey of Program Slicing Techniques.
Tip, F. (1994) · Centrum Wiskunde & Informatica (CWI), the national research institute for mathematics and computer science in the Netherlands · Cited by 952
Visualization of test information to assist fault localization
Jones, J. A., Harrold, M. J., & Stasko, J. (2002) · Proceedings/Proceedings - International Conference on Software Engineering · doi:10.1145/581339.581397 · Cited by 904
Agrawal, H., & Horgan, J. R. (1990) · Proceedings of the ACM SIGPLAN 1990 conference on Programming language design and implementation - PLDI '90 · doi:10.1145/93542.93576 · Cited by 867 · cited 1×
“To exclude such extra statements from a dice (as well as a slice), we need to use dynamic slicing [20], [202] instead of static slicing, as the former can identify the statements that do affect a particular value observed at a particular location, rather than possibly affecting such a value as with…”
BackgroundA Survey of Binary Similarity and Distance Measures
Choi, S.-S., Cha, S.-H., & Tappert, C. C. (2010) · Cited by 843
Is mutation an appropriate tool for testing experiments?
Andrews, J. H., Briand, L. C., & Labiche, Y. (2005) · Proceedings of the 27th international conference on Software engineering - ICSE '05 · doi:10.1145/1062455.1062530 · Cited by 777 · cited 1×
“Although mutation has been shown to be an effective approach to simulate realistic faults [29], [103], [223], [268], some real-life bugs are very delicate and cannot be modeled by simple first-order mutants.”
BackgroundKorel, B., & Laski, J. (1988) · Information Processing Letters · doi:10.1016/0020-0190(88)90054-3 · Cited by 759
On the Accuracy of Spectrum-based Fault Localization
Abreu, R., Zoeteweij, P., & van Gemund, A. J. C. (2007) · Testing: Academic and Industrial Conference Practice and Research Techniques - MUTATION (TAICPART-MUTATION 2007) · doi:10.1109/taic.part.2007.13 · Cited by 736 · cited 3×
“Many studies that evaluate the effectiveness of specific fault localization techniques have been reported [8], [10], [11], [12], [25], [31], [36], [49], [53], [91], [93], [94], [102], [124], [178], [185], [191], [207], [209], [253], [266], [267], [296], [299], [335], [366], [390], [391], [393],…”
BackgroundMethodSelf-citationScalable statistical bug isolation
Liblit, B., Naik, M., Zheng, A. X., Aiken, A., & Jordan, M. I. (2005) · ACM SIGPLAN Notices · doi:10.1145/1064978.1065014 · Cited by 729
RacerX: effective, static detection of race conditions and deadlocks
Engler, D., & Ashcraft, K. (2003) · ACM SIGOPS Operating Systems Review · doi:10.1145/1165389.945468 · Cited by 661
Fault Localization with Nearest Neighbor Queries
Renieris, M., & Reiss, S. P. (2003) · Cited by 582
Are automated debugging techniques actually helping programmers?
Parnin, C., & Orso, A. (2011) · Proceedings of the 2011 International Symposium on Software Testing and Analysis · doi:10.1145/2001420.2001445 · Cited by 576 · cited 2×
“Last but not least, it is important to realize that software fault localization techniques should not be evaluated only in terms of effectiveness as described above [285].”
BackgroundAn Evaluation of Similarity Coefficients for Software Fault Localization
Abreu, R., Zoeteweij, P., & Van Gemund, A. (2006) · 2006 12th Pacific Rim International Symposium on Dependable Computing (PRDC'06) · doi:10.1109/prdc.2006.18 · Cited by 563 · cited 2×
“specific fault localization techniques have been reported [8], [10], [11], [12], [25], [31], [36], [49], [53], [91], [93], [94], [102], [124], [178], [185], [191], [207], [209], [253], [266], [267], [296], [299], [335], [366], [390], [391], [393], [420], [406], [410], [424].”
MethodSelf-citationLocating causes of program failures
Cleve, H., & Zeller, A. (2005) · Proceedings of the 27th international conference on Software engineering - ICSE '05 · doi:10.1145/1062455.1062522 · Cited by 550 · cited 5×
“…using the ratio of the number of nodes in its smallest dependency sphere to the number of nodes in the entire PDG: The use of T-score requires that programmers are able to distinguish defects from non-defects at each location and can do so at the same cost for each location considered [77].”
BackgroundResultInfluentialFinding and Reproducing Heisenbugs in Concurrent Programs
Musuvathi, M., Qadeer, S., Nainar, P. A., Ball, T., Basler, G., & Neamtiu, I. (2008) · Cited by 541
A model for spectra-based software diagnosis
Naish, L., Lee, H. J., & Ramamohanarao, K. (2011) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/2000791.2000795 · Cited by 536 · cited 5×
“Ochiai2 [267] is an extension of Ochiai, and its formula is as follows:”
BackgroundInfluentialIsolating cause-effect chains from computer programs
Zeller, A. (2002) · Proceedings of the 10th ACM SIGSOFT symposium on Foundations of software engineering · doi:10.1145/587051.587053 · Cited by 530 · cited 4×
“The theory of coun-terfactual reasoning is also found in other studies such as [137], [179], [400].”
BackgroundMethodResultInfluentialVisualization of test information to assist fault localization
Jones, J. A., Harrold, M. J., & Stasko, J. (2002) · Proceedings of the 24th international conference on Software engineering - ICSE '02 · doi:10.1145/581396.581397 · Cited by 524 · cited 2×
“Based on the suspiciousness computed by Tarantula, studies like [186], [187] use different colors (from red to yellow to green) to provide a visual mapping of the participation of each program statement in the execution of a test suite.”
MethodZhou, J., Zhang, H., & Lo, D. (2012) · 2012 34th International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2012.6227210 · Cited by 507 · cited 1×
“Recently, some studies [85], [234], [298], [315], [425] have applied information retrieval techniques to software fault localization.”
ResultA practical evaluation of spectrum-based fault localization
Abreu, R., Zoeteweij, P., Golsteijn, R., & van Gemund, A. J. C. (2009) · Journal of Systems and Software · doi:10.1016/j.jss.2009.06.035 · Cited by 495 · cited 1×
“Many studies that evaluate the effectiveness of specific fault localization techniques have been reported [8], [10], [11], [12], [25], [31], [36], [49], [53], [91], [93], [94], [102], [124], [178], [185], [191], [207], [209], [253], [266], [267], [296], [299], [335], [366], [390], [391], [393],…”
Self-citationUsing Mutation Analysis for Assessing and Comparing Testing Coverage Criteria
Andrews, J. H., Briand, L. C., Labiche, Y., & Namin, A. S. (2006) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2006.83 · Cited by 481 · cited 1×
“Although mutation has been shown to be an effective approach to simulate realistic faults [29], [103], [223], [268], some real-life bugs are very delicate and cannot be modeled by simple first-order mutants.”
BackgroundThe DStar Method for Effective Software Fault Localization
Wong, W. E., Debroy, V., Gao, R., & Li, Y. (2014) · IEEE Transactions on Reliability · doi:10.1109/tr.2013.2285319 · Cited by 479 · cited 10×
“Tool for collecting execution trace information for C programs via author [22], [366], [367], [369], [370], [371], [372], [373], [374] x Vue Heuristics involving the control graph, execution trace, and the maintainer’s knowledge to help locate features and identify feature interactions via author…”
BackgroundMethodResultInfluentialSelf-citationScalable statistical bug isolation
Liblit, B., Naik, M., Zheng, A. X., Aiken, A., & Jordan, M. I. (2005) · Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation · doi:10.1145/1065010.1065014 · Cited by 472 · cited 3×
“A statistical debugging technique (Liblit05) that can isolate bugs in programs with instrumented predicates at particular points is presented in [222].”
MethodArisholm, E., Briand, L. C., & Johannessen, E. B. (2010) · Journal of Systems and Software · doi:10.1016/j.jss.2009.06.055 · Cited by 455 · cited 1×
“Many studies that evaluate the effectiveness of specific fault localization techniques have been reported [8], [10], [11], [12], [25], [31], [36], [49], [53], [91], [93], [94], [102], [124], [178], [185], [191], [207], [209], [253], [266], [267], [296], [299], [335], [366], [390], [391], [393],…”
The detection of fault-prone programs
Munson, J. C., & Khoshgoftaar, T. M. (1992) · IEEE Transactions on Software Engineering · doi:10.1109/32.135775 · Cited by 442 · cited 1×
“Unfortunately, this has also resulted in more software bugs, which often lead to execution failures with huge losses [260], [275], [365].”
BackgroundReinhart, C. M., & Rogoff, K. S. (2010) · American Economic Review · doi:10.1257/aer.100.2.573 · Cited by 438
Effective static race detection for Java
Naik, M., Aiken, A., & Whaley, J. (2006) · Proceedings of the 27th ACM SIGPLAN Conference on Programming Language Design and Implementation · doi:10.1145/1133981.1134018 · Cited by 431 · cited 1×
“Tools such as Chord [262] and RacerX [106] can statically analyze a program to find concurrency bugs.”
Fault localization with nearest neighbor queries
Renieres, M., & Reiss, S. P. (2004) · 18th IEEE International Conference on Automated Software Engineering, 2003. Proceedings. · doi:10.1109/ase.2003.1240292 · Cited by 427
Atomizer: a dynamic atomicity checker for multithreaded programs
Flanagan, C., & Freund, S. N. (2004) · Proceedings of the 31st ACM SIGPLAN-SIGACT symposium on Principles of programming languages · doi:10.1145/964001.964023 · Cited by 409 · cited 2×
“Concurrent programs suffer most from three kinds of access anomalies: data race [32], [321], atomicity violation [110], [113], [115], and atomic-set serializability violations [24], [44].”
BackgroundEffective static race detection for Java
Naik, M., Aiken, A., & Whaley, J. (2006) · ACM SIGPLAN Notices · doi:10.1145/1133255.1134018 · Cited by 408
Spectrum-Based Multiple Fault Localization
Abreu, R., Zoeteweij, P., & van Gemund, A. J. C. (2009) · 2009 IEEE/ACM International Conference on Automated Software Engineering · doi:10.1109/ase.2009.25 · Cited by 401 · cited 4×
“A potential weakness of most techniques based on Bayesian reasoning (e.g., [14], [64], [193]) is that they all assume program components fail independently; in other words, interferences between multiple bugs are ignored, which is not necessarily the case in practice.”
BackgroundMethodInfluentialSelf-citationSemFix: program repair via semantic analysis
Nguyen, H. D. T., Qi, D., Roychoudhury, A., & Chandra, S. (2013) · doi:10.5555/2486788.2486890 · Cited by 398
SemFix: Program repair via semantic analysis
Nguyen, H. D. T., Qi, D., Roychoudhury, A., & Chandra, S. (2013) · 2013 35th International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2013.6606623 · Cited by 395 · cited 1×
“Last but not least, the idea of modifying a program so that test cases that fail on the original program can be executed successfully on the modified program [133], [134], [200] is also used in other studies for automatic bug fixing [95], [152], [189], [270].”
Background(2006) · doi:10.1016/b978-1-55860-866-5.x5000-0 · Cited by 392
Improving bug localization using structured information retrieval
Saha, R. K., Lease, M., Khurshid, S., & Perry, D. E. (2013) · 2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2013.6693093 · Cited by 390 · cited 1×
“Recently, some studies [85], [234], [298], [315], [425] have applied information retrieval techniques to software fault localization.”
ResultZhou, J. H., Zhang, H., & Lo, D. (2012) · doi:10.5555/2337223.2337226 · Cited by 372
Is mutation an appropriate tool for testing experiments?
Andrews, J. H., Brand, L. C., & Labiche, Y. (2005) · Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005. · doi:10.1109/icse.2005.1553583 · Cited by 368
Optimally profiling and tracing programs
Ball, T., & Larus, J. R. (1994) · ACM Transactions on Programming Languages and Systems · doi:10.1145/183432.183527 · Cited by 362 · cited 1×
“, [43]); Identifying memory leaks or code that performs unexpectedly poorly (e.”
BackgroundExpertise in debugging computer programs: A process analysis
Vessey, I. (1985) · International Journal of Man-Machine Studies · doi:10.1016/s0020-7373(85)80054-7 · Cited by 359 · cited 1×
“, identifying the locations of faults, has historically been a manual task that has been recognized to be time consuming and tedious as well as prohibitively expensive [347], given the size and complexity of large-scale software systems today.”
BackgroundA theoretical analysis of the risk evaluation formulas for spectrum-based fault localization
Xie, X., Chen, T. Y., Kuo, F.-C., & Xu, B. (2013) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/2522920.2522924 · Cited by 353
Weiser, M. (1979) · Deep Blue (University of Michigan) · doi:10.7302/11363 · Cited by 341
CTrigger: exposing atomicity violation bugs from their hiding places
Park, S., Lu, S., & Zhou, Y. (2009) · Proceedings of the 14th international conference on Architectural support for programming languages and operating systems · doi:10.1145/1508244.1508249 · Cited by 339 · cited 1×
“Park et al. apply a CTrigger testing framework [282] to detect real atomicity violations by controlling the program execution to exercise low-probability thread inter-leavings.”
BackgroundA practical approach to programming with assertions
Rosenblum, D. S. (1995) · IEEE Transactions on Software Engineering · doi:10.1109/32.341844 · Cited by 334 · cited 1×
“More details of using assertions for program debugging can be found in [309], [310].”
BackgroundTheory of the Back Propagation Neural Network
Hecht-Nielsen, R. (1989) · International Joint Conference on Neural Networks (IJCNN) · Cited by 324
Automated support for classifying software failure reports
Podgurski, A., Leon, D., Francis, P., Masri, W., & Minch, M. (2003) · Cited by 314
A brief survey of program slicing
Xu, B., Qian, J., Zhang, X., Wu, Z., & Chen, L. (2005) · ACM SIGSOFT Software Engineering Notes · doi:10.1145/1050849.1050865 · Cited by 313 · cited 1×
“Hundreds of papers on this topic have been published [52], [344], [394] since Weiser first proposed static slicing in 1979 [361].”
BackgroundDebugging with dynamic slicing and backtracking
Agrawal, H., Demillo, R. A., & Spafford, E. H. (1993) · Software: Practice and Experience · doi:10.1002/spe.4380230603 · Cited by 311 · cited 1×
“Studies such as [18], [24], [28], [96], [104], [188], [192], [201], [219], [227], [237], [257], [277], [297], [334], [356], [378], [379], [406], [407], [410], which use the dynamic slicing concept in program debugging, have been reported.”
Debugging reinvented: asking and answering why and why not questions about program behavior
Ko, A. J., & Myers, B. A. (2008) · Proceedings of the 30th international conference on Software engineering · doi:10.1145/1368088.1368130 · Cited by 296
From symptom to cause: localizing errors in counterexample traces
Ball, T., Naik, M., & Rajamani, S. K. (2003) · Proceedings of the 30th ACM SIGPLAN-SIGACT symposium on Principles of programming languages · doi:10.1145/604131.604140 · Cited by 294 · cited 7×
“We now discuss model checking-based fault localization techniques that rely on the use of model checkers to locate bugs [44], [67], [133], [134], [136], [137], [138], [200].”
BackgroundMethodInfluentialFault localization using execution slices and dataflow tests
Agrawal, H., Horgan, J. R., London, S., & Wong, W. E. (2002) · Proceedings of Sixth International Symposium on Software Reliability Engineering. ISSRE'95 · doi:10.1109/issre.1995.497652 · Cited by 288 · cited 6×
“for spectrum-based fault localization, though this approach has subsequently been deemed ineffective [21], [185], [366].”
BackgroundMethodInfluentialSelf-citationStatistical Debugging: A Hypothesis Testing-Based Approach
Liu, C., Fei, L., Yan, X., Han, J., & Midkiff, S. P. (2006) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2006.105 · Cited by 288 · cited 6×
“The T-score [223], [303] estimates the percentage of code a programmer need not examine before the first faulty location is found.”
BackgroundMethodInfluentialOn the Use of Mutation Faults in Empirical Assessments of Test Case Prioritization Techniques
Do, H., & Rothermel, G. (2006) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2006.92 · Cited by 287 · cited 1×
“Although mutation has been shown to be an effective approach to simulate realistic faults [29], [103], [223], [268], some real-life bugs are very delicate and cannot be modeled by simple first-order mutants.”
BackgroundOn the value of static analysis for fault detection in software
Zheng, J., Williams, L., Nagappan, N., Snipes, W., Hudepohl, J. P., & Vouk, M. A. (2006) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2006.38 · Cited by 283 · cited 1×
“…the effectiveness of specific fault localization techniques have been reported [8], [10], [11], [12], [25], [31], [36], [49], [53], [91], [93], [94], [102], [124], [178], [185], [191], [207], [209], [253], [266], [267], [296], [299], [335], [366], [390], [391], [393], [420], [406], [410], [424].”
Locating faults through automated predicate switching
Zhang, X., Gupta, N., & Gupta, R. (2006) · Proceedings of the 28th international conference on Software engineering · doi:10.1145/1134285.1134324 · Cited by 277 · cited 1×
“Predicate switching [405] [352] present a similar technique that analyzes the execution path of a failed test and alters the outcome of branches in that path to produce a successful execution.”
BackgroundLightweight fault-localization using multiple coverage types
Santelices, R., Jones, J. A., Yu, Y., & Harrold, M. J. (2009) · 2009 IEEE 31st International Conference on Software Engineering · doi:10.1109/icse.2009.5070508 · Cited by 264 · cited 1×
“Furthermore, in [320], Santelices et al. study the fault localization effectiveness of Tarantula using three types of program coverage—statements, branches, and define-use pair.”
Two notions of correctness and their relation to testing
Budd, T. A., & Angluin, D. (1982) · Acta Informatica · doi:10.1007/bf00625279 · Cited by 260 · cited 1×
“The concept of coincidental correctness, introduced by Budd and Angluin in [59], discusses the circumstances under which a test case produces one or more errors in the program state but the output of the program is still correct.”
BackgroundRobust radial basis function neural networks
Lee, C.-C., Chung, P.-C., Tsai, J.-R., & Chang, C.-I. (1999) · IEEE Transactions on Systems, Man and Cybernetics, Part B (Cybernetics) · doi:10.1109/3477.809023 · Cited by 256 · cited 1×
“As BP neural networks are known to suffer from issues such as paralysis and local minima, Wong et al. [367] propose another approach based on radial basis function (RBF) networks, which are less susceptible to these problems and have a faster learning rate [211], [357].”
BackgroundRao, S., & Kak, A. (2011) · Proceedings of the 8th Working Conference on Mining Software Repositories · doi:10.1145/1985441.1985451 · Cited by 252 · cited 1×
“Recently, some studies [85], [234], [298], [315], [425] have applied information retrieval techniques to software fault localization.”
ResultFinding failures by cluster analysis of execution profiles
Dickinson, W. T., De, L., & Fodgurski, A. (2001) · doi:10.5555/381473.381509 · Cited by 252
Low-overhead memory leak detection using adaptive statistical profiling
Hauswirth, M., & Chilimbi, T. M. (2004) · Proceedings of the 11th international conference on Architectural support for programming languages and operating systems · doi:10.1145/1024393.1024412 · Cited by 246 · cited 1×
“, [150]);”
HOLMES: Effective statistical debugging via efficient path profiling
Chilimbi, T. M., Liblit, B., Mehra, K., Nori, A. V., & Vaswani, K. (2009) · 2009 IEEE 31st International Conference on Software Engineering · doi:10.1109/icse.2009.5070506 · Cited by 246 · cited 3×
“(from Microsoft) [74]”
BackgroundInfluentialJones, J. A., Bowring, J. F., & Harrold, M. J. (2007) · Proceedings of the 2007 international symposium on Software testing and analysis · doi:10.1145/1273463.1273468 · Cited by 245 · cited 2×
“In [184], Jones et al. suggest that multiple bugs in a program can be located in parallel.”
BackgroundSridharan, M., Fink, S. J., & Bodik, R. (2007) · Proceedings of the 28th ACM SIGPLAN Conference on Programming Language Design and Implementation · doi:10.1145/1250734.1250748 · Cited by 243 · cited 1×
“Authors of [330] propose thin slicing in order to find only producer statements that help compute and”
BackgroundSource Code Retrieval for Bug Localization Using Latent Dirichlet Allocation
Lukins, S. K., Kraft, N. A., & Etzkorn, L. H. (2008) · 2008 15th Working Conference on Reverse Engineering · doi:10.1109/wcre.2008.33 · Cited by 238 · cited 1×
“Recently, some studies [85], [234], [298], [315], [425] have applied information retrieval techniques to software fault localization.”
ResultAn empirical study of the effects of test-suite reduction on fault localization
Yu, Y., Jones, J. A., & Harrold, M. J. (2008) · Proceedings of the 13th international conference on Software engineering - ICSE '08 · doi:10.1145/1368088.1368116 · Cited by 236 · cited 1×
“…test case reduction by selecting only a subset of test cases or test case prioritization by assigning different priorities to different cases to improve the efficiency of fault localization techniques [45], [48], [53], [62], [63], [122], [126], [127], [128], [146], [175], [176], [348], [362], [399].”
BackgroundA family of code coverage-based heuristics for effective fault localization
Eric Wong, W., Debroy, V., & Choi, B. (2010) · Journal of Systems and Software · doi:10.1016/j.jss.2009.09.037 · Cited by 230 · cited 10×
“Studies in [366], [367], [369], [371], [374] also”
BackgroundMethodInfluentialSelf-citationThe use of program profiling for software maintenance with applications to the year 2000 problem
Reps, T., Ball, T., Das, M., & Larus, J. R. (1997) · doi:10.1145/267895.267925 · Cited by 230 · cited 1×
“It can be used to track program behavior [305].”
BackgroundCheckFence: checking consistency of concurrent data types on relaxed memory models
Burckhardt, S., Alur, R., & Martin, M. M. K. (2007) · Proceedings of the 28th ACM SIGPLAN Conference on Programming Language Design and Implementation · doi:10.1145/1250734.1250737 · Cited by 226 · cited 1×
“Another approach for bug localization in concurrent programs is to use model checking [60], [190], [261], [324].”
MethodAutomated support for classifying software failure reports
Podgurski, A., Leon, D., Francis, P., Masri, W., Minch, M., Sun, J., & Wang, B. (2003) · 25th International Conference on Software Engineering, 2003. Proceedings. · doi:10.1109/icse.2003.1201224 · Cited by 226 · cited 2×
“In response, studies have been conducted using programs with multiple bugs [87], [90], [101], [102], [125], [172], [173], [184], [224], [293], [331], [332], [359], [395], [423].”
BackgroundLightweight Defect Localization for Java
Dallmeier, V., Lindig, C., & Zeller, A. (2005) · Lecture Notes in Computer Science · doi:10.1007/11531142_23 · Cited by 223 · cited 1×
“In one study, Dallmeier et al. [84] collect execution data from Java programs and demonstrate fault localization through the identification and analysis of method call sequences.”
Velodrome: a sound and complete dynamic atomicity checker for multithreaded programs
Flanagan, C., Freund, S. N., & Yi, J. (2008) · Proceedings of the 29th ACM SIGPLAN Conference on Programming Language Design and Implementation · doi:10.1145/1375581.1375618 · Cited by 223 · cited 2×
“Among the approaches that have mushroomed in recent years, predictive analysis-based techniques haven drawn significant attention [111], [113], [115], [116], [322], [349].”
BackgroundCheck 'n' crash: combining static checking and testing
Csallner, C., & Smaragdakis, Y. (2005) · Proceedings of the 27th international conference on Software engineering - ICSE '05 · doi:10.1145/1062455.1062533 · Cited by 220 · cited 3×
“CnC Static checking and testing tool openly accessible [81]”
BackgroundMethodInfluentialExtraction of bug localization benchmarks from history
Dallmeier, V., & Zimmermann, T. (2007) · Proceedings of the twenty-second IEEE/ACM international conference on Automated software engineering · doi:10.1145/1321631.1321702 · Cited by 216 · cited 1×
“Recently, some studies [85], [234], [298], [315], [425] have applied information retrieval techniques to software fault localization.”
ResultAn empirical investigation of the relationship between spectra differences and regression faults
Harrold, M. J., Rothermel, G., Sayre, K., Wu, R., & Yi, L. (2000) · Software Testing, Verification and Reliability · doi:10.1002/1099-1689(200009)10:3<171::aid-stvr209>3.0.co;2-j · Cited by 205
What Went Wrong: Explaining Counterexamples
Groce, A., & Visser, W. (2003) · Lecture Notes in Computer Science · doi:10.1007/3-540-44829-2_8 · Cited by 204 · cited 5×
“We now discuss model checking-based fault localization techniques that rely on the use of model checkers to locate bugs [44], [67], [133], [134], [136], [137], [138], [200].”
BackgroundMethodInfluentialEmpirical Analysis of Software Fault Content and Fault Proneness Using Bayesian Methods
J. Pai, G., & Bechta Dugan, J. (2007) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2007.70722 · Cited by 203 · cited 1×
“Unfortunately, this has also resulted in more software bugs, which often lead to execution failures with huge losses [260], [275], [365].”
BackgroundModel-based diagnosis of hardware designs
Friedrich, G., Stumptner, M., & Wotawa, F. (1999) · Artificial Intelligence · doi:10.1016/s0004-3702(99)00034-x · Cited by 201 · cited 1×
“On the other hand, for model-based software fault localization [6], [40], [97], [117], [194], [242], [243], [246], [248], [378], [380], [381], [382], models are generated directly from the actual programs, which may contain bugs.”
BackgroundSelf-citationBP NEURAL NETWORK-BASED EFFECTIVE FAULT LOCALIZATION
WONG, W. E., & QI, Y. (2009) · International Journal of Software Engineering and Knowledge Engineering · doi:10.1142/s021819400900426x · Cited by 196 · cited 7×
“The EXAM [188], [366], [367], [369], [374] or Expense [185] score is the percentage of statements in a program that has to be examined until the first faulty statement is reached: In [185], the authors use the executable statements instead of the total number of statements.”
BackgroundMethodInfluentialSelf-citationImproving test suites for efficient fault localization
Baudry, B., Fleurey, F., & Le Traon, Y. (2006) · Proceedings of the 28th international conference on Software engineering · doi:10.1145/1134285.1134299 · Cited by 194 · cited 3×
“[50] use a bacteriological approach (which is an adaptation of genetic algorithms) to bridge the gap between testing and diagnosis (fault localization) based on a test-for-diagnosis criterion.”
MethodInfluentialArtho, C., Havelund, K., & Biere, A. (2003) · Software Testing, Verification and Reliability · doi:10.1002/stvr.281 · Cited by 189 · cited 1×
“Concurrent programs suffer most from three kinds of access anomalies: data race [32], [321], atomicity violation [110], [113], [115], and atomic-set serializability violations [24], [44].”
BackgroundEffective Software Fault Localization Using an RBF Neural Network
Wong, W. E., Debroy, V., Golden, R., Xu, X., & Thuraisingham, B. (2012) · IEEE Transactions on Reliability · doi:10.1109/tr.2011.2172031 · Cited by 189 · cited 10×
“The EXAM [188], [366], [367], [369], [374] or Expense [185] score is the percentage of statements in a program that has to be examined until the first faulty statement is reached: In [185], the authors use the executable statements instead of the total number of statements.”
BackgroundMethodResultInfluentialSelf-citationLocating faulty code using failure-inducing chops
Gupta, N., He, H., Zhang, X., & Gupta, R. (2005) · Proceedings of the 20th IEEE/ACM International Conference on Automated Software Engineering · doi:10.1145/1101908.1101948 · Cited by 186 · cited 1×
“Gupta et al. [141] introduce the concept of a failure-inducing chop as an extension to the cause transition technique to overcome this issue.”
BackgroundError explanation with distance metrics
Groce, A., Chaki, S., Kroening, D., & Strichman, O. (2006) · International Journal on Software Tools for Technology Transfer · doi:10.1007/s10009-005-0202-0 · Cited by 186
Precise dynamic slicing algorithms
Zhang, X., Gupta, R., & Zhang, Y. (2003) · doi:10.5555/776816.776855 · Cited by 186
Cause clue clauses: error localization using maximum satisfiability
Jose, M., & Majumdar, R. (2011) · ACM SIGPLAN Notices · doi:10.1145/1993316.1993550 · Cited by 185
Finding Bugs in Web Applications Using Dynamic Test Generation and Explicit-State Model Checking
Artzi, S., Kieżun, A., Dolby, J., Tip, F., Dig, D., Paradkar, A. M., & Ernst, M. D. (2010) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2010.31 · Cited by 181 · cited 2×
“A similar approach is repeated in which Taran-tula is replaced by Ochiai and Jaccard [34].”
MethodSymbolic Debugging of Optimized Code
Hennessy, J. (1982) · ACM Transactions on Programming Languages and Systems · doi:10.1145/357172.357173 · Cited by 169 · cited 1×
“Early studies (e.g., [80], [155]) use this approach to help developers locate bugs while a program is executed under the control of a symbolic debugger.”
BackgroundFault localization using value replacement
Jeffrey, D., Gupta, N., & Gupta, R. (2008) · Proceedings of the 2008 international symposium on Software testing and analysis · doi:10.1145/1390630.1390652 · Cited by 167 · cited 3×
“In response, studies have been conducted using programs with multiple bugs [87], [90], [101], [102], [125], [172], [173], [184], [224], [293], [331], [332], [359], [395], [423].”
MethodMonperrus, M. (2014) · Proceedings of the 36th International Conference on Software Engineering · doi:10.1145/2568225.2568324 · Cited by 166
Cause clue clauses: error localization using maximum satisfiability
Jose, M., & Majumdar, R. (2011) · ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI) · doi:10.1145/1993498.1993550 · Cited by 160 · cited 4×
“[180], [181] rely on an encoding of failed execution traces into error trace formulae.”
BackgroundMethodInfluentialCommon Trends in Software Fault and Failure Data
Hamill, M., & Goševa-Popstojanova, K. (2009) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2009.3 · Cited by 157 · cited 1×
“Results of a study [143] based on an analysis of fault and failure data from two large, real-world projects show that individual failures are often triggered by multiple bugs spread throughout the system.”
BackgroundFinding latent code errors via machine learning over program executions
Brun, Y., & Ernst, M. D. (2004) · doi:10.5555/998675.999452 · Cited by 156
A serializability violation detector for shared-memory server programs
Xu, M., Bodík, R., & Hill, M. D. (2005) · ACM SIGPLAN Notices · doi:10.1145/1064978.1065013 · Cited by 151
Causal inference for statistical fault localization
Baah, G. K., Podgurski, A., & Harrold, M. J. (2010) · International Symposium on Software Testing and Analysis · doi:10.1145/1831708.1831717 · Cited by 150 · cited 1×
“Authors of [38] apply causal-inference techniques to the problem of fault localization.”
BackgroundSteimann, F., Frenkel, M., & Abreu, R. (2013) · International Symposium on Software Testing and Analysis · doi:10.1145/2483760.2483767 · Cited by 149 · cited 1×
“Instrumentation overhead, interference within multiple bugs, and programming language also have an impact on effectiveness of fault localization [90], [333].”
BackgroundSelf-citationDirected test generation for effective fault localization
Artzi, S., Dolby, J., Tip, F., & Pistoia, M. (2010) · International Symposium on Software Testing and Analysis · doi:10.1145/1831708.1831715 · Cited by 149 · cited 1×
“In [33], [34], [35], Artzi et al. present a tool called Apollo to generate test cases automatically based on combined concrete and symbolic executions.”
MethodZheng, A. X., Jordan, M. I., Liblit, B., Naik, M., & Aiken, A. (2006) · International Conference on Machine Learning · doi:10.1145/1143844.1143983 · Cited by 149
Experimental evaluation of using dynamic slices for fault location
Zhang, X., He, H., Gupta, N., & Gupta, R. (2005) · Proceedings of the sixth international symposium on Automated analysis-driven debugging · doi:10.1145/1085130.1085135 · Cited by 149 · cited 2×
“…the effectiveness of specific fault localization techniques have been reported [8], [10], [11], [12], [25], [31], [36], [49], [53], [91], [93], [94], [102], [124], [178], [185], [191], [207], [209], [253], [266], [267], [296], [299], [335], [366], [390], [391], [393], [420], [406], [410], [424].”
Abstract debugging of higher-order imperative languages
Bourdoncle, F. (1993) · ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI) · doi:10.1145/155090.155095 · Cited by 146 · cited 1×
“To fix this problem, Mayer and Stumptner [246] propose an abstraction-based model in which abstract interpretation [55], [78] is applied to handle loops, recursive procedures, and heap data structures.”
MethodA Survey of Empirical Results on Program Slicing
Binkley, D., & Harman, M. (2004) · Advances in computers · doi:10.1016/s0065-2458(03)62003-6 · Cited by 142 · cited 1×
“Hundreds of papers on this topic have been published [52], [344], [394] since Weiser first proposed static slicing in 1979 [361].”
BackgroundFinding failures by cluster analysis of execution profiles
Dickinson, W., Leon, D., & Fodgurski, A. (2005) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2001.919107 · Cited by 140 · cited 1×
“[90], [101], [102], [125], [172], [173], [184], [224], [293], [331], [332], [359], [395], [423].”
Identifying bug signatures using discriminative graph mining
Cheng, H., Lo, D., Zhou, Y., Wang, X., & Yan, X. (2009) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/1572272.1572290 · Cited by 138 · cited 1×
“Authors of [72] propose a software fault localization technique that mines bug signatures within a program.”
BackgroundLiu, C., & Han, J. (2006) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/1181775.1181782 · Cited by 137 · cited 2×
“For techniques such as [224] that generate a ranking of predicates (instead of statements) sorted in descending order of their fault relevance, the EXAM score can also be computed in terms of percentage of predicates that need to be examined.”
MethodWang, X., Cheung, S.-C., Chan, W. K., & Zhang, Z. (2009) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2009.5070507 · Cited by 134
UCheck: A spreadsheet type checker for end users
Abraham, R., & Erwig, M. (2006) · Journal of Visual Languages & Computing · doi:10.1016/j.jvlc.2006.06.001 · Cited by 133 · cited 2×
“In response to this, many studies regarding spread-sheet fault localization have been reported [1], [3], [23], [56], [157], [158], [162], [164], [169], [311], [314].”
BackgroundAn efficient relevant slicing method for debugging
Gyimóthy, T., Beszédes, Á., & Forgács, I. (1999) · ACM SIGSOFT Software Engineering Notes · doi:10.1145/318774.319248 · Cited by 131 · cited 1×
“[142] propose the use of relevant slicing to locate faulty state-”
BackgroundHeader and Unit Inference for Spreadsheets Through Spatial Analyses
Abraham, R., & Erwig, M. (2005) · 2004 IEEE Symposium on Visual Languages - Human Centric Computing · doi:10.1109/vlhcc.2004.29 · Cited by 130 · cited 1×
“In response to this, many studies regarding spread-sheet fault localization have been reported [1], [3], [23], [56], [157], [158], [162], [164], [169], [311], [314].”
BackgroundMining Behavior Graphs for “Backtrace” of Noncrashing Bugs
Liu, C., Yan, X., Yu, H., Han, J., & Yu, P. S. (2005) · Proceedings of the 2005 SIAM International Conference on Data Mining · doi:10.1137/1.9781611972757.26 · Cited by 129 · cited 1×
“In another study, Liu et al. [225] construct software behavior graphs from collected program execution data, including the calling and transition relationships between functions.”
BackgroundA serializability violation detector for shared-memory server programs
Xu, M. G., Bodík, R., & Hill, M. D. (2005) · ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI) · doi:10.1145/1065010.1065013 · Cited by 128 · cited 1×
“A runtime analysis (such as [144], [321], [392]), on the other hand, is less powerful than a static analysis but also produces fewer false alarms.”
MethodSorrentino, F., Farzan, A., & Madhusudan, P. (2010) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/1882291.1882300 · Cited by 126 · cited 1×
“On the contrary, a study in [329] using a different tool, Penelope, for atomicity violations detection reports no false positive.”
ResultSimilarity-based approaches to virtual screening
Willett, P. (2003) · Biochemical Society Transactions · doi:10.1042/bst0310603 · Cited by 124 · cited 1×
“Ochiai2 [267] is an extension of Ochiai, and its formula is as follows: In Table 4 lists 31 similarity coefficient-based techniques, along with their algebraic forms, which have been used in different studies such as [75], [364], [371].”
MethodSlice-based statistical fault localization
Mao, X., Lei, Y., Dai, Z., Qi, Y., & Wang, C. (2013) · Journal of Systems and Software · doi:10.1016/j.jss.2013.08.031 · Cited by 123 · cited 1×
“Studies such as [18], [24], [28], [96], [104], [188], [192], [201], [219], [227], [237], [257], [277], [297], [334], [356], [378], [379], [406], [407], [410], which use the dynamic slicing concept in program debugging, have been reported.”
Automated Fault Localization Using Potential InvariantsPreprint
Pytlik, B., Renieris, M., Krishnamurthi, S., & Reiss, S. P. (2003) · arXiv (Cornell University) · doi:10.48550/arxiv.cs/0310040 · Cited by 120
Accurate and efficient runtime detection of atomicity errors in concurrent programs
Wang, L., & Stoller, S. D. (2006) · ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming · doi:10.1145/1122971.1122993 · Cited by 119 · cited 1×
“significant attention [111], [113], [115], [116], [322], [349].”
Towards Better Fault Localization: A Crosstab-Based Statistical Approach
Wong, W. E., Debroy, V., & Xu, D. (2011) · IEEE Transactions on Systems Man and Cybernetics Part C (Applications and Reviews) · doi:10.1109/tsmcc.2011.2118751 · Cited by 117 · cited 10×
“The EXAM [188], [366], [367], [369], [374] or Expense [185] score is the percentage of statements in a program that has to be examined until the first faulty statement is reached: In [185], the authors use the executable statements instead of the total number of statements.”
BackgroundMethodResultInfluentialSelf-citationCapturing propagation of infected program states
Zhang, Z., Chan, W. K., Tse, T. H., Jiang, B., & Wang, X. (2009) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/1595696.1595705 · Cited by 117 · cited 1×
“In [417], Zhang et al. claim that a bug within a statement may propagate a series of infected program states before it manifests the failure.”
BackgroundAn execution-backtracking approach to debugging
Agrawal, H., Millo, R. A. D., & Spafford, E. H. (1991) · IEEE Software · doi:10.1109/52.88940 · Cited by 116
Dynamic detection of atomic-set-serializability violations
Hammer, C., Dolby, J., Vaziri, M., & Tip, F. (2008) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/1368088.1368120 · Cited by 115 · cited 1×
“A runtime analysis (such as [144], [321], [392]), on the other hand, is less powerful than a static analysis but also produces fewer false alarms.”
MethodEngler, D., & Ashcraft, K. (2003) · ACM SIGOPS Symposium on Operating Systems Principles (SOSP) · doi:10.1145/945465.945468 · Cited by 113
Metamorphic slice: An application in spectrum-based fault localization
Xie, X., Wong, W. E., Chen, T. Y., & Xu, B. (2012) · Information and Software Technology · doi:10.1016/j.infsof.2012.08.008 · Cited by 112 · cited 1×
“Thus, Xie et al. [387] propose a fault localization technique based on the integration of metamorphic relations and slices, in which a program execution slice is replaced by a metamorphic slice; an individual test case is replaced by a metamorphic test group; and the success/failure result of a…”
MethodSelf-citationError Explanation with Distance Metrics
Groce, A. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-24730-2_8 · Cited by 111
Software Fault Localization Using DStar (D*)
Wong, W. E., Debroy, V., Li, Y., & Gao, R. (2012) · doi:10.1109/sere.2012.12 · Cited by 110
Self-citationSpreadsheet Errors: What We Know. What We Think We Can Do
Panko, R. R. (2008) · arXiv · doi:10.48550/arxiv.0802.3457 · Cited by 107 · cited 1×
“Such wide usage, however, has not been accompanied by effective mechanisms for bug prevention and detection, as shown by studies such as [278], [281].”
BackgroundThe Probabilistic Program Dependence Graph and Its Application to Fault Diagnosis
Baah, G. K., Podgurski, A., & Harrold, M. J. (2010) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2009.87 · Cited by 105
Evolving Readable String Test Inputs Using a Natural Language Model to Reduce Human Oracle Cost
Afshan, S., McMinn, P., & Stevenson, M. (2013) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst.2013.11 · Cited by 103 · cited 1×
“In response to this challenge, researchers have presented various solutions [16], [147], [148].”
BackgroundCritical slicing for software fault localization
DeMillo, R. A., Pan, H., & Spafford, E. H. (1996) · ACM SIGSOFT Software Engineering Notes · doi:10.1145/226295.226310 · Cited by 103
Extended comprehensive study of association measures for fault localization
Lucia, L., Lo, D., Jiang, L., Thung, F., & Budi, A. (2014) · Journal of Software: Evolution and Process · doi:10.1002/smr.1616 · Cited by 103 · cited 1×
“A few additional techniques using similar approaches can be found in [230].”
MethodLiblit, B. R., & Aiken, A. (2007) · Cited by 103
On the relationship between model-based debugging and program slicing
Wotawa, F. (2002) · Artificial Intelligence · doi:10.1016/s0004-3702(01)00161-8 · Cited by 102 · cited 3×
“On the other hand, for model-based software fault localization [6], [40], [97], [117], [194], [242], [243], [246], [248], [378], [380], [381], [382], models are generated directly from the actual programs, which may contain bugs.”
BackgroundMethodSelf-citationA Low-Cost Approximate Minimal Hitting Set Algorithm and Its Application to Model-Based Diagnosis ∗
Abreu, R., & van Gemund, A. J. C. (2009) · Symposium on Abstraction, Reformulation and Approximation · Cited by 101
Self-citationOn the influence of multiple faults on coverage-based fault localization
DiGiuseppe, N., & Jones, J. A. (2011) · International Symposium on Software Testing and Analysis · doi:10.1145/2001420.2001446 · Cited by 101 · cited 4×
“[90], [101], [102], [125], [172], [173], [184], [224], [293], [331], [332], [359], [395], [423].”
BackgroundMethodInfluentialUsing automated program repair for evaluating the effectiveness of fault localization techniques
Qi, Y., Mao, X., Lei, Y., & Wang, C. (2013) · International Symposium on Software Testing and Analysis · doi:10.1145/2483760.2483785 · Cited by 99 · cited 1×
“…the effectiveness of specific fault localization techniques have been reported [8], [10], [11], [12], [25], [31], [36], [49], [53], [91], [93], [94], [102], [124], [178], [185], [191], [207], [209], [253], [266], [267], [296], [299], [335], [366], [390], [391], [393], [420], [406], [410], [424].”
Using likely invariants for automated software fault localization
Sahoo, S. K., Criswell, J., Geigle, C., & Adve, V. (2013) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/2451116.2451131 · Cited by 99 · cited 1×
“Potential invariants [294], also called likely invariants [317], are program properties that are observed to hold in some sets of successful executions but, unlike invariants , may not necessarily hold for all possible executions.”
BackgroundUsing Machine Learning to Support Debugging with Tarantula
Briand, L., Labiche, Y., & Liu, X. (2007) · Proceedings/Proceedings - International Symposium on Software Reliability Engineering · doi:10.1109/issre.2007.31 · Cited by 99 · cited 2×
“In [57] Briand et al. use the C4.5 decision tree algorithm to construct rules that classify test cases into various partitions such that failed test cases in the same partition most likely fail due to the same causative fault.”
MethodPELAS-program error-locating assistant system
Korel, B. (1988) · IEEE Transactions on Software Engineering · doi:10.1109/32.6169 · Cited by 97 · cited 2×
“Early studies [19], [201], [203], [341] only use failed test cases for spectrum-based fault localization, though this approach has subsequently been deemed ineffective [21], [185], [366].”
BackgroundPrevalence of coincidental correctness and mitigation of its impact on fault localization
Masri, W., & Assi, R. A. (2014) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/2559932 · Cited by 97 · cited 1×
“Their technique is further enhanced by using fuzzy test suites and clustering analysis [240].”
MethodSupporting professional spreadsheet users by generating leveled dataflow diagrams
Hermans, F., Pinzger, M., & Arie Van Deursen, A. (2011) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/1985793.1985855 · Cited by 96 · cited 1×
“firms use spreadsheets for financial reporting [280], 90 percent of all analysts in the industry perform calculations in spreadsheets [280] and 50 percent of all spreadsheets are the basis for decisions [156].”
BackgroundWhat You See Is What You Test: A Methodology for Testing Form-Based Visual Programs
Rothermel, G., Li, L., Dupuis, C., & Burnett, M. (1998) · Cited by 95
Harman, M., Kim, S. G., Lakhotia, K., McMinn, P., & Yoo, S. (2010) · IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) · doi:10.1109/icstw.2010.31 · Cited by 94 · cited 1×
“In response to this challenge, researchers have presented various solutions [16], [147], [148].”
BackgroundForward computation of dynamic program slices
Korel, B., & Yalamanchili, S. (1994) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/186258.186514 · Cited by 93 · cited 1×
“Collecting dynamic slices may consume excessive time and file space, even though different algorithms [51], [204], [408], [409] have been proposed to address these issues.”
BackgroundMintHint: automated synthesis of repair hints
Kaleeswaran, S., Tulsian, V., Kanade, A., & Orso, A. (2014) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/2568225.2568258 · Cited by 92 · cited 1×
“Last but not least, the idea of modifying a program so that test cases that fail on the original program can be executed successfully on the modified program [133], [134], [200] is also used in other studies for automatic bug fixing [95], [152], [189], [270].”
BackgroundEvaluating Models for Model-Based Debugging
Mayer, W., & Stumptner, M. (2008) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2008.23 · Cited by 92 · cited 2×
“Differences between the behaviors of a model and the actual observed behaviors of the program are used to help find bugs in the program [249], [250].”
BackgroundMethodRecent Catastrophic Accidents: Investigating How Software was Responsible
Wong, W. E., Debroy, V., Surampudi, A., Kim, H.-J., & Siok, M. F. (2010) · Secure Software Integration and Reliability Improvement · doi:10.1109/ssiri.2010.38 · Cited by 92 · cited 1×
“Furthermore, software faults in safety-critical systems have significant ramifications, including not only financial loss, but also potential loss of life, which is an alarming prospect [368].”
BackgroundSelf-citationApplying Code Inspection to Spreadsheet Testing
Panko, R. R. (1999) · Journal of Management Information Systems · doi:10.1080/07421222.1999.11518250 · Cited by 91 · cited 1×
“Other techniques aimed at reducing the occurrence of errors in spreadsheets include code inspection [279], refactoring [42], and adoption of better spreadsheet design practices [82], [83].”
MethodAutomated software testing using a metaheuristic technique based on Tabu search
Diaz, E., Tuya, J., & Blanco, R. (2004) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2003.1240327 · Cited by 91
Efficient program execution indexing
Xin, B., Sumner, W. N., & Zhang, X. (2008) · ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI) · doi:10.1145/1375581.1375611 · Cited by 90 · cited 1×
“further improve the robustness, precision, and efficiency of delta debugging by combining it with more precise execution alignment techniques [338], [339], [389].”
MethodZimmermann, T., & Zeller, A. (2002) · Lecture notes in computer science · doi:10.1007/3-540-45875-1_15 · Cited by 87 · cited 1×
“Zeller and Hildebrandt propose a technique, delta debugging [400], [401], by contrasting program states between executions of a successful test and a failed test via their memory graphs which are described in [426].”
MethodInsights on Fault Interference for Programs with Multiple Bugs
Debroy, V., & Wong, W. E. (2009) · Proceedings/Proceedings - International Symposium on Software Reliability Engineering · doi:10.1109/issre.2009.14 · Cited by 87 · cited 3×
“Debroy and Wong [90] examine possible interactions that may take place between different bugs, and they find that such interferences may manifest themselves to either trigger or mask some execution failures.”
BackgroundSelf-citationPrecise dynamic slicing algorithms
Zhang, X., Gupta, R., & Zhang, Y. (2003) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2003.1201211 · Cited by 87 · cited 1×
“Collecting dynamic slices may consume excessive time and file space, even though different algorithms [51], [204], [408], [409] have been proposed to address these issues.”
BackgroundTIES WITHIN FAULT LOCALIZATION RANKINGS: EXPOSING AND ADDRESSING THE PROBLEM
Xu, X., Debroy, V., Wong, W. E., & Guo, D. (2011) · International Journal of Software Engineering and Knowledge Engineering · doi:10.1142/s0218194011005505 · Cited by 86 · cited 4×
“Other techniques such as confidence-based strategy and data dependency-based strategy are also used to break ties [369], [366], [386], [393].”
BackgroundMethodInfluentialSelf-citationTowards a method of programming with assertions
Rosenblum, D. S. (1992) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/143062.143098 · Cited by 86 · cited 1×
“More details of using assertions for program debugging can be found in [309], [310].”
BackgroundGoalDebug: A Spreadsheet Debugger for End Users
Abraham, R., & Erwig, M. (2007) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2007.39 · Cited by 86 · cited 1×
“Abraham and Erwig [2] describe a tool, GoalDebug, for debugging spreadsheets, using a constraint-based approach similar to that in [164].”
MethodEffective program debugging based on execution slices and inter-block data dependency
Wong, W. E., & Qi, Y. (2006) · Journal of Systems and Software · doi:10.1016/j.jss.2005.06.045 · Cited by 85 · cited 5×
“Tool for collecting execution trace information for C programs via author [22], [366], [367], [369], [370], [371], [372], [373], [374] x Vue Heuristics involving the control graph, execution trace, and the maintainer’s knowledge to help locate features and identify feature interactions via author…”
BackgroundMethodInfluentialSelf-citationTorlak, E., Vaziri, M., & Dolby, J. (2010) · ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI) · doi:10.1145/1806596.1806635 · Cited by 85 · cited 1×
“Authors of [345] propose a tool, MEMSAT, to help in debugging memory models.”
MethodModel-Based Debugging or How to Diagnose Programs Automatically
Wotawa, F., Stumptner, M., & Mayer, W. (2002) · Lecture notes in computer science · doi:10.1007/3-540-48035-8_72 · Cited by 84 · cited 2×
“Wotawa et al. [380] use first order logic to construct dependency-based models based on source code analysis of target programs to represent program structures and behaviors.”
BackgroundSelf-citationA type system for statically detecting spreadsheet errors
Ahmad, Y., Antoniu, T., Goldwater, S. J., & Krishnamurthi, S. (2004) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2003.1240305 · Cited by 84 · cited 1×
“In response to this, many studies regarding spread-sheet fault localization have been reported [1], [3], [23], [56], [157], [158], [162], [164], [169], [311], [314].”
BackgroundVisualization for Fault Localization
Jones, J. A., Harrold, M. J., & Stasko, J. (2003) · Cited by 83
Mining system tests to aid software maintenance
Agrawal, H., Alberi, J. L., Horgan, J. R., Li, J. J., London, S., Wong, W. E., Ghosh, S., & Wilde, N. (1998) · Computer · doi:10.1109/2.689678 · Cited by 83
Self-citationSemi-Proving: An Integrated Method for Program Proving, Testing, and Debugging
Chen, T. Y., Tse, T. H., & Zhou, Z. Q. (2010) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2010.23 · Cited by 83 · cited 1×
“Authors of [71] also use metamorphic relations with symbolic testing for program debugging.”
MethodDetecting and visualizing inter-worksheet smells in spreadsheets
Hermans, F., Pinzger, M., & Arie Van Deursen, A. (2012) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2012.6227171 · Cited by 82 · cited 2×
“In response to this, many studies regarding spread-sheet fault localization have been reported [1], [3], [23], [56], [157], [158], [162], [164], [169], [311], [314].”
BackgroundUnderstanding Counterexamples with explain
Groce, A., Kroening, D., & Lerda, F. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-27813-9_35 · Cited by 82
Automated error localization and correction for imperative programs
Könighofer, R., & Bloem, R. (2011) · doi:10.5555/2157654.2157671 · Cited by 81
Runtime safety analysis of multithreaded programs
Sen, K., Roşu, G., & Agha, G. (2003) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/940071.940116 · Cited by 81
Cleansing Test Suites from Coincidental Correctness to Enhance Fault-Localization
Masri, W., & Assi, R. A. (2010) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst.2010.22 · Cited by 81 · cited 2×
“Studies discussing coincidental correctness have been reported in recent years [44], [46], [159], [218], [239], [254], [355], [419].”
BackgroundMethodFinding latent code errors via machine learning over program executions
Brun, Y., & Ernst, M. D. (2004) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2004.1317470 · Cited by 81 · cited 1×
“Daikon Dynamic invariant detector open source [58], [107]”
BackgroundDynamic slicing method for maintenance of large C programs
Beszédes, Á., Gergely, T., Szabo, Z. M., Csirik, J., & Gyimóthy, T. (2002) · Euromicro Conference on Software Maintenance and Reengineering (CSMR) · doi:10.1109/.2001.914974 · Cited by 80
A slicing-based approach for locating type errors
Tip, F., & Dinesh, T. B. (2001) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/366378.366379 · Cited by 80 · cited 1×
“Static slicing is also applied for fault localization in binary executables [192], and type-checkers [343].”
MethodAdversarial memory for detecting destructive races
Cormac Flanagan, A., & Freund, S. N. (2010) · ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI) · doi:10.1145/1806596.1806625 · Cited by 80 · cited 1×
“For example, Flanagan and Freund use a prototype tool JUMBLE to explore the non-determinism of relaxed memory models and to detect destructive races in the program [114].”
MethodBugFix: A learning-based tool to assist developers in fixing bugs
Jeffrey, D., Feng, M., Gupta, N., & Gupta, R. (2009) · 2009 IEEE 17th International Conference on Program Comprehension · doi:10.1109/icpc.2009.5090029 · Cited by 80 · cited 1×
“BugFix A machine learning-based tool for program debugging via author [174]”
MethodTowards locating execution omission errors
Zhang, X., Tallam, S., Gupta, N., & Gupta, R. (2007) · ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI) · doi:10.1145/1250734.1250782 · Cited by 79 · cited 2×
“To address this problem, Zhang et al. [411] introduce the concept of implicit dependencies , in which dependencies can be obtained by predicate switching.”
BackgroundUsing HTML5 visualizations in software fault localization
Gouveia, C. M. D. S., Campos, J., & Abreu, R. (2013) · IEEE Working Conference on Software Visualization (VISSOFT) · doi:10.1109/vissoft.2013.6650539 · Cited by 78 · cited 1×
“GZoltar An automated testing and debugging framework openly accessible [236], [130]”
MethodSelf-citationMitigating the confounding effects of program dependences for effective fault localization
Baah, G. K., Podgurski, A., & Harrold, M. J. (2011) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/2025113.2025136 · Cited by 76 · cited 1×
“In [39], they further enhance the linear model toward better fault localization effectiveness by including information on data-flow dependence.”
BackgroundA Grouping-Based Strategy to Improve the Effectiveness of Fault Localization Techniques
Debroy, V., Wong, W. E., Xu, X., & Choi, B. (2010) · Asia-Pacific Conference on Quality Software (APAQS) · doi:10.1109/qsic.2010.80 · Cited by 75 · cited 2×
“In [94] For discussion purposes, let’s use the code in Table 2 again.”
MethodSelf-citationEntropy-based test generation for improved fault localization
Campos, J., Abreu, R., Fraser, G., & d’Amorim, M. (2013) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2013.6693085 · Cited by 75 · cited 1×
“…test case reduction by selecting only a subset of test cases or test case prioritization by assigning different priorities to different cases to improve the efficiency of fault localization techniques [45], [48], [53], [62], [63], [122], [126], [127], [128], [146], [175], [176], [348], [362], [399].”
BackgroundSelf-citationPractical fault localization for dynamic web applications
Artzi, S., Dolby, J., Tip, F., & Pistoia, M. (2010) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/1806799.1806840 · Cited by 74 · cited 2×
“In [33], [34], [35], Artzi et al. present a tool called Apollo to generate test cases automatically based on combined concrete and symbolic executions.”
MethodExperimental Evaluation of Program Slicing for Fault Localization
Kusumoto, S., Nishimatsu, A., Nishie, K., & Inoue, K. (2002) · Empirical Software Engineering · doi:10.1023/a:1014823126938 · Cited by 73 · cited 2×
“Although static slice-based techniques have been experimentally evaluated and confirmed to be useful in fault localization [207], one problem is that handling pointer variables can make data-flow analysis inefficient because large sets of data facts that are introduced by dereferences of pointer…”
BackgroundIsolating failure causes through test case generation
Röβler, J., Fraser, G., Zeller, A., & Orso, A. (2012) · International Symposium on Software Testing and Analysis · doi:10.1145/2338965.2336790 · Cited by 73
An approach to software fault localization and revalidation based on incremental data flow analysis
Taha, A.-B., Thebaut, S. M., & Liu, S.-S. (2003) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/cmpsac.1989.65142 · Cited by 73 · cited 1×
“Early studies [19], [201], [203], [341] only use failed test cases”
MDSheet: A framework for model-driven spreadsheet engineering
Cunha, J., Fernandes, J. P., Mendes, J., & Saraiva, J. (2012) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2012.6227239 · Cited by 73 · cited 1×
“Other techniques aimed at reducing the occurrence of errors in spreadsheets include code inspection [279], refactoring [42], and adoption of better spreadsheet design practices [82], [83].”
MethodF3: fault localization for field failures
Jin, W., & Orso, A. (2013) · International Symposium on Software Testing and Analysis · doi:10.1145/2483760.2483763 · Cited by 72 · cited 2×
“terfactual reasoning is also found in other studies such as [137], [179], [400].”
MethodResultAutomated Fault Localization for C Programs
Griesmayer, A., Staber, S., & Bloem, R. (2007) · Electronic Notes in Theoretical Computer Science · doi:10.1016/j.entcs.2006.12.032 · Cited by 71 · cited 6×
“We now discuss model checking-based fault localization techniques that rely on the use of model checkers to locate bugs [44], [67], [133], [134], [136], [137], [138], [200].”
BackgroundMethodInfluentialA study of effectiveness of dynamic slicing in locating real faults
Zhang, X., Gupta, N., & Gupta, R. (2006) · Empirical Software Engineering · doi:10.1007/s10664-006-9007-3 · Cited by 71 · cited 2×
“…the effectiveness of specific fault localization techniques have been reported [8], [10], [11], [12], [25], [31], [36], [49], [53], [91], [93], [94], [102], [124], [178], [185], [191], [207], [209], [253], [266], [267], [296], [299], [335], [366], [390], [391], [393], [420], [406], [410], [424].”
HSFal: Effective fault localization using hybrid spectrum of full slices and execution slices
Ju, X., Jiang, S., Chen, X., Wang, X., Zhang, Y., & Cao, H. (2013) · Journal of Systems and Software · doi:10.1016/j.jss.2013.11.1109 · Cited by 71 · cited 3×
“In [188], Ju et al. propose a hybrid slice-based fault localization technique combining dynamic and execution slices.”
BackgroundMethodCritical slicing for software fault localization
DeMillo, R. A., Pan, H., & Spafford, E. H. (1996) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/229000.226310 · Cited by 71 · cited 1×
“Studies such as [18], [24], [28], [96], [104], [188], [192], [201], [219], [227], [237], [257], [277], [297], [334], [356], [378], [379], [406], [407], [410], which use the dynamic slicing concept in program debugging, have been reported.”
Bug-Assist: Assisting Fault Localization in ANSI-C Programs
Jose, M., & Majumdar, R. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-22110-1_40 · Cited by 70 · cited 3×
“Formula-based fault localization techniques [76], [109], [180], [181] rely on an encoding of failed execution traces into error trace formulae .”
BackgroundMethod
Research works printed but not linked: 17
- jannach, 0, Toward model-based debugging of spreadsheet programs, Proc Knowledge-Based Software Eng Conf, 252
- panko, 2006, Facing the problem of spreadsheet errors, Decision Line, 37, 8
- he, 0, Automated debugging using path-based weakest preconditions, Proc 8th Int Conf Fundam Approaches Softw Eng, 267
- lewis, 1973, Counterfactuals
- lucia lo, 0, Fusing fault localizers, IEEE Int Conf on Auto Soft Eng, 127
- lyle, 0, Automatic program bug location by program slicing, Proc Int Conf Comput Appl, 877
- grottke, 0, A classification of software faults, Proc 16th IEEE Int Symp Softw Rel Eng, 419
- mayer, 0, Prioritizing model-based debugging diagnostic reports, Proc Int Workshop Principles Diagnosis, 127
- mayer, 2004, Approximate modeling for debugging of program loops, Proc Int Workshop Principles Diagnosis, 87
- moon, 2014, Effective software fault localization using dynamic program behaviors, Master thesis
- gong, 0, Diversity maximization speedup for fault, Proc Int Conf Autom Softw Eng, 30
- torlak, 0, MemSAT: Checking axiomatic specifications of memory models, Proc ACM SIGPLAN Conf Programming Language Design and Implementation, 341
- vayani, 2007, Improving automatic software fault localization, Master thesis
- wu, 0, A statistical model to locate faults at input levels, Proc Int Conf Autom Softw Eng, 274
- collofello, 0, Towards automatic software fault localization through decision-to-decision path analysis, Proc Nat Comput Conf, 539
- stumptner, 0, Debugging functional program, Proc Int Joint Conf Artif Intell, 440
- xie, 0, Provably optimal and human-competitive results in SBSE for spectrum based fault localization, Proc 2nd Int Symp Search Based Software Eng, 224
Cited by
Demystifying LLM-Based Software Engineering Agents
Xia, C. S., Deng, Y., Dunn, S., & Zhang, L. (2025) · Proceedings of the ACM on Software Engineering · doi:10.1145/3715754 · Cited by 849
“Fault localization (FL) [Wong et al. 2016] techniques aim to identify the suspicious locations (e.g., statements or methods) in source code related to bugs.”
The DStar Method for Effective Software Fault Localization
Wong, W. E., Debroy, V., Gao, R., & Li, Y. (2014) · IEEE Transactions on Reliability · doi:10.1109/tr.2013.2285319 · Cited by 479
Self-citationEvaluating and Improving Fault Localization
Pearson, S., Campos, J., Just, R., Fraser, G., Abreu, R., Ernst, M. D., Pang, D., & Keller, B. (2017) · 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2017.62 · Cited by 426
Self-citationAutomated Program Repair via Conversation: Fixing 162 out of 337 Bugs for $0.42 Each using ChatGPT
Xia, C. S., & Zhang, L. (2024) · Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis · doi:10.1145/3650212.3680323 · Cited by 203
Automatic Software Repair: A Bibliography
Monperrus, M. (2019) · ACM Computing Surveys · doi:10.1145/3105906 · Cited by 321
Metadata
- Authors
- Wong, W. E., Gao, R., Li, Y., Abreu, R., & Wotawa, F.
- Affiliations
- The University of Texas at Dallas (US)
- Palo Alto Research Center (US)
- Graz University of Technology (AT)
- Venue
- IEEE Transactions on Software Engineering
- Published
- 2016-08-01
- Volume
- 42
- Issue
- 8
- Pages
- 707-740
- ISSN
- 0098-5589 (ISSN-L)
- 1939-3520
- OpenAlex
- W2343875716
- Cited by
- 1,110
- References
- 451
- Citations by year
- 2026: 91 · 2025: 96 · 2024: 127 · 2023: 113 · 2022: 132 · 2021: 118 · 2020: 117 · 2019: 117 · 2018: 88 · 2017: 54 · 2016: 26
- FWCI
- 324.71
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Software Engineering (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Computer Networks and Communications › Software System Performance and Reliability
- Keywords
- software fault localization, debugging, automated fault diagnosis, software defects
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.