The DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index.
Statistical Debugging: A Hypothesis Testing-Based Approach
Abstract
Manual debugging is tedious, as well as costly. The high cost has motivated the development of fault localization techniques, which help developers search for fault locations. In this paper, we propose a new statistical method, called SOBER, which automatically localizes software faults without any prior knowledge of the program semantics. Unlike existing statistical approaches that select predicates correlated with program failures, SOBER models the predicate evaluation in both correct and incorrect executions and regards a predicate as fault-relevant if its evaluation pattern in incorrect executions significantly diverges from that in correct ones. Featuring a rationale similar to that of hypothesis testing, SOBER quantifies the fault relevance of each predicate in a principled way. We systematically evaluate SOBER under the same setting as previous studies. The result clearly demonstrates the effectiveness: SOBER could help developers locate 68 out of the 130 faults in the Siemens suite by examining no more than 10 percent of the code, whereas the cause transition approach proposed by Holger et al. [2005] and the statistical approach by Liblit et al. [2005] locate 34 and 52 faults, respectively. Moreover, the effectiveness of SOBER is also evaluated in an "imperfect world", where the test suite is either inadequate or only partially labeled. The experiments indicate that SOBER could achieve competitive quality under these harsh circumstances. Two case studies with grep 2.2 and bc 1.06 are reported, which shed light on the applicability of SOBER on reasonably large programs
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Elements of Information Theory
Cover, T. M., & Thomas, J. A. (2001) · Wiley eBooks · doi:10.1002/0471200611 · Cited by 36,808
Clarke, E. M., Grümberg, O., & Long, D. (1996) · Cited by 6,813
Casella, G., & Berger, R. (2024) · Chapman & Hall/CRC eBooks · doi:10.1201/9781003456285 · Cited by 5,711
Visser, W., Havelund, K., Brat, G., Park, S., & Lerda, F. (2003) · Automated Software Engineering · doi:10.1023/a:1022920129859 · Cited by 1,343
Empirical evaluation of the tarantula automatic fault-localization technique
Jones, J. A., & Harrold, M. J. (2005) · Proceedings of the 20th IEEE/ACM International Conference on Automated Software Engineering · doi:10.1145/1101908.1101949 · Cited by 1,238
Do, H., Elbaum, S., & Rothermel, G. (2005) · Empirical Software Engineering · doi:10.1007/s10664-005-3861-2 · Cited by 1,146
Simplifying and isolating failure-inducing input
Zeller, A., & Hildebrandt, R. (2002) · IEEE Transactions on Software Engineering · doi:10.1109/32.988498 · Cited by 1,079
Dynamically discovering likely program invariants to support program evolution
Ernst, M. D., Cockrell, J., Griswold, W. G., & Notkin, D. (2001) · IEEE Transactions on Software Engineering · doi:10.1109/32.908957 · Cited by 1,050
A Survey of Program Slicing Techniques.
Tip, F. (1994) · Centrum Wiskunde & Informatica (CWI), the national research institute for mathematics and computer science in the Netherlands · Cited by 952
Visualization of test information to assist fault localization
Jones, J. A., Harrold, M. J., & Stasko, J. (2002) · Proceedings/Proceedings - International Conference on Software Engineering · doi:10.1145/581339.581397 · Cited by 904
Experiments on the effectiveness of dataflow- and control-flow-based test adequacy criteria
Hutchins, M., Foster, H. D., Goradia, T., & Ostrand, T. J. (1994) · doi:10.5555/257734.257766 · Cited by 774
Bugs as deviant behavior: a general approach to inferring errors in systems code
Engler, D., Chen, D. Y., Hallem, S., Chou, A., & Chelf, B. (2001) · Proceedings of the eighteenth ACM symposium on Operating systems principles · doi:10.1145/502034.502041 · Cited by 737
Scalable statistical bug isolation
Liblit, B., Naik, M., Zheng, A. X., Aiken, A., & Jordan, M. I. (2005) · ACM SIGPLAN Notices · doi:10.1145/1064978.1065014 · Cited by 729
Tracking down software bugs using automatic anomaly detection
Hangal, S., & Lam, M. S. (2002) · Proceedings/Proceedings - International Conference on Software Engineering · doi:10.1145/581339.581377 · Cited by 602
Korat: automated testing based on Java predicates
Boyapati, C., Khurshid, S., & Marinov, D. (2002) · ACM SIGSOFT Software Engineering Notes · doi:10.1145/566171.566191 · Cited by 593
Locating causes of program failures
Cleve, H., & Zeller, A. (2005) · Proceedings of the 27th international conference on Software engineering - ICSE '05 · doi:10.1145/1062455.1062522 · Cited by 550
Isolating cause-effect chains from computer programs
Zeller, A. (2002) · Proceedings of the 10th ACM SIGSOFT symposium on Foundations of software engineering · doi:10.1145/587051.587053 · Cited by 530
Visualization of test information to assist fault localization
Jones, J. A., Harrold, M. J., & Stasko, J. (2002) · Proceedings of the 24th international conference on Software engineering - ICSE '02 · doi:10.1145/581396.581397 · Cited by 524
Bug isolation via remote program sampling
Liblit, B., Aiken, A., Zheng, A. X., & Jordan, M. I. (2003) · ACM SIGPLAN Notices · doi:10.1145/780822.781148 · Cited by 504
Scalable statistical bug isolation
Liblit, B., Naik, M., Zheng, A. X., Aiken, A., & Jordan, M. I. (2005) · Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation · doi:10.1145/1065010.1065014 · Cited by 472
Fault localization with nearest neighbor queries
Renieres, M., & Reiss, S. P. (2004) · 18th IEEE International Conference on Automated Software Engineering, 2003. Proceedings. · doi:10.1109/ase.2003.1240292 · Cited by 427
JCrasher: an automatic robustness tester for Java
Csallner, C., & Smaragdakis, Y. (2004) · Software: Practice and Experience · doi:10.1002/spe.602 · Cited by 426
Experiments on the effectiveness of dataflow- and control-flow-based test adequacy criteria
Hutchins, M., Foster, H., Goradia, T., & Ostrand, T. (2002) · Proceedings of 16th International Conference on Software Engineering · doi:10.1109/icse.1994.296778 · Cited by 375
CMC: a pragmatic approach to model checking real code
Musuvathi, M., Park, D. Y. W., Chou, A., Engler, D. R., & Dill, D. L. (2002) · ACM SIGOPS Operating Systems Review · doi:10.1145/844128.844136 · Cited by 361
Designing the whyline: a debugging interface for asking questions about program behavior
Ko, A. J., & Myers, B. A. (2004) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/985692.985712 · Cited by 361
Expertise in debugging computer programs: A process analysis
Vessey, I. (1985) · International Journal of Man-Machine Studies · doi:10.1016/s0020-7373(85)80054-7 · Cited by 359
SOBER: statistical model-based bug localization
Liu, C., Yan, X., Fei, L., Han, J., & Midkiff, S. P. (2005) · Proceedings of the 10th European software engineering conference held jointly with 13th ACM SIGSOFT international symposium on Foundations of software engineering · doi:10.1145/1081706.1081753 · Cited by 357
Self-citationVisser, W., Havelund, K., Brat, G., & Park, S. (2000) · Proceedings ASE 2000. Fifteenth IEEE International Conference on Automated Software Engineering · doi:10.1109/ase.2000.873645 · Cited by 341
Automated support for classifying software failure reports
Podgurski, A., Leon, D., Francis, P., Masri, W., & Minch, M. (2003) · Cited by 314
Korat: automated testing based on Java predicates
Boyapati, C., Khurshid, S., & Marinov, D. (2002) · Proceedings of the 2002 ACM SIGSOFT international symposium on Software testing and analysis · doi:10.1145/566172.566191 · Cited by 302
Fault localization using execution slices and dataflow tests
Agrawal, H., Horgan, J. R., London, S., & Wong, W. E. (2002) · Proceedings of Sixth International Symposium on Software Reliability Engineering. ISSRE'95 · doi:10.1109/issre.1995.497652 · Cited by 288
Finding failures by cluster analysis of execution profiles
Dickinson, W. T., De, L., & Fodgurski, A. (2001) · doi:10.5555/381473.381509 · Cited by 252
Eclat: Automatic Generation and Classification of Test Inputs
Pacheco, C., & Ernst, M. D. (2005) · Lecture Notes in Computer Science · doi:10.1007/11531142_22 · Cited by 250
Tracking down software bugs using automatic anomaly detection
Hangal, S., & Lam, M. S. (2002) · Proceedings of the 24th international conference on Software engineering - ICSE '02 · doi:10.1145/581376.581377 · Cited by 250
Bug isolation via remote program sampling
Liblit, B., Aiken, A., Zheng, A. X., & Jordan, M. I. (2003) · Proceedings of the ACM SIGPLAN 2003 conference on Programming language design and implementation · doi:10.1145/781131.781148 · Cited by 241
Automated support for classifying software failure reports
Podgurski, A., Leon, D., Francis, P., Masri, W., Minch, M., Sun, J., & Wang, B. (2003) · 25th International Conference on Software Engineering, 2003. Proceedings. · doi:10.1109/icse.2003.1201224 · Cited by 226
An empirical investigation of the relationship between spectra differences and regression faults
Harrold, M. J., Rothermel, G., Sayre, K., Wu, R., & Yi, L. (2000) · Software Testing, Verification and Reliability · doi:10.1002/1099-1689(200009)10:3<171::aid-stvr209>3.0.co;2-j · Cited by 205
von Lieshout, M. N. M. (2000) · Markov Point Processes and Their Applications · doi:10.1142/9781860949760_0003 · Cited by 202
Locating faulty code using failure-inducing chops
Gupta, N., He, H., Zhang, X., & Gupta, R. (2005) · Proceedings of the 20th IEEE/ACM International Conference on Automated Software Engineering · doi:10.1145/1101908.1101948 · Cited by 186
Finding latent code errors via machine learning over program executions
Brun, Y., & Ernst, M. D. (2004) · doi:10.5555/998675.999452 · Cited by 158
Finding failures by cluster analysis of execution profiles
Dickinson, W., Leon, D., & Fodgurski, A. (2005) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2001.919107 · Cited by 140
Automated Fault Localization Using Potential InvariantsPreprint
Pytlik, B., Renieris, M., Krishnamurthi, S., & Reiss, S. P. (2003) · arXiv (Cornell University) · doi:10.48550/arxiv.cs/0310040 · Cited by 120
Zimmermann, T., & Zeller, A. (2002) · Lecture notes in computer science · doi:10.1007/3-540-45875-1_15 · Cited by 87
Finding latent code errors via machine learning over program executions
Brun, Y., & Ernst, M. D. (2004) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2004.1317470 · Cited by 81
EXPERTISE IN DEBUGGING COMPUTER PROGRAMS
Vessey, I. (1984) · Faculty Digital Archive (New York University Florence) · Cited by 49
An empirical study of fault localization for end-user programmers
Ruthruff, J. R., Burnett, M., & Rothermel, G. (2005) · Proceedings/Proceedings - International Conference on Software Engineering · doi:10.1145/1062455.1062523 · Cited by 41
Spreadsheet DebuggingPreprint
Ayalew, Y., & Mittermeir, R. T. (2008) · arXiv (Cornell University) · doi:10.48550/arxiv.0801.4280 · Cited by 24
Cited by
A Survey on Software Fault Localization
Wong, W. E., Gao, R., Li, Y., Abreu, R., & Wotawa, F. (2016) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2016.2521368 · Cited by 1,110
Zhou, J., Zhang, H., & Lo, D. (2012) · 2012 34th International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2012.6227210 · Cited by 507
The DStar Method for Effective Software Fault Localization
Wong, W. E., Debroy, V., Gao, R., & Li, Y. (2014) · IEEE Transactions on Reliability · doi:10.1109/tr.2013.2285319 · Cited by 479
Evaluating and Improving Fault Localization
Pearson, S., Campos, J., Just, R., Fraser, G., Abreu, R., Ernst, M. D., Pang, D., & Keller, B. (2017) · 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2017.62 · Cited by 426
Zhou, J. H., Zhang, H., & Lo, D. (2012) · doi:10.5555/2337223.2337226 · Cited by 372
Metadata
- Authors
- Liu, C., Fei, L., Yan, X., Han, J., & Midkiff, S. P.
- Venue
- IEEE Transactions on Software Engineering
- Published
- 2006-10
- Volume
- 32
- Issue
- 10
- Pages
- 831-848
- ISSN
- 0098-5589 (ISSN-L)
- 1939-3520
- OpenAlex
- W2137839528
- Cited by
- 288
- References
- 47
- FWCI
- 40.19
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.