The DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Are automated debugging techniques actually helping programmers?
Abstract
Debugging is notoriously difficult and extremely time consuming. Researchers have therefore invested a considerable amount of effort in developing automated techniques and tools for supporting various debugging tasks. Although potentially useful, most of these techniques have yet to demonstrate their practical effectiveness. One common limitation of existing approaches, for instance, is their reliance on a set of strong assumptions on how developers behave when debugging (e.g., the fact that examining a faulty statement in isolation is enough for a developer to understand and fix the corresponding bug). In more general terms, most existing techniques just focus on selecting subsets of potentially faulty statements and ranking them according to some criterion. By doing so, they ignore the fact that understanding the root cause of a failure typically involves complex activities, such as navigating program dependencies and rerunning the program with different inputs. The overall goal of this research is to investigate how developers use and benefit from automated debugging tools through a set of human studies. As a first step in this direction, we perform a preliminary study on a set of developers by providing them with an automated debugging tool and two tasks to be performed with and without the tool. Our results provide initial evidence that several assumptions made by automated debugging techniques do not hold in practice. Through an analysis of the results, we also provide insights on potential directions for future work in the area of automated debugging.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Weiser, M. (1984) · IEEE Transactions on Software Engineering · doi:10.1109/tse.1984.5010248 · Cited by 3,204
Empirical evaluation of the tarantula automatic fault-localization technique
Jones, J. A., & Harrold, M. J. (2005) · Proceedings of the 20th IEEE/ACM International Conference on Automated Software Engineering · doi:10.1145/1101908.1101949 · Cited by 1,238
Do, H., Elbaum, S., & Rothermel, G. (2005) · Empirical Software Engineering · doi:10.1007/s10664-005-3861-2 · Cited by 1,146
Simplifying and isolating failure-inducing input
Zeller, A., & Hildebrandt, R. (2002) · IEEE Transactions on Software Engineering · doi:10.1109/32.988498 · Cited by 1,079
Visualization of test information to assist fault localization
Jones, J. A., Harrold, M. J., & Stasko, J. (2002) · Proceedings/Proceedings - International Conference on Software Engineering · doi:10.1145/581339.581397 · Cited by 904
Scalable statistical bug isolation
Liblit, B., Naik, M., Zheng, A. X., Aiken, A., & Jordan, M. I. (2005) · ACM SIGPLAN Notices · doi:10.1145/1064978.1065014 · Cited by 729
Eye-tracking analysis of user behavior in WWW search
Granka, L. A., Joachims, T., & Gay, G. (2004) · Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/1008992.1009079 · Cited by 696
Scalable statistical bug isolation
Liblit, B., Naik, M., Zheng, A. X., Aiken, A., & Jordan, M. I. (2005) · Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation · doi:10.1145/1065010.1065014 · Cited by 472
Fault localization with nearest neighbor queries
Renieres, M., & Reiss, S. P. (2004) · 18th IEEE International Conference on Automated Software Engineering, 2003. Proceedings. · doi:10.1109/ase.2003.1240292 · Cited by 427
Expertise in debugging computer programs: A process analysis
Vessey, I. (1985) · International Journal of Man-Machine Studies · doi:10.1016/s0020-7373(85)80054-7 · Cited by 359
SOBER: statistical model-based bug localization
Liu, C., Yan, X., Fei, L., Han, J., & Midkiff, S. P. (2005) · Proceedings of the 10th European software engineering conference held jointly with 13th ACM SIGSOFT international symposium on Foundations of software engineering · doi:10.1145/1081706.1081753 · Cited by 357
Debugging reinvented: asking and answering why and why not questions about program behavior
Ko, A. J., & Myers, B. A. (2008) · Proceedings of the 30th international conference on Software engineering · doi:10.1145/1368088.1368130 · Cited by 296
From symptom to cause: localizing errors in counterexample traces
Ball, T., Naik, M., & Rajamani, S. K. (2003) · Proceedings of the 30th ACM SIGPLAN-SIGACT symposium on Principles of programming languages · doi:10.1145/604131.604140 · Cited by 294
Jones, J. A., Bowring, J. F., & Harrold, M. J. (2007) · Proceedings of the 2007 international symposium on Software testing and analysis · doi:10.1145/1273463.1273468 · Cited by 245
The use of program profiling for software maintenance with applications to the year 2000 problem
Reps, T., Ball, T., Das, M., & Larus, J. R. (1997) · doi:10.1145/267895.267925 · Cited by 230
Precise dynamic slicing algorithms
Zhang, X., Gupta, R., & Zhang, Y. (2003) · doi:10.5555/776816.776855 · Cited by 186
Finding causes of program output with the Java Whyline
Ko, A. J., & Myers, B. A. (2009) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1518701.1518942 · Cited by 146
Identifying bug signatures using discriminative graph mining
Cheng, H., Lo, D., Zhou, Y., Wang, X., & Yan, X. (2009) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/1572272.1572290 · Cited by 138
Liu, C., & Han, J. (2006) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/1181775.1181782 · Cited by 137
An efficient relevant slicing method for debugging
Gyimóthy, T., Beszédes, Á., & Forgács, I. (1999) · ACM SIGSOFT Software Engineering Notes · doi:10.1145/318774.319248 · Cited by 131
Critical slicing for software fault localization
DeMillo, R. A., Pan, H., & Spafford, E. H. (1996) · ACM SIGSOFT Software Engineering Notes · doi:10.1145/226295.226310 · Cited by 103
Experimental Evaluation of Program Slicing for Fault Localization
Kusumoto, S., Nishimatsu, A., Nishie, K., & Inoue, K. (2002) · Empirical Software Engineering · doi:10.1023/a:1014823126938 · Cited by 73
Understanding Counterexamples with explain
Groce, A., Kroening, D., & Lerda, F. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-27813-9_35 · Cited by 73
Critical slicing for software fault localization
DeMillo, R. A., Pan, H., & Spafford, E. H. (1996) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/229000.226310 · Cited by 71
Evaluating Cues for Resuming Interrupted Programming Tasks
Parnin, C., & DeLine, R. (2010) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1753326.1753342 · Cited by 60 · cited 1×
“From our previous experience in running programming experiments [20], popular games are ideal subjects, as participants are typically familiar with the behavior of the games and can readily identify game concepts in the source code.”
BackgroundSelf-citationEXPERTISE IN DEBUGGING COMPUTER PROGRAMS
Vessey, I. (1984) · Faculty Digital Archive (New York University Florence) · Cited by 49
Managing software change tasks: an exploratory study
Sillito, J., Volder, K. D., Fisher, B., & Murphy, G. C. (2005) · International Symposium on Empirical Software Engineering · doi:10.1109/isese.2005.1541811 · Cited by 45
The value of slicing while debugging
Francel, M. A., & Rugaber, S. (2001) · Science of Computer Programming · doi:10.1016/s0167-6423(01)00013-2 · Cited by 39
Cited by
A Survey on Software Fault Localization
Wong, W. E., Gao, R., Li, Y., Abreu, R., & Wotawa, F. (2016) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2016.2521368 · Cited by 1,110
“Last but not least, it is important to realize that software fault localization techniques should not be evaluated only in terms of effectiveness as described above [285].”
BackgroundEvaluating and Improving Fault Localization
Pearson, S., Campos, J., Just, R., Fraser, G., Abreu, R., Ernst, M. D., Pang, D., & Keller, B. (2017) · 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2017.62 · Cited by 426
LineVul: a transformer-based line-level vulnerability prediction
Fu, M., & Tantithamthavorn, C. (2022) · Proceedings of the 19th International Conference on Mining Software Repositories · doi:10.1145/3524842.3528452 · Cited by 384
Metallaxis‐FL: mutation‐based fault localization
Papadakis, M., & Le Traon, Y. (2015) · Software Testing, Verification and Reliability · doi:10.1002/stvr.1509 · Cited by 371
A theoretical analysis of the risk evaluation formulas for spectrum-based fault localization
Xie, X., Chen, T. Y., Kuo, F.-C., & Xu, B. (2013) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/2522920.2522924 · Cited by 353
Metadata
- Authors
- Parnin, C., & Orso, A.
- Venue
- Proceedings of the 2011 International Symposium on Software Testing and Analysis
- Publisher
- ACM
- Published
- 2011-07-17
- Pages
- 199-209
- OpenAlex
- W2067436653
- Cited by
- 576
- References
- 28
- FWCI
- 24.86
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.