BugRedux: Reproducing field failures for in-house debugging
The DOI matches its Crossref record; 34 of the 35 references it cites were found in a registry or an index.
Abstract
A recent survey conducted among developers of the Apache, Eclipse, and Mozilla projects showed that the ability to recreate field failures is considered of fundamental importance when investigating bug reports. Unfortunately, the information typically contained in a bug report, such as memory dumps or call stacks, is usually insufficient for recreating the problem. Even more advanced approaches for gathering field data and help in-house debugging tend to collect either too little information, and be ineffective, or too much information, and be inefficient. To address these issues, we present BugRedux, a novel general approach for in-house debugging of field failures. BugRedux aims to synthesize, using execution data collected in the field, executions that mimic the observed field failures. We define several instances of BugRedux that collect different types of execution data and perform, through an empirical study, a cost-benefit analysis of the approach and its variations. In the study, we apply BugRedux to 16 failures of 14 real-world programs. Our results are promising in that they show that it is possible to synthesize in-house executions that reproduce failures observed in the field using a suitable set of execution data.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Bug reproduction is a critical step in investigating bug reports [Jin and Orso 2012], and is integrated into many recent software engineering agents [Arora et al. 2024;Factory 2024;Honeycomb 2024;MentatBot 2024;OpenDevin 2024;Ruan et al. 2024;Yang et al. 2024a].”
From Demystifying LLM-Based Software Engineering Agents (2025) · cites this paper as Jin and Orso 2012
Citations
Cites
Symbolic execution and program testing
King, J. C. (1976) · Communications of the ACM · doi:10.1145/360248.360252 · Cited by 3,031 · cited 1×
“In this work, we decided to use an approach based on symbolic execution [18].”
MethodKLEE: unassisted and automatic generation of high-coverage tests for complex systems programs
Cadar, C., Dunbar, D., & Engler, D. (2008) · doi:10.5555/1855741.1855756 · Cited by 2,620
DART: directed automated random testing
Godefroid, P., Klarlund, N., & Sen, K. (2005) · Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation · doi:10.1145/1065010.1065036 · Cited by 1,570 · cited 1×
“, [20], [27]–[29]) and random generation (e.”
Automated Whitebox Fuzz Testing.
Godefroid, P., Levin, M. Y., & Molnar, D. A. (2008) · Cited by 1,095
Simplifying and isolating failure-inducing input
Zeller, A., & Hildebrandt, R. (2002) · IEEE Transactions on Software Engineering · doi:10.1109/32.988498 · Cited by 1,079 · cited 1×
“, [13], [14]), or are still in their early phase of development and in need of a more thorough”
BackgroundAre automated debugging techniques actually helping programmers?
Parnin, C., & Orso, A. (2011) · Proceedings of the 2011 International Symposium on Software Testing and Analysis · doi:10.1145/2001420.2001445 · Cited by 576 · cited 1×
“Although such an evaluation would be extremely useful, and we plan to do it in the future as we did for earlier work [26], we believe that it would be premature at this point.”
BackgroundSelf-citationExtracting usability information from user interface events
Hilbert, D. M., & Redmiles, D. F. (2000) · ACM Computing Surveys · doi:10.1145/371578.371593 · Cited by 477
Scalable statistical bug isolation
Liblit, B., Naik, M., Zheng, A. X., Aiken, A., & Jordan, M. I. (2005) · Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation · doi:10.1145/1065010.1065014 · Cited by 472 · cited 1×
“Among these techniques, some collect only limited amounts of information ( e.g., sampled branch profiles in the case of CBI [6], [7]).”
BackgroundA Theoretical and Empirical Study of Search-Based Testing: Local, Global, and Hybrid Search
Harman, M., & McMinn, P. (2010) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2009.71 · Cited by 400 · cited 1×
“, [38]), and techniques that take advantage of existing test inputs using some fort of fuzzing (e.”
BackgroundCompilers: Principles, Techniques, and Tools (2nd Edition)
Aho, A. V., Lam, M. S., Sethi, R., & Ullman, J. D. (2006) · Addison-Wesley Longman Publishing Co., Inc. eBooks · Cited by 387
Zimmermann, T., Premraj, R., Bettenburg, N., Just, S., Schroter, A., & Weiss, C. (2010) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2010.63 · Cited by 337
PRES: probabilistic replay with execution sketching on multiprocessors
Park, S., Zhou, Y., Xiong, W., Yin, Z., Kaushik, R., Lee, K. H., & Lu, S. (2009) · Proceedings of the ACM SIGOPS 22nd symposium on Operating systems principles · doi:10.1145/1629575.1629593 · Cited by 303
CUTE: A Concolic Unit Testing Engine for C
Sen, K., Marinov, D., & Agha, G. (2005) · doi:10.21236/ada482657 · Cited by 300
HOLMES: Effective statistical debugging via efficient path profiling
Chilimbi, T. M., Liblit, B., Mehra, K., Nori, A. V., & Vaswani, K. (2009) · 2009 IEEE 31st International Conference on Software Engineering · doi:10.1109/icse.2009.5070506 · Cited by 246
Execution synthesis: a technique for automated software debugging
Zamfir, C., & Candea, G. (2010) · Proceedings of the 5th European conference on Computer systems · doi:10.1145/1755913.1755946 · Cited by 222 · cited 2×
“Zamfir and Candea introduce a technique for automated debugging based on input generation [34].”
BackgroundMethodAEG: Automatic Exploit Generation
(2018) · doi:10.1184/r1/6468296.v1 · Cited by 205
AEG: Automatic Exploit Generation
Avgerinos, T., Kil, S., Hao, B. L. T., & Brumley, D. (2018) · doi:10.1184/r1/6468296 · Cited by 202
SherLog: error diagnosis by connecting clues from run-time logs
Yuan, D., Mai, H., Xiong, W., Tan, L., Zhou, Y., & Pasupathy, S. (2010) · Proceedings of the fifteenth International Conference on Architectural support for programming languages and operating systems · doi:10.1145/1736020.1736038 · Cited by 176 · cited 2×
“Another approach, SherLog [33], makes use of run-time logs to reconstruct paths near logging statements to help developers to identify bugs.”
BackgroundImproving software diagnosability via log enhancement
Yuan, D., Zheng, J., Park, S., Zhou, Y., & Savage, S. (2011) · Proceedings of the sixteenth international conference on Architectural support for programming languages and operating systems · doi:10.1145/1950365.1950369 · Cited by 154
Residual test coverage monitoring
Pavlopoulou, C., & Young, M. (1999) · Proceedings/Proceedings - International Conference on Software Engineering · doi:10.1145/302405.302637 · Cited by 153
Extended static checking for Java
Cormac Flanagan, A., Leino, K. R. M., Lillibridge, M., Nelson, G., Saxe, J. B., & Stata, R. (2002) · ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI) · doi:10.1145/512557.512558 · Cited by 132
Context-aware statistical debugging: from bug predictors to faulty control flow paths
Jiang, L., & Su, Z. (2007) · doi:10.1145/1321631.1321660 · Cited by 124 · cited 1×
“…the usefulness of the information collected for debugging increases when more (and more detailed) data is collected, and researchers have defined novel techniques that gather a wide spectrum of richer data, ranging from path profiles to complete execution recordings ( e.g., [4], [5], [11], [12]).”
BackgroundChandra, S., Fink, S. J., & Sridharan, M. (2009) · Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation · doi:10.1145/1542476.1542517 · Cited by 123 · cited 2×
“, [32]), techniques based on genetic algorithms (e.”
BackgroundMethodReCrash: Making Software Failures Reproducible by Preserving Object States
Artzi, S., Kim, S., & Ernst, M. D. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-70592-5_23 · Cited by 117 · cited 1×
“Artzi and colleagues present ReCrash [37], a technique that records partial object states at method levels dynamically to recreate an observed crash at different levels of stack depth.”
BackgroundBetter bug reporting with better privacy
Castro, M., Costa, M., & Martin, J.-P. (2008) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/1346281.1346322 · Cited by 116 · cited 1×
“…whose goal is to sanitize or anonymize collected data, they are either defined for a different goal, which means that they would eliminate sensitive data only by chance ( e.g., [13], [14]), or are still in their early phase of development and in need of more thorough evaluation ( e.g., [15], [16]).”
BackgroundTest input generation with java PathFinder
Visser, W., Păsăreanu, C. S., & Khurshid, S. (2004) · ACM SIGSOFT Software Engineering Notes · doi:10.1145/1013886.1007526 · Cited by 116
DebugAdvisor: a recommender system for debugging
Ashok, B., Joy, J. M., Liang, H., Rajamani, S. K., Srinivasa, G., & Vangala, V. (2009) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/1595696.1595766 · Cited by 105 · cited 1×
“DebugAdvisor [41], for instance, is a tool that analyzes crash reports to help find a solution to the reported problem by identifying developers, code, and other known bugs that may be correlated to the report.”
BackgroundFinding errors in .net with feedback-directed random testing
Pacheco, C., Lahiri, S. K., & Ball, T. (2008) · International Symposium on Software Testing and Analysis · doi:10.1145/1390630.1390643 · Cited by 102
A Technique for Enabling and Supporting Debugging of Field Failures
Clause, J., & Orso, A. (2007) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2007.10 · Cited by 100 · cited 1×
“…the usefulness of the information collected for debugging increases when more (and more detailed) data is collected, and researchers have defined novel techniques that gather a wide spectrum of richer data, ranging from path profiles to complete execution recordings ( e.g., [4], [5], [11], [12]).”
BackgroundSelf-citationProfiling deployed software: assessing strategies and testing opportunities
Elbaum, S. G., & Diep, M. M. (2005) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2005.50 · Cited by 91 · cited 1×
“Tobetter support developers in their debugging activities, researchers have also investigated more sophisticated techniques for capturing data from deployed applications ( e.g., [4]–[10]) and using them for debugging.”
BackgroundAccurate Interprocedural Null-Dereference Analysis for Java
Nanda, M. G., & Sinha, S. K. (2009) · doi:10.1109/icse.2009.5070515 · Cited by 83
Clause, J., & Orso, A. (2011) · Proceedings/Proceedings - International Conference on Software Engineering · doi:10.1145/1985793.1985797 · Cited by 45 · cited 1×
“…whose goal is to sanitize or anonymize collected data, they are either defined for a different goal, which means that they would eliminate sensitive data only by chance ( e.g., [13], [14]), or are still in their early phase of development and in need of more thorough evaluation ( e.g., [15], [16]).”
BackgroundSelf-citationStriking a new balance between program instrumentation and debugging time
Crameri, O., Bianchini, R., & Zwaenepoel, W. (2011) · doi:10.1145/1966445.1966464 · Cited by 31 · cited 1×
“The two approaches most related to ours are ESD, by Zamfir and Candea [24] and the technique by Crameri, Bianchini, and Zwaenepoel [38].”
MethodNainar, P. A., & Liblit, B. (2007) · Springer eBooks · doi:10.1007/978-3-540-71878-9 · Cited by 28 · cited 1×
“Among these techniques, some collect only limited amounts of information ( e.g., sampled branch profiles in the case of CBI [6], [7]).”
Background
Research works printed but not linked: 9
- jin, 2011, BugRedux Reproducing Field Failures for In-house Debugging
- 2012, Tmin Fuzzing Test Case Optimizer
- 2012, The Amazing VM Record/Replay Feature in VMware Workstation 6
- 2012, Software-artifact infrastructure repository
- 2012, Breakpad
- lu, 0, BugBench: Benchmarks for Evaluating Bug Detection Tools, Workshop on the Evaluation of Software Defect Detection Tools 2005
- 2012, Technical Note TN2123 CrashReporter
- 2012, Windows Error Reporting Getting Started
- pacheco, 0, Randoop: Feedback-Directed Random Testing for Java, Proceedings of the 22nd ACM SIGPLAN Conference on Object-oriented Programming Systems and Applications 2007, 815
Cited by
Böhme, M., Pham, V.-T., Nguyen, M.-D., & Roychoudhury, A. (2017) · Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3133956.3134020 · Cited by 747
AutoCodeRover: Autonomous Program Improvement
Zhang, Y., Ruan, H., Fan, Z., & Roychoudhury, A. (2024) · Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis · doi:10.1145/3650212.3680384 · Cited by 202
“In this section, we discuss a few possible directions for future improvements on AutoCodeRover.”
AutoCodeRover: Autonomous Program Improvement
Zhang, Y., Ruan, H., Fan, Z., & Roychoudhury, A. (2024) · arXiv · doi:10.48550/arxiv.2404.05427 · Cited by 202
“In this section, we discuss a few possible directions for future improvements on AutoCodeRover.”
Automatically Discovering, Reporting and Reproducing Android Application Crashes
Moran, K., Linares‐Vásquez, M., Bernal-Cardenas, C., Vendome, C., & Poshyvanyk, D. (2016) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst.2016.34 · Cited by 181
“Several approaches for detecting and reproducing crashes are available in literature [30]– [32], [39], [40], [43], [51], [52], [63], [64], [69], [74], [78], [80]–[82]; however, we forgo discussion of these approaches, as they are not presented in the context of mobile apps, and hence do not consider the unique associa…”
BackgroundBEACON: Directed Grey-Box Fuzzing with Provable Path Pruning
Huang, H., Guo, Y., Shi, Q., Yao, P., Wu, R., & Zhang, C. (2022) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp46214.2022.9833751 · Cited by 147
“The idea of directed fuzzing begins with white-box fuzzing [5], [11], [48], which mainly depends on symbolic or concolic execution, such as Klee [49], to generate an exploitable input for bug reproduction.”
Background
Metadata
- Authors
- Jin, W., & Orso, A.
- Affiliations
- Georgia Institute of Technology (US)
- ORCID
- Orso, Alessandro
- Venue
- 2012 34th International Conference on Software Engineering (ICSE)
- Publisher
- IEEE
- Published
- 2012-06
- Pages
- 474-484
- ISSN
- 0270-5257 (ISSN-L)
- OpenAlex
- W3140804348
- Cited by
- 122
- References
- 35
- Citations by year
- 2026: 2 · 2025: 4 · 2024: 11 · 2023: 13 · 2022: 10 · 2021: 7 · 2020: 11 · 2019: 11 · 2018: 6 · 2017: 11 · 2016: 10 · 2015: 8 · 2014: 3 · 2013: 13 · 2012: 2
- FWCI
- 5.77
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Computer Networks and Communications › Software System Performance and Reliability
- Keywords
- field failures, insect reproduction, execution replay, bug reports, software debugging, cost-benefit analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.