BugRedux: Reproducing field failures for in-house debugging
The DOI matches its Crossref record; 34 of the 35 references it cites were found in a registry or an index.
Abstract
A recent survey conducted among developers of the Apache, Eclipse, and Mozilla projects showed that the ability to recreate field failures is considered of fundamental importance when investigating bug reports. Unfortunately, the information typically contained in a bug report, such as memory dumps or call stacks, is usually insufficient for recreating the problem. Even more advanced approaches for gathering field data and help in-house debugging tend to collect either too little information, and be ineffective, or too much information, and be inefficient. To address these issues, we present BugRedux, a novel general approach for in-house debugging of field failures. BugRedux aims to synthesize, using execution data collected in the field, executions that mimic the observed field failures. We define several instances of BugRedux that collect different types of execution data and perform, through an empirical study, a cost-benefit analysis of the approach and its variations. In the study, we apply BugRedux to 16 failures of 14 real-world programs. Our results are promising in that they show that it is possible to synthesize in-house executions that reproduce failures observed in the field using a suitable set of execution data.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Bug reproduction is a critical step in investigating bug reports [Jin and Orso 2012], and is integrated into many recent software engineering agents [Arora et al. 2024;Factory 2024;Honeycomb 2024;MentatBot 2024;OpenDevin 2024;Ruan et al. 2024;Yang et al. 2024a].”
From Demystifying LLM-Based Software Engineering Agents (2025) · cites this paper as Jin and Orso 2012
Citations
Cites
Symbolic execution and program testing
King, J. C. (1976) · Communications of the ACM · doi:10.1145/360248.360252 · Cited by 3,031 · cited 1×
“In this work, we decided to use an approach based on symbolic execution [18].”
MethodKLEE: unassisted and automatic generation of high-coverage tests for complex systems programs
Cadar, C., Dunbar, D., & Engler, D. (2008) · doi:10.5555/1855741.1855756 · Cited by 2,620
DART: directed automated random testing
Godefroid, P., Klarlund, N., & Sen, K. (2005) · Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation · doi:10.1145/1065010.1065036 · Cited by 1,570 · cited 1×
“, [20], [27]–[29]) and random generation (e.”
Automated Whitebox Fuzz Testing.
Godefroid, P., Levin, M. Y., & Molnar, D. A. (2008) · Cited by 1,095
Simplifying and isolating failure-inducing input
Zeller, A., & Hildebrandt, R. (2002) · IEEE Transactions on Software Engineering · doi:10.1109/32.988498 · Cited by 1,079 · cited 1×
“, [13], [14]), or are still in their early phase of development and in need of a more thorough”
BackgroundAre automated debugging techniques actually helping programmers?
Parnin, C., & Orso, A. (2011) · Proceedings of the 2011 International Symposium on Software Testing and Analysis · doi:10.1145/2001420.2001445 · Cited by 576 · cited 1×
“Although such an evaluation would be extremely useful, and we plan to do it in the future as we did for earlier work [26], we believe that it would be premature at this point.”
BackgroundSelf-citationExtracting usability information from user interface events
Hilbert, D. M., & Redmiles, D. F. (2000) · ACM Computing Surveys · doi:10.1145/371578.371593 · Cited by 477
Scalable statistical bug isolation
Liblit, B., Naik, M., Zheng, A. X., Aiken, A., & Jordan, M. I. (2005) · Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation · doi:10.1145/1065010.1065014 · Cited by 472 · cited 1×
“Among these techniques, some collect only limited amounts of information ( e.g., sampled branch profiles in the case of CBI [6], [7]).”
BackgroundA Theoretical and Empirical Study of Search-Based Testing: Local, Global, and Hybrid Search
Harman, M., & McMinn, P. (2010) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2009.71 · Cited by 400 · cited 1×
“, [38]), and techniques that take advantage of existing test inputs using some fort of fuzzing (e.”
BackgroundCompilers: Principles, Techniques, and Tools (2nd Edition)
Aho, A. V., Lam, M. S., Sethi, R., & Ullman, J. D. (2006) · Addison-Wesley Longman Publishing Co., Inc. eBooks · Cited by 387
Research works printed but not linked: 9
- jin, 2011, BugRedux Reproducing Field Failures for In-house Debugging
- 2012, Tmin Fuzzing Test Case Optimizer
- 2012, The Amazing VM Record/Replay Feature in VMware Workstation 6
- 2012, Software-artifact infrastructure repository
- 2012, Breakpad
- lu, 0, BugBench: Benchmarks for Evaluating Bug Detection Tools, Workshop on the Evaluation of Software Defect Detection Tools 2005
- 2012, Technical Note TN2123 CrashReporter
- 2012, Windows Error Reporting Getting Started
- pacheco, 0, Randoop: Feedback-Directed Random Testing for Java, Proceedings of the 22nd ACM SIGPLAN Conference on Object-oriented Programming Systems and Applications 2007, 815
Cited by
Böhme, M., Pham, V.-T., Nguyen, M.-D., & Roychoudhury, A. (2017) · Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3133956.3134020 · Cited by 747
AutoCodeRover: Autonomous Program Improvement
Zhang, Y., Ruan, H., Fan, Z., & Roychoudhury, A. (2024) · Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis · doi:10.1145/3650212.3680384 · Cited by 202
“In this section, we discuss a few possible directions for future improvements on AutoCodeRover.”
AutoCodeRover: Autonomous Program Improvement
Zhang, Y., Ruan, H., Fan, Z., & Roychoudhury, A. (2024) · arXiv · doi:10.48550/arxiv.2404.05427 · Cited by 202
“In this section, we discuss a few possible directions for future improvements on AutoCodeRover.”
Automatically Discovering, Reporting and Reproducing Android Application Crashes
Moran, K., Linares‐Vásquez, M., Bernal-Cardenas, C., Vendome, C., & Poshyvanyk, D. (2016) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst.2016.34 · Cited by 181
“Several approaches for detecting and reproducing crashes are available in literature [30]– [32], [39], [40], [43], [51], [52], [63], [64], [69], [74], [78], [80]–[82]; however, we forgo discussion of these approaches, as they are not presented in the context of mobile apps, and hence do not consider the unique associa…”
BackgroundBEACON: Directed Grey-Box Fuzzing with Provable Path Pruning
Huang, H., Guo, Y., Shi, Q., Yao, P., Wu, R., & Zhang, C. (2022) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp46214.2022.9833751 · Cited by 147
“The idea of directed fuzzing begins with white-box fuzzing [5], [11], [48], which mainly depends on symbolic or concolic execution, such as Klee [49], to generate an exploitable input for bug reproduction.”
BackgroundWhen Software Security Meets Large Language Models: A Survey
Zhu, X., Zhou, W., Han, Q., Ma, W., Wen, S., & Xiang, Y. (2025) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2024.124971 · Cited by 127
ReCDroid: Automatically Reproducing Android Application Crashes from Bug Reports
Zhao, Y., Yu, T., Su, T., Liu, Y., Zheng, W., Zhang, J., & Halfond, W. G. J. (2019) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2019.00030 · Cited by 113
“There are tools for automatically reproducing in-field failures from various sources, including core dumps [49], [56], function call sequences [31], call stack [50], and runtime logs [53], [54].”
MethodAuto-completing bug reports for Android applications
Moran, K., Linares‐Vásquez, M., Bernal-Cárdenas, C., & Poshyvanyk, D. (2015) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/2786805.2786857 · Cited by 93
“A body of work known as in-field failure reproduction [20, 43, 77, 30, 42, 16, 44, 27] shares similar goals with our approach.”
ResultR2Fix: Automatically Generating Bug Fixes from Bug Reports
Liu, C., Yang, J., Tan, L., & Hafiz, M. (2013) · 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation · doi:10.1109/icst.2013.24 · Cited by 96
DeepFault: Fault Localization for Deep Neural Networks
Enişer, H. F., Gerasimou, S., & Şen, A. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-16722-6_10 · Cited by 120
Demystifying LLM-Based Software Engineering Agents
Xia, C. S., Deng, Y., Dunn, S., & Zhang, L. (2025) · Proceedings of the ACM on Software Engineering · doi:10.1145/3715754 · Cited by 849
“Lambdify misinterprets some matrix expressions Using lambdify on an ... Bug reproduction is a critical step in investigating bug reports [Jin and Orso 2012], and is integrated into many recent software engineering agents [Arora et al. 2024; Factory 2024; Hon-eycomb 2024; MentatBot 2024; OpenDevin…”
STAR: Stack Trace Based Automatic Crash Reproduction via Symbolic Execution
Chen, N., & Kim, S. (2014) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2014.2363469 · Cited by 80
“Record-replay approaches can reliably replay recorded crashes, but they incur substantial performance overhead to program executions.”
BackgroundKhatiri, S., Panichella, S., & Tonella, P. (2023) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst57152.2023.00034 · Cited by 91
“Here, the questions are: 1) how to enable Alice to faithfully replicate a real UAV flight in simulation, by analyzing the flight log coming from an unknown environment? and 2) how to enable Alice to test the UAV in a set of diversified possible scenarios in the neighborhood of the given field test? Software engineerin…”
BackgroundNuances are the Key: Unlocking ChatGPT to Find Failure-Inducing Tests with Differential Prompting
Li, T.-O., Zong, W., Wang, Y., Tian, H., Wang, Y., Cheung, S.-C., & Kramer, J. (2023) · 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase56229.2023.00089 · Cited by 57
“To the best of our knowledge, the most closely-related works would be those target the problem of failing test reproduction [36], [37], [38], failing test augmentation [39], [40], or fuzzing [41], [42], [43].”
BackgroundCharacteristic studies of loop problems for structural test generation via symbolic execution
Xiao, X., Li, S., Xie, T., & Tillmann, N. (2013) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2013.6693084 · Cited by 96
“1: Are loop problems a kind of major problems to compromise the effectiveness of symbolic execution in various tasks? Symbolic execution has been applied for various tasks, such as test generation [18], [25], [38], [41], [43], [46], [47], [51], [53], debugging [31], security analysis [9], [12], [49], verification [21]…”
BackgroundChangeLocator: locate crash-inducing changes based on crash reports
Wu, R., Wen, M., Cheung, S.-C., & Zhang, H. (2017) · Empirical Software Engineering · doi:10.1007/s10664-017-9567-4 · Cited by 55
“BugRedux (Jin and Orso 2012) records different kinds of execution data, and reproduces crashes using symbolic analysis.”
BackgroundSearch-Based Crash Reproduction and Its Impact on Debugging
Soltani, M., Panichella, A., & Arie Van Deursen, A. (2018) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2018.2877664 · Cited by 53
Rößler, J., Zeller, A., Fraser, G., Zamfir, C., & Candea, G. (2013) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst.2013.18 · Cited by 51
“The basic idea of RECORE is to use EVOSUITE [11] to produce a test case that is as similar to the failure as possible.”
MethodA Guided Genetic Algorithm for Automated Crash Reproduction
Soltani, M., Panichella, A., & Arie Van Deursen, A. (2017) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2017.27 · Cited by 51
“BugRedux [18] also uses forward symbolic execution but it can analyze different types of execution data, such as crash stack traces.”
MethodCausal testing: understanding defects' root causes
Johnson, B., Brun, Y., & Meliou, A. (2020) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3377811.3380377 · Cited by 78
“Reproducing field failures [37] is an important part of debugging complementary to most of the above-described techniques, including Causal Testing, which require a failing test case.”
MethodDebugging Revisited: Toward Understanding the Debugging Needs of Contemporary Software Developers
Layman, L., Diep, M., Nagappan, M., Singer, J., Deline, R., & Venolia, G. (2013) · 2013 ACM / IEEE International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem.2013.43 · Cited by 42
Sequence Coverage Directed Greybox Fuzzing
Liang, H., Zhang, Y., Yu, Y., Xie, Z., & Jiang, L. C. (2019) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc.2019.00044 · Cited by 43
“We evaluate the effectiveness of LOLLY in crash reproduction and compare it with AFLGo and BugRedux [21].”
MethodGenerating Reproducible and Replayable Bug Reports from Android Application Crashes
White, M., Linares‐Vásquez, M., Johnson, P., Bernal-Cárdenas, C., & Poshyvanyk, D. (2015) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc.2015.14 · Cited by 42
“A number of techniques for faultlocalization based on crash reports have been proposed [33]– [43].”
BackgroundSok: The Progress, Challenges, and Perspectives of Directed Greybox Fuzzing
Wang, P., Zhou, X., Yue, T., Lin, P., Liu, Y., & Lü, K. (2022) · Software Testing Verification and Reliability · doi:10.1002/stvr.1869 · Cited by 40
1dVul: Discovering 1-Day Vulnerabilities through Binary Patches
Peng, J., Li, F., Liu, B., Xu, L., Liu, B.-H., Chen, K., & Huo, W. (2019) · International Conference on Dependable Systems & Networks/Proceedings · doi:10.1109/dsn.2019.00066 · Cited by 37
“Techniques focus on crash reproduction, such as BugRedux [29], RECORE [30] and Star [31], require an input that can already reach the target location, while approaches solving the ToIG problem need to generate such an input by themselves.”
BackgroundSequence Directed Hybrid Fuzzing
Liang, H., Jiang, L. C., Ai, L., & Wei, J. (2020) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner48275.2020.9054807 · Cited by 37
“BugRedux [30] takes as input a sequence of program statements and generates as output a test case that exercises that sequence and crashes the program.”
BackgroundJCHARMING: A bug reproduction approach using crash traces and directed model checking
Nayrolles, M., Hamou‐Lhadj, A., Tahar, S., & Larsson, A. (2015) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner.2015.7081820 · Cited by 36
Monitoring user interactions for supporting failure reproduction
Roehm, T., Gurbanova, N., Bruegge, B., Joubert, C., & Maalej, W. (2013) · International Conference on Program Comprehension · doi:10.1109/icpc.2013.6613835 · Cited by 31
“Orso and Kennedy [15] selectively monitor the executed methods, field accesses and exceptions.”
MethodA benchmark-based evaluation of search-based crash reproduction
Soltani, M., Derakhshanfar, P., Devroey, X., & Arie Van Deursen, A. (2019) · Empirical Software Engineering · doi:10.1007/s10664-019-09762-1 · Cited by 31
“They, however, differ in their means to achieve this task: for instance, ESD (Zamfir and Candea 2010) and BugRedux (Jin and Orso 2012) use forward symbolic execution; STAR (Chen and Kim 2015) applies optimized backward symbolic execution and a novel technique for method sequence composition;…”
BackgroundMethodStatSym: Vulnerable Path Discovery through Statistics-Guided Symbolic Execution
Yao, F., Li, Y., Chen, Y., Xue, H., Lan, T., & Venkataramani, G. (2017) · International Conference on Dependable Systems & Networks/Proceedings · doi:10.1109/dsn.2017.57 · Cited by 30
Soremekun, E., Pavese, E., Havrikov, N., Grunske, L., & Zeller, A. (2020) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2020.3013716 · Cited by 35
“The structural measure is the frequency of call sequences induced in the program, a similar measure was used in [9].”
MethodEnhancing Mobile App Bug Reporting via Real-Time Understanding of Reproduction Steps
Fazzini, M., Moran, K., Bernal-Cárdenas, C., Wendland, T., Orso, A., & Poshyvanyk, D. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2022.3174028 · Cited by 31
“Related work on field failure reproduction proposed techniques to reproduce failures using genetic programming [72], [73], crowdsourcing [74], symbolic execution [75], [76], [77], [78], static analysis combined with symbolic execution [79], model checking [80], dynamic exploration [81], sequential pattern minin on use…”
MethodSelf-citationChronicler: lightweight recording to reproduce field failures
Bell, J., Sarda, N., & Kaiser, G. E. (2013) · doi:10.5555/2486788.2486836 · Cited by 30
Behavioral Execution Comparison: Are Tests Representative of Field Behavior?
Wang, Q., Brun, Y., & Orso, A. (2017) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst.2017.36 · Cited by 29
“We elected to work at the method level because prior work has argued that method call sequences represent the best cost-benefit tradeoff for reproducing field failures [31].”
MethodSelf-citationChronicler: Lightweight recording to reproduce field failures
Bell, J., Sarda, N., & Kaiser, G. E. (2013) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2013.6606582 · Cited by 30
“BugRedux [28] can use four different kinds of execution data: points of failure, call stack at failure, call sequences and complete program traces.”
BackgroundMethodInfluentialA Large-Scale Empirical Study on Control Flow Identification of Smart Contracts
Chen, T., Zhang, X., Li, Z., Zhang, Y., Luo, X., Wang, T., Hu, T., Xiao, X., Wang, D., & Huang, J. (2019) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem.2019.8870156 · Cited by 31
“, bug diagnosis [35], feature location [36], test case extraction [37], task offloading [38], fuzzing [39], and software protection [40].”
BackgroundMethodLocalizing Vulnerabilities Statistically From One Exploit
Shen, S., Kolluri, A., Dong, Z., Saxena, P., & Roychoudhury, A. (2021) · ACM Asia Conference on Computer and Communications Security (ASIA CCS) · doi:10.1145/3433210.3437528 · Cited by 43
“It builds on the techniques proposed in BugRedux [17].”
MethodReproducing Field Failures for Programs with Complex Grammar-Based Input
Kifetew, F. M., Jin, W., Tiella, R., Orso, A., & Tonella, P. (2014) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst.2014.29 · Cited by 25
“…technique, called SBFR (Search-Based Failure Reproduction), that is specifically designed to handle complex programs with highly structured inputs (e.g., compilers and interpreters), hence going beyond the features of the programs used in previous failure reproduction studies [3, 4, 5, 6, 7, 8, 9].”
BackgroundMethodInfluentialSelf-citationDemystifying the challenges and benefits of analyzing user-reported logs in bug reports
Chen, A. R., Chen, T.-H. P., & Wang, S. (2021) · Empirical Software Engineering · doi:10.1007/s10664-020-09893-w · Cited by 37
“To assist developers with debugging, prior studies (Jin and Orso 2012; Yuan et al. 2010, 2011; Wu et al. 2014; Soltani et al. 2018; Bianchi et al. 2017; Hassani et al. 2018; Li et al. 2020a) propose approaches to analyze crash reports or system logs.”
BackgroundMethodExecution reconstruction: harnessing failure reoccurrences for failure reproduction
Zuo, G., Ma, J., Quinn, A., Bhatotia, P., Fonseca, P., & Kasikci, B. (2021) · ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI) · doi:10.1145/3453483.3454101 · Cited by 24
“BugRedux [69] uses symbolic execution to synthesize an execution that reaches the locations traced by one of two approaches: complete tracing records all control flow (up to 10× overhead), or the more efficient call sequence tracing, which records each function call (between 2%ś50% overhead).”
BackgroundMethodGenerating Reproducible and Replayable Bug Reports from Android Application Crashes
White, M., Johnson, P., & Poshyvanyk, D. (2016) · Cited by 24
Hercules: Reproducing Crashes in Real-World Application Binaries
Pham, V.-T., Ng, W. B., Rubinov, K., & Roychoudhury, A. (2015) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2015.99 · Cited by 24
Lightweight control-flow instrumentation and postmortem analysis in support of debugging
Ohmann, P., & Liblit, B. (2013) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2013.6693096 · Cited by 23
A bug reproduction approach based on directed model checking and crash traces
Nayrolles, M., Hamou‐Lhadj, A., Tahar, S., & Larsson, A. (2016) · Journal of Software Evolution and Process · doi:10.1002/smr.1789 · Cited by 23
“Jin et al. proposed BugRedux for reproducing fi eld failures for in-house debugging [11].”
BackgroundReproducing concurrency failures from crash stacks
Bianchi, F. A., Pezzè, M., & Terragni, V. (2017) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3106237.3106292 · Cited by 23
“A crash stack trace (or simply crash stack) reports the ordered sequences of functions on the call stack at the time of the failure and terminates the sequence with the exception that results from the failure (NullPointerException at line 1 in Figure 2) [25].”
BackgroundDESCRY: reproducing system-level concurrency failures
Yu, T., Zaman, T. S., & Wang, C. (2017) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3106237.3106266 · Cited by 23
“[18] use various runtime information of the failed execution to guide the generation of program inputs.”
MethodSingle-objective Versus Multi-objectivized Optimization for Evolutionary Crash Reproduction
Soltani, M., Derakhshanfar, P., Panichella, A., Devroey, X., Zaidman, A., & Arie Van Deursen, A. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-99241-9_18 · Cited by 21
“While BugRedux [14] is based on forward symbolic execution, Star [4] is a more recent approach that applies optimized backward symbolic execution on the method calls recorded in a stack trace in order to compute the input parameters that trigger the target crash.”
MethodCausal testing: understanding defects' root causesPreprint
Johnson, B., Brun, Y., & Meliou, A. (2020) · arXiv (Cornell University) · Cited by 21
Agentless: Demystifying LLM-based Software Engineering Agents
Xia, C. S., Deng, Y., Dunn, S., & Zhang, L. (2024) · arXiv · doi:10.48550/arxiv.2407.01489 · Cited by 24
“Bug reproduction is a critical step in investigating bug reports [46], and is integrated into many recent software engineering agents [17, 81, 10, 14, 9, 101, 27].”
BackgroundAn effective formulation of the multi-criteria test suite minimization problem
Özener, O. Ö., & Sözer, H. (2020) · Journal of Systems and Software · doi:10.1016/j.jss.2020.110632 · Cited by 17
Fonte: Finding Bug Inducing Commits from Failures
An, G., Hong, J., Kim, N., & Yoo, S. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse48619.2023.00059 · Cited by 25
Causal Distance-Metric-Based Assistance for Debugging after Compiler Fuzzing
Holmes, J., & Groce, A. (2018) · Proceedings/Proceedings - International Symposium on Software Reliability Engineering · doi:10.1109/issre.2018.00027 · Cited by 17
“[83] show that FPF based on a simple Hamming distance over branches covered can significantly improve the effectiveness of seeded symbolic execution [53], [63], [37].”
MethodClassifying Bugs with Interpolants
Podelski, A., Schäf, M., & Wies, T. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-41135-4_9 · Cited by 16
“Recently, techniques have been explored to automatically reconstruct the actual error trace from a field failure by using symbolic execution [18].”
MethodG-Fuzz: A Directed Fuzzing Framework for gVisor
Li, Y., Chen, Y., Ji, S., Zhang, X., Yan, G., Liu, A. X., Wu, C., Pan, Z., & Lin, P. (2023) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2023.3244825 · Cited by 15
“The currentdirectedfuzzingtechniquescanbemainlycategorized intotwoclasses:symbolicexecutionbasedwhiteboxdirected fuzzing[5],[6]anddirectedgreyboxfuzzing(DGF)[7],[8].”
RENN: Efficient Reverse Execution with Neural-Network-Assisted Alias Analysis
Mu, D., Guo, W., Cuevas, A., Chen, Y., Gai, J., Xing, X., Mao, B., & Song, C. (2019) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2019.00090 · Cited by 14
Help, help, i'm being suppressed! The significance of suppressors in software testing
Groce, A., Zhang, C., Alipour, M. A., Eide, E. N., Chen, Y., & Regehr, J. D. (2013) · Proceedings/Proceedings - International Symposium on Software Reliability Engineering · doi:10.1109/issre.2013.6698892 · Cited by 14
“As Jin and Orso have shown, symbolic testing can scale better for finding some faults when given a call sequence [28].”
BackgroundEnhancing Android application bug reporting
Moran, K. (2015) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/2786805.2807557 · Cited by 14
“In-Field Failure Reproduction: A body of work known as in-field failure reproduction [8, 15, 21, 12, 14, 7, 16, 11] shares similar goals with our approach.”
Mimicking Production Behavior With Generated Mocks
Tiwari, D., Monperrus, M., & Baudry, B. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3458448 · Cited by 20
“The captured sequence of events leading up to a crash or failure allows for more efficient debugging when replayed offline by developers [74], [75], as well as”
Masri, W. (2015) · Advances in computers · doi:10.1016/bs.adcom.2015.05.001 · Cited by 15
“[43], Jin and Orso presented BugRedux, an approach for synthesizing executions that mimic an observed field failure.”
BackgroundMethodA bounded symbolic-size model for symbolic execution
Trabish, D., Itzhaky, S., & Rinetzky, N. (2021) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3468264.3468596 · Cited by 11
“It lies at the core of many applications such as high-coverage test generation [14, 15, 32], bug finding [14, 20], debugging [23], automatic program repair [28, 30], cross checking [16, 25], and side-channel”
BackgroundImproving Fault Localization Using Model-domain Synthesized Failing Test Generation
Zhang, Z., Lei, Y., Mao, X., Yan, M. L., & Xia, X. (2022) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme55016.2022.00026 · Cited by 12
“[23] propose BugRedux, which collects different types of execution data in the field and mimics the observed field failures for F3 technique [22].”
MethodHytrace: A Hybrid Approach to Performance Bug Diagnosis in Production Cloud Infrastructures
Dai, T., Dean, D. J., Wang, P., Gu, X., & Lu, S. (2018) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2018.2858800 · Cited by 12
UCov: a user‐defined coverage criterion for test case intent verification
Assi, R. A., Masri, W., & Zaraket, F. A. (2016) · Software Testing Verification and Reliability · doi:10.1002/stvr.1607 · Cited by 10
“For that purpose, work related to test suite augmentation [23, 25, 26], field failure reproduction [24] and coverage of software patches [48], could be leveraged.”
BackgroundGuided Algebraic Specification Mining for Failure Simplification
Elyasov, A., Prasetya, I. S. W. B., & Hage, J. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-41707-8_15 · Cited by 9
“On the other hand, when a failure occurs logs, also become valuable for the failure context that helps to track down the root cause of the failure, or at least to reproduce the failure [13,11].”
BackgroundStatistical Reasoning About Programs
Böhme, M. (2022) · IEEE/ACM International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER) · doi:10.1109/icse-nier55298.2022.9793535 · Cited by 10
Du, X., Liu, M., Wang, H., Li, J., Peng, X., & Lou, Y. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3788866 · Cited by 16
“Software crashes pose an enduring challenge in software development, driving research in areas such as crash reproduction [42, 56], crash localization [32, 61, 70, 73], and crash repair [35, 52, 68].”
Automated extraction of failure reproduction steps from user interaction traces
Roehm, T., Nosović, S., & Bruegge, B. (2015) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner.2015.7081822 · Cited by 8
“F3 [12] extends BugRedux to synthesize passing and failing executions and exploits them to identify potentially faulty program entities.”
FDG: a precise measurement of fault diagnosability gain of test cases
An, G., & Yoo, S. (2022) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3533767.3534370 · Cited by 11
“> 3 [24] is a fault localisation technique for "eld failures, which extends a bug reproduction technique, BugRedux [23]: it synthesises failing and passing executions similar to the "eld failure and uses them for its customised fault localisation technique.”
MethodAn empirical study of non-reproducible bugs
Goyal, A., & Sardana, N. (2019) · International Journal of Systems Assurance Engineering and Management · doi:10.1007/s13198-019-00850-5 · Cited by 8
SBFR: A search based approach for reproducing failures of programs with grammar based input
Kifetew, F. M., Jin, W., Tiella, R., Orso, A., & Tonella, P. (2013) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2013.6693120 · Cited by 7
“Similar to ESD and CBZ, BugRedux [6] is a general approach for synthesizing, in-house, executions that mimic observed field failures.”
BackgroundMethodResultInfluentialSelf-citationIn the Field Monitoring of Interactive Application
Cornejo, O., Briola, D., Micucci, D., & Mariani, L. (2017) · IEEE/ACM International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER) · doi:10.1109/icse-nier.2017.19 · Cited by 7
“Many techniques exploit this type of field data, such as techniques for reproducing failures [9], profiling users [6], and controlling software evolution [11].”
BackgroundMethodReproducing failures based on semiformal failure scenario descriptions
Karagöz, G., & Sözer, H. (2016) · Software Quality Journal · doi:10.1007/s11219-016-9310-1 · Cited by 7
“A promising tool to reproduce field failures for debugging was presented (Jin and Orso 2012).”
MethodYe, H., Chen, Z., & Goues, C. L. (2023) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase56229.2023.00163 · Cited by 14
“More-over, there exist bug repositories that also contain failing tests that expose the bugs; this is crucial for buggy code reproduction and execution [46].”
BackgroundAhmad, A. (2022) · Linköping studies in science and technology. Dissertations · doi:10.3384/9789179294236 · Cited by 7
Derakhshanfar, P., Devroey, X., Zaidman, A., Arie Van Deursen, A., & Panichella, A. (2020) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3324884.3416643 · Cited by 6
“Among the various approaches solely using a stack trace as input, STAR [6] and BugRedux [23] use backward and forward symbolic execution, respectively; MuCrash [44] mutates the existing test cases of the classes involved in the stack trace; JCharming [31, 32] applies model checking and program slicing for crash reprod…”
BackgroundMethodImproving Testing by Mimicking User Behavior
Wang, Q., & Orso, A. (2020) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme46990.2020.00053 · Cited by 6
“BugRedux [9] uses guided symbolic execution to synthesize executions that reproduce failed failures.”
MethodResultSelf-citationOvercoming language dichotomies: toward effective program comprehension for mobile app development
Moran, K., Bernal-Cárdenas, C., Linares‐Vásquez, M., & Poshyvanyk, D. (2018) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1145/3196321.3196322 · Cited by 7
“Bug and Error reporting is an important activity for any type of software system, and techniques for bug triaging [22, 54, 62, 63, 78, 101, 120], duplicate report detection [46, 48, 58, 103, 124, 131], summarization [27, 34, 64, 87, 113, 126], and reporting of in-field failures [25, 28, 33, 55, 56, 61, 132] have been…”
BackgroundROSE: An IDE-Based Interactive Repair Framework for Debugging
Reiss, S. P., Wei, X., Yuan, J. P., & Xin, Q. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3705306 · Cited by 7
“In a realistic debugging scenario, recreating the environment for the immediate failure caused by the error can be difficult [44, 48, 99, 106] since the failure can be identified in a long run or in an interactive session.”
State Merging with Quantifiers in Symbolic Execution
Trabish, D., Rinetzky, N., Shoham, S., & Sharma, V. (2023) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3611643.3616287 · Cited by 5
“It lies at the core of many applications such as high-coverage test generation [16, 17, 39], bug finding [16, 28], debugging [31], automatic program repair [36, 37], cross checking [19, 32], and sidechannel analysis [14, 15, 38].”
BackgroundAutomatic Reproducible Crash Detection
Gu, Y., Xuan, J., & Qian, T. (2016) · International Conference on Software Analysis, Testing and Evolution (SATE) · doi:10.1109/sate.2016.15 · Cited by 5
“Many automatic test generation approaches [5], [6], [7], [8], [9] have been proposed in recent years to reduce the manual cost of crash reproduction.”
BackgroundMethodRoBin: Facilitating the Reproduction of Configuration-Related Vulnerability
Chen, L., Guo, J., He, Z., Mu, D., & Mao, B. (2021) · IEEE International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom) · doi:10.1109/trustcom53373.2021.00030 · Cited by 5
“BugRedux [6] conducted one empirical analysis, to understand the trade-offs between different execution data recording and the effectiveness of approaches.”
MethodAddress-Aware Query Caching for Symbolic Execution
Trabish, D., Itzhaky, S., & Rinetzky, N. (2021) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst49551.2021.00023 · Cited by 4
“Symbolic execution (SE) is a popular program analysis technique, that lies at the core of many applications including automatic test generation [16], [17], [39], bug finding [16], [26], debugging [29], patch testing [42], [33], automatic program repair [36], [34], and quantitative program analysis [22], [15].”
BackgroundConcretely Mapped Symbolic Memory Locations for Memory Error Detection
Tu, H., Jiang, L., Hong, J., Ding, X., & Jiang, H. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3395412 · Cited by 4
“…become a popular technique broadly adopted in many domains, e.g., test case generation [13], [16], [17], bug-detection [13], [18], [19], debugging and repairing [20], [21], [22], [23], [24], cross-checking [25], [26], side-channel analysis [27], [28], [29], and exploit generation [30], [31], [32].”
BackgroundOh, C., Schäf, M., Schwartz‐Narbonne, D., & Wies, T. (2014) · IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM) · doi:10.1109/scam.2014.22 · Cited by 4
“The actual source code and the test input that reveals the bugs are taken from the BugRedux [14], [16] distribution. sed is a UNIX utility which parses and transforms text, based on a compact scripting language.”
MethodInfluentialCustomized Regression Testing Using Telemetry Usage Patterns
Anderson, J., Do, H., & Salem, S. (2016) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme.2016.30 · Cited by 4
“[3]), and failure reproduction (Jin and Orso [11]).”
BackgroundContextAug: model-domain failing test augmentation with contextual information
Zhang, Z., Xue, J., Yang, D., & Mao, X. (2023) · Frontiers of Computer Science · doi:10.1007/s11704-023-2521-2 · Cited by 4
A study of event frequency profiling with differential privacy
Zhang, H., Hao, Y., Latif, S., Bassily, R., & Rountev, A. (2020) · doi:10.1145/3377555.3377887 · Cited by 6
“Other related uses of such remote analysis include debugging [26, 34] and reproduction of field failures [11, 24].”
BackgroundMethodCornejo, O., Briola, D., Micucci, D., & Mariani, L. (2017) · Electronic Proceedings in Theoretical Computer Science · doi:10.4204/eptcs.254.5 · Cited by 4
“On the other hand, Jin and Orso [18] proposed BugRedux, an approach for supporting in-house debugging of field failures.”
MethodInfluentialRLF: Directed Fuzzing based on Deep Reinforcement Learning
Liang, X., & Xiao, T. (2022) · International Conference on Machine Learning, Control, and Robotics (MLCR) · doi:10.1109/mlcr57210.2022.00032 · Cited by 4
“In the crash reproduction comparison experiment with directed white-box fuzzing BugRedux [14], AFLGO reproduced three times as many methods calls as BugRedux.”
MethodA family of experiments about how developers perceive delayed system response time
Cornejo, O., Briola, D., Micucci, D., Ginelli, D., Mariani, L., Parrilla, A. S., & Juristo, N. (2024) · Software Quality Journal · doi:10.1007/s11219-024-09660-w · Cited by 4
“In particular, field monitoring has been used to collect data that can support the reproduction and analysis of failures (Jin & Orso, 2012; Clause & Orso, 2007; Liblit et al., 2003; Jin et al., 2010), but also to collect additional evidence about the correctness of software systems, such as…”
MethodAhmad, A., Neto, F. G. D. O., Enoiu, E. P., Sandahl, K., & Leifler, O. (2023) · IEEE International Conference on Software Quality, Reliability and Security (QRS) · doi:10.1109/qrs60937.2023.00041 · Cited by 4
“On the other hand, their automated tests still faces obstacles with flaky tests and the ability to reproduce failures when debugging which require specific techniques, such as, respectively, flaky test detectors [32], [33] or logging and approaches that mimic test execution [34], [35].”
BackgroundPP-CSA: Practical Privacy-Preserving Software Call Stack Analysis
Wang, Z., Ma, P., Wang, H., & Wang, S. (2024) · Proceedings of the ACM on Programming Languages · doi:10.1145/3649856 · Cited by 4
“It facilitates analyzing the runtime behavior of software in order to optimize software performance, identify defects, and profile software [Han et al. 2012; Hao et al. 2021; Jin and Orso 2012].”
MethodField Monitoring With Delayed Saving
Cornejo, O., Ginelli, D., Briola, D., Micucci, D., & Mariani, L. (2019) · IEEE Access · doi:10.1109/access.2019.2925855 · Cited by 3
“Although the presence of inaccuracy is not acceptable by all the techniques, it is indeed tolerable by several others, such as user profiling [2], failure reproduction [16], and mining techniques [17], [18].”
InfluentialGoyal, A., & Sardana, N. (2020) · Journal of Software Evolution and Process · doi:10.1002/smr.2290 · Cited by 3
“Joshi and Orso 43 proposed SCRAPE, a tool for selective capture and replay of program executions.”
BackgroundRDE: Replay DEbugging for Diagnosing Production Site Failures
Wang, P., Nguyen, H., Gu, X., & Lu, S. (2016) · IEEE Symposium on Reliable Distributed Systems (SRDS) · doi:10.1109/srds.2016.050 · Cited by 3
“Researchers have proposed different techniques, ranging from complete execution recording [14], [15], [17], [27] to partial record-replay [3], [8], [18], [19], [24], [28], [31], [32], in order to achieve debug determinism.”
MethodCBR: Controlled Burst Recording
Cornejo, O., Briola, D., Micucci, D., & Mariani, L. (2020) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst46399.2020.00033 · Cited by 2
DCLINK: Bridging Data Constraint Changes and Implementations in FinTech Systems
Tange, W., Wang, C., Yao, P., Wu, R., Fu, X., Fan, G., & Zhang, C. (2023) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase56229.2023.00170 · Cited by 2
“The other takes natural language descriptions of data constraints as the inputs for the mapping [3].”
BackgroundMimicking User Behavior to Improve In-House Test Suites
Wang, Q., & Orso, A. (2019) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse-companion.2019.00133 · Cited by 2
“This allows Replica to explore the execution space without over-constraining the search—which we found to be problematic in previous work [3]—and by adding additional constraints only as needed.”
MethodSelf-citationAn Empirical Study on the Distance Metric in Guiding Directed Grey-box Fuzzing
Wen, T., Li, Y., Zhang, L., Ma, H., & Pan, Z. (2024) · Proceedings/Proceedings - International Symposium on Software Reliability Engineering · doi:10.1109/issre62328.2024.00038 · Cited by 2
“The directed white-box fuzzing mainly uses symbolic execution engines to generate inputs that can efficiently explore the state space of feasible paths [6, 30, 31].”
Flexible In-the-Field Monitoring
Cornejo, O. (2017) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse-c.2017.37 · Cited by 2
“Data retrieved from the field have been also exploited to profile applications [2], debug software [3], and reproduce failures [4].”
Method
Metadata
- Authors
- Jin, W., & Orso, A.
- Affiliations
- Georgia Institute of Technology (US)
- ORCID
- Orso, Alessandro
- Venue
- 2012 34th International Conference on Software Engineering (ICSE)
- Publisher
- IEEE
- Published
- 2012-06
- Pages
- 474-484
- ISSN
- 0270-5257 (ISSN-L)
- OpenAlex
- W3140804348
- Cited by
- 122
- References
- 35
- Citations by year
- 2026: 2 · 2025: 4 · 2024: 11 · 2023: 13 · 2022: 10 · 2021: 7 · 2020: 11 · 2019: 11 · 2018: 6 · 2017: 11 · 2016: 10 · 2015: 8 · 2014: 3 · 2013: 13 · 2012: 2
- FWCI
- 5.77
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Computer Networks and Communications › Software System Performance and Reliability
- Keywords
- field failures, insect reproduction, execution replay, bug reports, software debugging, cost-benefit analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.