The DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index.
Mutation-Driven Generation of Unit Tests and Oracles
Abstract
To assess the quality of test suites, mutation analysis seeds artificial defects (mutations) into programs; a nondetected mutation indicates a weakness in the test suite. We present an automated approach to generate unit tests that detect these mutations for object-oriented classes. This has two advantages: First, the resulting test suite is optimized toward finding defects modeled by mutation operators rather than covering code. Second, the state change caused by mutations induces oracles that precisely detect the mutants. Evaluated on 10 open source libraries, our μtest prototype generates test suites that find significantly more seeded defects than the original manually written test suites.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Greedy Heuristic for the Set-Covering Problem
Chvatal, V. (1979) · Mathematics of Operations Research · doi:10.1287/moor.4.3.233 · Cited by 2,581
Hints on Test Data Selection: Help for the Practicing Programmer
DeMillo, R. A., Lipton, R. J., & Sayward, F. G. (1978) · Computer · doi:10.1109/c-m.1978.218136 · Cited by 2,081
DART: directed automated random testing
Godefroid, P., Klarlund, N., & Sen, K. (2005) · ACM SIGPLAN Notices · doi:10.1145/1064978.1065036 · Cited by 2,067
An Analysis and Survey of the Development of Mutation Testing
Jia, Y., & Harman, M. (2011) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2010.62 · Cited by 1,745
CUTE: a concolic unit testing engine for C
Sen, K., Marinov, D., & Agha, G. (2005) · Proceedings of the 10th European software engineering conference held jointly with 13th ACM SIGSOFT international symposium on Foundations of software engineering · doi:10.1145/1081706.1081750 · Cited by 1,651
DART: directed automated random testing
Godefroid, P., Klarlund, N., & Sen, K. (2005) · Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation · doi:10.1145/1065010.1065036 · Cited by 1,570
Search‐based software test data generation: a survey
McMinn, P. (2004) · Software Testing, Verification and Reliability · doi:10.1002/stvr.294 · Cited by 1,368
Arcuri, A., & Briand, L. (2011) · Proceedings of the 33rd International Conference on Software Engineering · doi:10.1145/1985793.1985795 · Cited by 968
A Critique and Improvement of the CL Common Language Effect Size Statistics of McGraw and Wong
Vargha, A., & Delaney, H. D. (2000) · Journal of Educational and Behavioral Statistics · doi:10.3102/10769986025002101 · Cited by 938
A Critique and Improvement of the "CL" Common Language Effect Size Statistics of McGraw and Wong
Vargha, A., Delaney, H. D., & Vargha, A. (2000) · Journal of Educational and Behavioral Statistics · doi:10.2307/1165329 · Cited by 828
Constraint-based automatic test data generation
DeMilli, R. A., & Offutt, A. J. (1991) · IEEE Transactions on Software Engineering · doi:10.1109/32.92910 · Cited by 804
Pex–White Box Test Generation for .NET
Tillmann, N., & de Halleux, J. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-79124-9_10 · Cited by 791
Is mutation an appropriate tool for testing experiments?
Andrews, J. H., Briand, L. C., & Labiche, Y. (2005) · Proceedings of the 27th international conference on Software engineering - ICSE '05 · doi:10.1145/1062455.1062530 · Cited by 777
A methodology for controlling the size of a test suite
Harrold, M. J., Gupta, R., & Soffa, M. L. (1993) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/152388.152391 · Cited by 657
An experimental determination of sufficient mutant operators
Offutt, A. J., Lee, A., Rothermel, G., Untch, R. H., & Zapf, C. (1996) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/227607.227610 · Cited by 650
Randoop: feedback-directed random testing for Java
Pacheco, C., & Ernst, M. D. (2007) · Companion to the 22nd ACM SIGPLAN conference on Object-oriented programming systems and applications companion · doi:10.1145/1297846.1297902 · Cited by 558
Ali, S., Briand, L. C., Hemmati, H., & Panesar-Walawege, R. K. (2010) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2009.52 · Cited by 452
JCrasher: an automatic robustness tester for Java
Csallner, C., & Smaragdakis, Y. (2004) · Software: Practice and Experience · doi:10.1002/spe.602 · Cited by 426
Evolutionary testing of classes
Tonella, P. (2004) · Proceedings of the 2004 ACM SIGSOFT international symposium on Software testing and analysis · doi:10.1145/1007512.1007528 · Cited by 410
A Theoretical and Empirical Study of Search-Based Testing: Local, Global, and Hybrid Search
Harman, M., & McMinn, P. (2010) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2009.71 · Cited by 400
Investigations of the software testing coupling effect
Offutt, A. J. (1992) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/125489.125473 · Cited by 371
Is mutation an appropriate tool for testing experiments?
Andrews, J. H., Brand, L. C., & Labiche, Y. (2005) · Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005. · doi:10.1109/icse.2005.1553583 · Cited by 368
An empirical study of the effects of minimization on the fault detection capabilities of test suites
Rothermel, G., Harrold, M. J., Ostrin, J., & Hong, C. (2002) · International Conference on Software Maintenance · doi:10.1109/icsm.1998.738487 · Cited by 303
Harman, M., Hu, L., Hierons, R., Wegener, J., Sthamer, H., Baresel, A., & Roper, M. (2004) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2004.1265732 · Cited by 265
Eclat: Automatic Generation and Classification of Test Inputs
Pacheco, C., & Ernst, M. D. (2005) · Lecture Notes in Computer Science · doi:10.1007/11531142_22 · Cited by 250
PathCrawler: Automatic Generation of Path Tests by Combining Static and Dynamic Analysis
Williams, N., Marre, B., Mouy, P., & Roger, M. (2005) · Lecture Notes in Computer Science · doi:10.1007/11408901_21 · Cited by 199
ARTOO: adaptive random testing for object-oriented software
Ciupa, I., Leitner, A., Oriol, M., & Meyer, B. (2008) · Proceedings of the 13th international conference on Software engineering - ICSE '08 · doi:10.1145/1368088.1368099 · Cited by 198
Purity and Side Effect Analysis for Java Programs
Sălcianu, A., & Rinard, M. (2005) · Lecture notes in computer science · doi:10.1007/978-3-540-30579-8_14 · Cited by 193
The dynamic domain reduction procedure for test data generation
Offutt, A. J., Jin, Z., & Pan, J. (1999) · Software: Practice and Experience · doi:10.1002/(sici)1097-024x(199902)29:2<167::aid-spe225>3.0.co;2-v · Cited by 179
Sufficient mutation operators for measuring test effectiveness
Siami Namin, A., Andrews, J. H., & Murdoch, D. J. (2008) · Proceedings of the 13th international conference on Software engineering - ICSE '08 · doi:10.1145/1368088.1368136 · Cited by 178
Efficient mutation testing by checking invariant violations
Schüler, D., Dallmeier, V., & Zeller, A. (2009) · doi:10.1145/1572272.1572282 · Cited by 162
Self-citationMutation-driven generation of unit tests and oracles
Fraser, G., & Zeller, A. (2010) · International Symposium on Software Testing and Analysis · doi:10.1145/1831708.1831728 · Cited by 159
Self-citationInkumsah, K., & Xie, T. (2008) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2008.40 · Cited by 136
Augmenting Automatically Generated Unit-Test Suites with Regression Oracle Checking
Xie, T. (2006) · Lecture notes in computer science · doi:10.1007/11785477_23 · Cited by 128
Test generation via Dynamic Symbolic Execution for mutation testing
Zhang, L., Xie, T., Zhang, L., Tillmann, N., de Halleux, J., & Mei, H. (2010) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsm.2010.5609672 · Cited by 117
Search based software testing of object-oriented containers
Arcuri, A., & Yao, X. (2008) · Information Sciences · doi:10.1016/j.ins.2007.11.024 · Cited by 113
A Strategy for using Genetic Algorithms to Automate Branch and Fault-based Testing
Jones, B. F. (1998) · The Computer Journal · doi:10.1093/comjnl/41.2.98 · Cited by 112
Efficient unit test case minimization
Leitner, A., Oriol, M., Zeller, A., et al. (2007) · doi:10.1145/1321631.1321698 · Cited by 108
Self-citationUsing evolutionary algorithms for the unit testing of object-oriented software
Wappler, S., & Lammermann, F. (2005) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/1068009.1068187 · Cited by 104
DiffGen: Automated Regression Unit-Test Generation
Taneja, K., & Xie, T. (2008) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2008.60 · Cited by 79
Minimization of Randomized Unit Test Cases
Yong, L., & Andrews, J. H. (2006) · Proceedings/Proceedings - International Symposium on Software Reliability Engineering · doi:10.1109/issre.2005.28 · Cited by 69
Testing Container Classes: Random or Systematic?
Sharma, R., Gligoric, M., Arcuri, A., Fraser, G., & Marinov, D. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-19811-3_19 · Cited by 63
Self-citationOn the Predictability of Random Tests for Object-Oriented Software
Ciupa, I., Pretschner, A., Leitner, A., Oriol, M., & Meyer, B. (2008) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst.2008.20 · Cited by 57
Tool support for randomized unit testing
Andrews, J. H., Haldar, S., Yong, L., & Li, F. C. H. (2006) · doi:10.1145/1145735.1145741 · Cited by 56
Random Test Run Length and Effectiveness
Andrews, J. H., Groce, A., Weston, M., & Xu, R.-G. (2008) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2008.12 · Cited by 54
BERT: BEhavioral Regression Testing
Orso, A., & Xie, T. (2008) · doi:10.1145/1401827.1401835 · Cited by 28
It is Not the Length That Matters, It is How You Control It
Fraser, G., & Arcuri, A. (2011) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst.2011.54 · Cited by 27
Self-citation
Cited by
Are mutants a valid substitute for real faults in software testing?
Just, R., Jalali, D., Inozemtseva, L., Ernst, M. D., Holmes, R., & Fraser, G. (2014) · Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering · doi:10.1145/2635868.2635929 · Cited by 595
Self-citationFraser, G., & Arcuri, A. (2013) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2012.14 · Cited by 586
Self-citationMutation Testing Advances: An Analysis and Survey
(2019) · Advances in Computers · doi:10.1016/bs.adcom.2018.03.015 · Cited by 425
Metallaxis‐FL: mutation‐based fault localization
Papadakis, M., & Le Traon, Y. (2015) · Software Testing, Verification and Reliability · doi:10.1002/stvr.1509 · Cited by 371
Panichella, A., Kifetew, F. M., & Tonella, P. (2018) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2017.2663435 · Cited by 352
Metadata
- Authors
- Fraser, G., & Zeller, A.
- Venue
- IEEE Transactions on Software Engineering
- Published
- 2012-03
- Volume
- 38
- Issue
- 2
- Pages
- 278-292
- ISSN
- 0098-5589 (ISSN-L)
- OpenAlex
- W2018017297
- Cited by
- 283
- References
- 47
- FWCI
- 63.30
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.