The DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
An experimental determination of sufficient mutant operators
Open accessAbstract
Mutation testing is a technique for unit-testing software that, although powerful, is computationally expensive, The principal expense of mutation is that many variants of the test program, called mutants, must be repeatedly executed.This article quantifies the expense of mutation in terms of the number of mutants that are created, then proposes and evaluates a technique that reduces the number of mutants by an order of magnitude.Selective mutation reduces.the cost of mutation testing by reducing the number of mutants, This article reports experimental results that compare selective mutation testing with standard, or nonselective, mutation testing, and results that quantify the savings achieved by selective mutation testing, The results support the hypothesis that selective mutation is almost as strong as nonselective mutation: in experimental trials selective mutation provides almost the same coverage as nonselective mutation.with a four-fold or more reduction in the number of mutants.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
An Introduction to Statistical Methods and Data Analysis
(1994) · Technometrics · doi:10.1080/00401706.1994.10485832 · Cited by 5,346
Hints on Test Data Selection: Help for the Practicing Programmer
DeMillo, R. A., Lipton, R. J., & Sayward, F. G. (1978) · Computer · doi:10.1109/c-m.1978.218136 · Cited by 2,081
An Introduction to Statistical Methods and Data Analysis
Prybutok, V. R. (1989) · Technometrics · doi:10.1080/00401706.1989.10488572 · Cited by 1,499
Constraint-based automatic test data generation
DeMilli, R. A., & Offutt, A. J. (1991) · IEEE Transactions on Software Engineering · doi:10.1109/32.92910 · Cited by 804
Self-citationTesting Programs with the Aid of a Compiler
Hamlet, R. G. (1977) · IEEE Transactions on Software Engineering · doi:10.1109/tse.1977.231145 · Cited by 612
SAS for linear models : a guide to the ANOVA and GLM procedures
Freund, R. J., Littell, R. C., & Ray, A. A. (1981) · SAS Institute eBooks · Cited by 443
A fortran language system for mutation‐based software testing
King, K. N., & Offutt, A. J. (1991) · Software: Practice and Experience · doi:10.1002/spe.4380210704 · Cited by 379
Self-citationInvestigations of the software testing coupling effect
Offutt, A. J. (1992) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/125489.125473 · Cited by 371
Self-citationTwo notions of correctness and their relation to testing
Budd, T. A., & Angluin, D. (1982) · Acta Informatica · doi:10.1007/bf00625279 · Cited by 260
An extended overview of the Mothra software testing environment
DeMillo, R. A., Guindi, D. S., McCracken, W. M., Offutt, A. J., & King, K. N. (2003) · [1988] Proceedings. Second Workshop on Software Testing, Verification, and Analysis · doi:10.1109/wst.1988.5369 · Cited by 257
Self-citation
Cited by
An Analysis and Survey of the Development of Mutation Testing
Jia, Y., & Harman, M. (2011) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2010.62 · Cited by 1,745
Prioritizing test cases for regression testing
Rothermel, G., Untch, R. H., Chu, C., & Harrold, M. J. (2001) · IEEE Transactions on Software Engineering · doi:10.1109/32.962562 · Cited by 1,338
Self-citationDo, H., Elbaum, S., & Rothermel, G. (2005) · Empirical Software Engineering · doi:10.1007/s10664-005-3861-2 · Cited by 1,146
Self-citationIntroduction to Software Testing
(2021) · Essentials of Software Testing · doi:10.1017/9781108974073.004 · Cited by 1,056
Test case prioritization: a family of empirical studies
Elbaum, S., Malishevsky, A. G., & Rothermel, G. (2002) · IEEE Transactions on Software Engineering · doi:10.1109/32.988497 · Cited by 943
Self-citationIs mutation an appropriate tool for testing experiments?
Andrews, J. H., Briand, L. C., & Labiche, Y. (2005) · Proceedings of the 27th international conference on Software engineering - ICSE '05 · doi:10.1145/1062455.1062530 · Cited by 777
MuJava: an automated class mutation system
Ma, Y.-S., Offutt, J., & Kwon, Y. R. (2005) · Software Testing, Verification and Reliability · doi:10.1002/stvr.308 · Cited by 627
Are mutants a valid substitute for real faults in software testing?
Just, R., Jalali, D., Inozemtseva, L., Ernst, M. D., Holmes, R., & Fraser, G. (2014) · Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering · doi:10.1145/2635868.2635929 · Cited by 595
Using Mutation Analysis for Assessing and Comparing Testing Coverage Criteria
Andrews, J. H., Briand, L. C., Labiche, Y., & Namin, A. S. (2006) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2006.83 · Cited by 481
Mutation Testing Advances: An Analysis and Survey
(2019) · Advances in Computers · doi:10.1016/bs.adcom.2018.03.015 · Cited by 425
Testing and validating machine learning classifiers by metamorphic testing
Xie, X., Ho, J. W. K., Murphy, C., Kaiser, G., Xu, B., & Chen, T. Y. (2011) · Journal of Systems and Software · doi:10.1016/j.jss.2010.11.920 · Cited by 389
Metallaxis‐FL: mutation‐based fault localization
Papadakis, M., & Le Traon, Y. (2015) · Software Testing, Verification and Reliability · doi:10.1002/stvr.1509 · Cited by 371
Is mutation an appropriate tool for testing experiments?
Andrews, J. H., Brand, L. C., & Labiche, Y. (2005) · Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005. · doi:10.1109/icse.2005.1553583 · Cited by 368
Le, X. B. D., Lo, D., & Le Goues, C. (2016) · 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER) · doi:10.1109/saner.2016.76 · Cited by 332
Automatic Software Repair: A Survey
Gazzola, L., Micucci, D., & Mariani, L. (2019) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2017.2755013 · Cited by 327
Jia, Y., & Harman, M. (2009) · Information and Software Technology · doi:10.1016/j.infsof.2009.04.016 · Cited by 287
Mutation-Driven Generation of Unit Tests and Oracles
Fraser, G., & Zeller, A. (2012) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2011.93 · Cited by 283
PIT: a practical mutation testing tool for Java (demo)
Coles, H., Laurent, T., Henard, C., Papadakis, M., & Ventresque, A. (2016) · Proceedings of the 25th International Symposium on Software Testing and Analysis · doi:10.1145/2931037.2948707 · Cited by 270
A mutation carol: Past, present and future
Offutt, J. (2011) · Information and Software Technology · doi:10.1016/j.infsof.2011.03.007 · Cited by 81
Hajrizi, E., Zylfiu, B., & Menxhiqi, L. (2019) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2019.12.493 · Cited by 3
TestGenEval: A Real World Unit Test Generation and Test Completion Benchmark
Jain, K., Synnaeve, G., & Rozière, B. (2024) · arXiv · doi:10.48550/arxiv.2410.00752 · Cited by 3
Metadata
- Authors
- Offutt, A. J., Lee, A., Rothermel, G., Untch, R. H., & Zapf, C.
- Venue
- ACM Transactions on Software Engineering and Methodology
- Published
- 1996-04
- Volume
- 5
- Issue
- 2
- Pages
- 99-118
- ISSN
- 1049-331X (ISSN-L)
- 1557-7392
- OpenAlex
- W2121084350
- Cited by
- 650
- References
- 25
- FWCI
- 25.11
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.