The DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.
Is mutation an appropriate tool for testing experiments?
Abstract
The empirical assessment of test techniques plays an important role in software testing research. One common practice is to instrument faults, either manually or by using mutation operators. The latter allows the systematic, repeatable seeding of large numbers of faults; however, we do not know whether empirical results obtained this way lead to valid, representative conclusions. This paper investigates this important question based on a number of programs with comprehensive pools of test cases and known faults. It is concluded that, based on the data available thus far, the use of mutation operators is yielding trustworthy results (generated mutants are similar to real faults). Mutants appear however to be different from hand-seeded faults that seem to be harder to detect than real faults.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(1963) · Medical Entomology and Zoology · Cited by 15,085
Hints on Test Data Selection: Help for the Practicing Programmer
DeMillo, R. A., Lipton, R. J., & Sayward, F. G. (1978) · Computer · doi:10.1109/c-m.1978.218136 · Cited by 2,081
Experimentation in Software Engineering
Wohlin, C., Runeson, P., Höst, M., Ohlsson, M. C., Regnell, B., & Wesslén, A. (2000) · The Kluwer International Series in Software Engineering · doi:10.1007/978-1-4615-4625-2 · Cited by 1,776
The category-partition method for specifying and generating functional tests
Ostrand, T. J., & Balcer, M. J. (1988) · Communications of the ACM · doi:10.1145/62959.62964 · Cited by 791
Experiments on the effectiveness of dataflow- and control-flow-based test adequacy criteria
Hutchins, M., Foster, H. D., Goradia, T., & Ostrand, T. J. (1994) · doi:10.5555/257734.257766 · Cited by 774
An experimental determination of sufficient mutant operators
Offutt, A. J., Lee, A., Rothermel, G., Untch, R. H., & Zapf, C. (1996) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/227607.227610 · Cited by 650
Testing Programs with the Aid of a Compiler
Hamlet, R. G. (1977) · IEEE Transactions on Software Engineering · doi:10.1109/tse.1977.231145 · Cited by 612
Probability and Statistics for Engineering and the Sciences.
Vardeman, S., & Devore, J. L. (1982) · Journal of the American Statistical Association · doi:10.2307/2287340 · Cited by 417
Experiments on the effectiveness of dataflow- and control-flow-based test adequacy criteria
Hutchins, M., Foster, H., Goradia, T., & Ostrand, T. (2002) · Proceedings of 16th International Conference on Software Engineering · doi:10.1109/icse.1994.296778 · Cited by 375
Mutation 2000: Uniting the Orthogonal
Offutt, A. J., & Untch, R. H. (2001) · Springer eBooks · doi:10.1007/978-1-4757-5939-6_7 · Cited by 374
Investigations of the software testing coupling effect
Offutt, A. J. (1992) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/125489.125473 · Cited by 371
Automatically detecting equivalent mutants and infeasible paths
Offutt, A. J., & Pan, J. (1997) · Software Testing Verification and Reliability · Cited by 296
Two notions of correctness and their relation to testing
Budd, T. A., & Angluin, D. (1982) · Acta Informatica · doi:10.1007/bf00625279 · Cited by 260
Improving test suites via operational abstraction
Harder, M. P. F., Mellen, J., & Ernst, M. D. (2003) · doi:10.5555/776816.776824 · Cited by 163
Detecting equivalent mutants and the feasible path problem
Offutt, A. J., & Pan, J. (2002) · Annual Conference on Computer Assurance, Systems Integrity, Software Safety and Process Security (COMPASS) · doi:10.1109/cmpass.1996.507890 · Cited by 126
General test result checking with log file analysis
Andrews, J. H., & Zhang, Y. (2003) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2003.1214327 · Cited by 92
Self-citationImproving test suites via operational abstraction
Harder, M. P. F., Mellen, J., & Ernst, M. D. (2003) · Proceedings/Proceedings - International Conference on Software Engineering · doi:10.1109/icse.2003.1201188 · Cited by 89
An experimental study on software structural testing: deterministic versus random input generation
Thévenod-Fosse, P., Waeselynck, H., & Crouzet, Y. (2002) · IEEE International Symposium on Fault-Tolerant Computing · doi:10.1109/ftcs.1991.146694 · Cited by 63
Cited by
An Analysis and Survey of the Development of Mutation Testing
Jia, Y., & Harman, M. (2011) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2010.62 · Cited by 1,745
Regression testing minimization, selection and prioritization: a survey
Yoo, S., & Harman, M. (2012) · Software Testing, Verification and Reliability · doi:10.1002/stv.430 · Cited by 1,322
A Survey on Software Fault Localization
Wong, W. E., Gao, R., Li, Y., Abreu, R., & Wotawa, F. (2016) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2016.2521368 · Cited by 1,110
A taxonomy of model‐based testing approaches
Utting, M., Pretschner, A., & Legeard, B. (2012) · Software Testing, Verification and Reliability · doi:10.1002/stvr.456 · Cited by 632
Roy, C. K., & Cordy, J. R. (2008) · 2008 16th IEEE International Conference on Program Comprehension · doi:10.1109/icpc.2008.41 · Cited by 533
Metadata
- Authors
- Andrews, J. H., Brand, L. C., & Labiche, Y.
- Venue
- Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005.
- Publisher
- IEEe
- Published
- 2005-12-22
- Pages
- 402-411
- ISSN
- 0270-5257 (ISSN-L)
- OpenAlex
- W1501076421
- Cited by
- 368
- References
- 18
- FWCI
- 8.60
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.