The DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index.
Search-Based Software Testing: Past, Present and Future
Abstract
Search-Based Software Testing is the use of a meta-heuristic optimizing search technique, such as a Genetic Algorithm, to automate or partially automate a testing task, for example the automatic generation of test data. Key to the optimization process is a problem-specific fitness function. The role of the fitness function is to guide the search to good solutions from a potentially infinite search space, within a practical time limit. Work on Search-Based Software Testing dates back to 1976, with interest in the area beginning to gather pace in the 1990s. More recently there has been an explosion of the amount of work. This paper reviews past work and the current state of the art, and discusses potential future research areas and open problems that remain in the field.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Optimization by Simulated Annealing
Kirkpatrick, S., Gelatt, C. D., & Vecchi, M. P. (1983) · Science · doi:10.1126/science.220.4598.671 · Cited by 45,134
Whitley, D. (1994) · Statistics and Computing · doi:10.1007/bf00175354 · Cited by 4,522
What every computer scientist should know about floating-point arithmetic
Goldberg, D. (1991) · ACM Computing Surveys · doi:10.1145/103162.103163 · Cited by 1,750
Search‐based software test data generation: a survey
McMinn, P. (2004) · Software Testing, Verification and Reliability · doi:10.1002/stvr.294 · Cited by 1,368
Self-citationSimplifying and isolating failure-inducing input
Zeller, A., & Hildebrandt, R. (2002) · IEEE Transactions on Software Engineering · doi:10.1109/32.988498 · Cited by 1,079
The N-Version Approach to Fault-Tolerant Software
Avizienis, A. (1985) · IEEE Transactions on Software Engineering · doi:10.1109/tse.1985.231893 · Cited by 1,049
Automated software test data generation
Korel, B. (1990) · IEEE Transactions on Software Engineering · doi:10.1109/32.57624 · Cited by 971
Search-based software engineering
Harman, M., & Jones, B. F. (2001) · Information and Software Technology · doi:10.1016/s0950-5849(01)00189-6 · Cited by 914
Search Algorithms for Regression Test Case Prioritization
Li, Z., Harman, M., & Hierons, R. M. (2007) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2007.38 · Cited by 738
The Current State and Future of Search Based Software Engineering
Harman, M. (2007) · Future of Software Engineering (FOSE '07) · doi:10.1109/fose.2007.29 · Cited by 631
Evolutionary test environment for automatic structural testing
Wegener, J., Baresel, A., & Sthamer, H. (2001) · Information and Software Technology · doi:10.1016/s0950-5849(01)00190-2 · Cited by 519
An overview of evolutionary algorithms: practical issues and common pitfalls
Whitley, D. (2001) · Information and Software Technology · doi:10.1016/s0950-5849(01)00188-4 · Cited by 448
Evolutionary testing of classes
Tonella, P. (2004) · Proceedings of the 2004 ACM SIGSOFT international symposium on Software testing and analysis · doi:10.1145/1007512.1007528 · Cited by 410
Pareto efficient multi-objective test case selection
Yoo, S., & Harman, M. (2007) · Proceedings of the 2007 international symposium on Software testing and analysis · doi:10.1145/1273463.1273483 · Cited by 357
TimeAware test suite prioritization
Walcott, K. R., Soffa, M. L., Kapfhammer, G. M., & Roos, R. S. (2006) · Proceedings of the 2006 international symposium on Software testing and analysis · doi:10.1145/1146238.1146240 · Cited by 340
Constructing test suites for interaction testing
Cohen, M. B., Gibbons, P. B., Mugridge, W. B., & Colbourn, C. J. (2003) · doi:10.5555/776816.776822 · Cited by 305
Harman, M., Hu, L., Hierons, R., Wegener, J., Sthamer, H., Baresel, A., & Roper, M. (2004) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2004.1265732 · Cited by 265
Automatic Generation of Floating-Point Test Data
Miller, W., & Spooner, D. L. (1976) · IEEE Transactions on Software Engineering · doi:10.1109/tse.1976.233818 · Cited by 252
An automated framework for structural test-data generation
Tracey, N., Clark, J., Mander, K., & McDermid, J. (2002) · Proceedings 13th IEEE International Conference on Automated Software Engineering (Cat. No.98EX239) · doi:10.1109/ase.1998.732680 · Cited by 205
Constructing Subtle Faults Using Higher Order Mutation Testing
Jia, Y., & Harman, M. (2008) · 2008 Eighth IEEE International Working Conference on Source Code Analysis and Manipulation · doi:10.1109/scam.2008.36 · Cited by 195
A multi-objective approach to search-based test data generation
Lakhotia, K., Harman, M., & McMinn, P. (2007) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/1276958.1277175 · Cited by 158
Self-citationEvolutionary Generation of Whole Test Suites
Fraser, G., & Arcuri, A. (2011) · Proceedings of the ... International Conference on Quality Software/Proceedings · doi:10.1109/qsic.2011.19 · Cited by 141
Evolutionary testing in the presence of loop-assigned flags: a testability transformation approach
Baresel, A., Binkley, D., Harman, M., & Korel, B. (2004) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/1007512.1007527 · Cited by 111
Efficient unit test case minimization
Leitner, A., Oriol, M., Zeller, A., et al. (2007) · doi:10.1145/1321631.1321698 · Cited by 108
Pseudo-oracles for non-testable programs
Davis, M. D., & Weyuker, E. J. (1981) · ACM Annual Conference · doi:10.1145/800175.809889 · Cited by 103
Dynamic method for software test data generation
Korel, B. (1992) · Software Testing Verification and Reliability · doi:10.1002/stvr.4370020405 · Cited by 87
Harman, M., Kim, S. G., Lakhotia, K., McMinn, P., & Yoo, S. (2010) · IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) · doi:10.1109/icstw.2010.31 · Cited by 85
Self-citationNormalizing Source Code Vocabulary
Lawrie, D., Binkley, D., & Morrell, C. H. (2010) · Working Conference on Reverse Engineering (WCRE) · doi:10.1109/wcre.2010.10 · Cited by 74
An empirical investigation into branch coverage for C programs using CUTE and AUSTIN
Lakhotia, K., McMinn, P., & Harman, M. (2010) · Journal of Systems and Software · doi:10.1016/j.jss.2010.07.026 · Cited by 69
Self-citationReducing qualitative human oracle costs associated with automatically generated test data
McMinn, P., Stevenson, M., & Harman, M. (2010) · doi:10.1145/1868048.1868049 · Cited by 64
Self-citationEmpirical evaluation of a nesting testability transformation for evolutionary testing
McMinn, P., Binkley, D., & Harman, M. (2009) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/1525880.1525884 · Cited by 58
Self-citationSearch-based failure discovery using testability transformations to generate pseudo-oracles
McMinn, P. (2009) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/1569901.1570127 · Cited by 46
Self-citationData Dependence Based Testability Transformation in Automated Test Generation
Korel, B., Harman, M., Chung, S., Apirukvorapinit, P., Gupta, R., & Zhang, Q. (2006) · Proceedings/Proceedings - International Symposium on Software Reliability Engineering · doi:10.1109/issre.2005.16 · Cited by 31
It is Not the Length That Matters, It is How You Control It
Fraser, G., & Arcuri, A. (2011) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst.2011.54 · Cited by 27
Cited by
The oracle problem in software testing: A survey
Barr, E. T., Harman, M., McMinn, P., Shahbaz, M., & Yoo, S. (2015) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2014.2372785 · Cited by 1,104
Self-citationAn orchestrated survey of methodologies for automated software test case generation
Anand, S., Burke, E. K., Chen, T. Y., Clark, J., Cohen, M. B., Grieskamp, W., Harman, M., Harrold, M. J., McMinn, P., Bertolino, A., et al. (2013) · Journal of Systems and Software · doi:10.1016/j.jss.2013.02.061 · Cited by 668
Self-citationFairFuzz: a targeted mutation strategy for increasing greybox fuzz testing coverage
Lemieux, C., & Sen, K. (2018) · Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering · doi:10.1145/3238147.3238176 · Cited by 477
An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generation
Schäfer, M., Nadi, S., Eghbali, A., & Tip, F. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2023.3334955 · Cited by 431
“…an alternative to (and potentially complements) traditional techniques for automated test generation, including feedback-directed random test generation [7], [8], [9], [10], [11], search-based and evolutionary techniques [16], [17], [63], [64], and dynamic symbolic execution [12], [13], [14], [15].”
BackgroundPanichella, A., Kifetew, F. M., & Tonella, P. (2018) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2017.2663435 · Cited by 352
Metadata
- Authors
- McMinn, P.
- Venue
- 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops
- Publisher
- IEEE
- Published
- 2011-03
- Pages
- 153-163
- OpenAlex
- W2107729695
- Cited by
- 322
- References
- 34
- FWCI
- 11.62
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.