The DOI matches its Crossref record; 56 of the 56 references it cites were found in a registry or an index.
Regression testing minimization, selection and prioritization: a survey
Abstract
Regression testing is a testing activity that is performed to provide confidence that changes do not harm the existing behaviour of the software. Test suites tend to grow in size as software evolves, often making it too costly to execute entire test suites. A number of different approaches have been studied to maximize the value of the accrued test suite: minimization, selection and prioritization. Test suite minimization seeks to eliminate redundant test cases in order to reduce the number of tests to run. Test case selection seeks to identify the test cases that are relevant to some set of recent changes. Test case prioritization seeks to order test cases in such a way that early fault detection is maximized. This paper surveys each area of minimization, selection and prioritization technique and discusses open problems and potential directions for future research. Copyright (C) 2010 John Wiley & Sons, Ltd.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(1979) · Cited by 43,841
Proceedings of the 25th international conference on Machine learning - ICML '08
(2008) · doi:10.1145/1390156 · Cited by 6,680
A Short Introduction to Boosting
Freund, Y., & Schapire, R. E. (1999) · Cited by 2,984
Search‐based software test data generation: a survey
McMinn, P. (2004) · Software Testing, Verification and Reliability · doi:10.1002/stvr.294 · Cited by 1,368
Prioritizing test cases for regression testing
Rothermel, G., Untch, R. H., Chu, C., & Harrold, M. J. (2001) · IEEE Transactions on Software Engineering · doi:10.1109/32.962562 · Cited by 1,338
Do, H., Elbaum, S., & Rothermel, G. (2005) · Empirical Software Engineering · doi:10.1007/s10664-005-3861-2 · Cited by 1,146
Dynamically discovering likely program invariants to support program evolution
Ernst, M. D., Cockrell, J., Griswold, W. G., & Notkin, D. (2001) · IEEE Transactions on Software Engineering · doi:10.1109/32.908957 · Cited by 1,050
Test case prioritization: a family of empirical studies
Elbaum, S., Malishevsky, A. G., & Rothermel, G. (2002) · IEEE Transactions on Software Engineering · doi:10.1109/32.988497 · Cited by 943
An Efficient Boosting Algorithm for Combining Preferences
Freund, Y., Iyer, R. D., Schapire, R. E., & Singer, Y. (1998) · Cited by 806
Search Algorithms for Regression Test Case Prioritization
Li, Z., Harman, M., & Hierons, R. M. (2007) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2007.38 · Cited by 738
Self-citationA safe, efficient regression test selection technique
Rothermel, G., & Harrold, M. J. (1997) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/248233.248262 · Cited by 712
Analyzing regression test selection techniques
Rothermel, G., & Harrold, M. J. (1996) · IEEE Transactions on Software Engineering · doi:10.1109/32.536955 · Cited by 708
A methodology for controlling the size of a test suite
Harrold, M. J., Gupta, R., & Soffa, M. L. (1993) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/152388.152391 · Cited by 657
Test case prioritization: an empirical study
Rothermel, G., Untch, R. H., Chu, C., & Harrold, M. J. (1999) · Proceedings IEEE International Conference on Software Maintenance - 1999 (ICSM'99). 'Software Maintenance for Business Change' (Cat. No.99CB36360) · doi:10.1109/icsm.1999.792604 · Cited by 625
An evaluation of methods for prioritizing software requirements
Karlsson, J., Wohlin, C., & Regnell, B. (1998) · Information and Software Technology · doi:10.1016/s0950-5849(97)00053-0 · Cited by 433
A study of effective regression testing in practice
Wong, W. E., Horgan, J. R., London, S., & Agrawal, H. (2002) · Proceedings The Eighth International Symposium on Software Reliability Engineering · doi:10.1109/issre.1997.630875 · Cited by 390
Experiments on the effectiveness of dataflow- and control-flow-based test adequacy criteria
Hutchins, M., Foster, H., Goradia, T., & Ostrand, T. (2002) · Proceedings of 16th International Conference on Software Engineering · doi:10.1109/icse.1994.296778 · Cited by 375
Is mutation an appropriate tool for testing experiments?
Andrews, J. H., Brand, L. C., & Labiche, Y. (2005) · Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005. · doi:10.1109/icse.2005.1553583 · Cited by 368
Yesterday, my program worked. Today, it does not. Why?
Zeller, A. (1999) · ACM SIGSOFT Software Engineering Notes · doi:10.1145/318774.318946 · Cited by 360
Pareto efficient multi-objective test case selection
Yoo, S., & Harman, M. (2007) · Proceedings of the 2007 international symposium on Software testing and analysis · doi:10.1145/1273463.1273483 · Cited by 357
Self-citation(1998) · Choice Reviews Online · doi:10.5860/choice.35-5705 · Cited by 343
TimeAware test suite prioritization
Walcott, K. R., Soffa, M. L., Kapfhammer, G. M., & Roos, R. S. (2006) · Proceedings of the 2006 international symposium on Software testing and analysis · doi:10.1145/1146238.1146240 · Cited by 340
An empirical study of the effects of minimization on the fault detection capabilities of test suites
Rothermel, G., Harrold, M. J., Ostrin, J., & Hong, C. (2002) · International Conference on Software Maintenance · doi:10.1109/icsm.1998.738487 · Cited by 303
On the Use of Mutation Faults in Empirical Assessments of Test Case Prioritization Techniques
Do, H., & Rothermel, G. (2006) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2006.92 · Cited by 287
TestTube: a system for selective regression testing
Chen, Y.-F. R., Rosenblum, D. S., & Vo, K.-P. (1994) · doi:10.5555/257734.257769 · Cited by 262
Empirical studies of test‐suite reduction
Rothermel, G., Harrold, M. J., von Ronne, J., & Hong, C. (2002) · Software Testing, Verification and Reliability · doi:10.1002/stvr.256 · Cited by 253
Selecting a Cost-Effective Test Case Prioritization Technique
Elbaum, S. G., Rothermel, G., Kanduri, S., & Malishevsky, A. G. (2004) · Software Quality Journal · doi:10.1023/b:sqjo.0000034708.84524.22 · Cited by 246
Mutation analysis of program test data
Budd, T. A. (1980) · Cited by 236
An empirical study of the effects of test-suite reduction on fault localization
Yu, Y., Jones, J. A., & Harrold, M. J. (2008) · Proceedings of the 13th international conference on Software engineering - ICSE '08 · doi:10.1145/1368088.1368116 · Cited by 236
Incremental program testing using program dependence graphs
Bates, S., & Horwitz, S. (1993) · Proceedings of the 20th ACM SIGPLAN-SIGACT symposium on Principles of programming languages - POPL '93 · doi:10.1145/158511.158694 · Cited by 222
A study of integration testing and software regression at the integration level
Leung, H. K. N., & White, L. (2002) · Proceedings. Conference on Software Maintenance 1990 · doi:10.1109/icsm.1990.131377 · Cited by 195
Constructing Subtle Faults Using Higher Order Mutation Testing
Jia, Y., & Harman, M. (2008) · 2008 Eighth IEEE International Working Conference on Source Code Analysis and Manipulation · doi:10.1109/scam.2008.36 · Cited by 195
Self-citationImproving test suites via operational abstraction
Harder, M. P. F., Mellen, J., & Ernst, M. D. (2003) · doi:10.5555/776816.776824 · Cited by 163
Test-Suite Augmentation for Evolving Software
Santelices, R., Chittimalli, P. K., Apiwattanapong, T., Orso, A., & Harrold, M. J. (2008) · 2008 23rd IEEE/ACM International Conference on Automated Software Engineering · doi:10.1109/ase.2008.32 · Cited by 160
Automatically repairing event sequence-based GUI test suites for regression testing
Memon, A. M. (2008) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/1416563.1416564 · Cited by 148
Test Case Prioritization Using Relevant Slices
Jeffrey, D., & Gupta, N. (2006) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac.2006.80 · Cited by 122
Cleve, A., Ricca, F., & Cerioli, M. (2013) · Proceedings of the ... European Conference on Software Maintenance and Reengineering/Proceedings of the European Conference on Software Maintenance and Reengineering · doi:10.1109/csmr.2013.1 · Cited by 119
On test suite composition and cost-effective regression testing
Rothermel, G., Elbaum, S. G., Malishevsky, A. G., et al. (2004) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/1027092.1027093 · Cited by 115
MINTS: A general framework and tool for supporting test-suite minimization
Hsu, H.-Y., & Orso, A. (2009) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2009.5070541 · Cited by 115
Efficient unit test case minimization
Leitner, A., Oriol, M., Zeller, A., et al. (2007) · doi:10.1145/1321631.1321698 · Cited by 108
Empirical evaluations of regression test selection techniques: a systematic review
Engström, E., Skoglund, M., & Runeson, P. (2008) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1145/1414004.1414011 · Cited by 104
Techniques for selective revalidation
Hartmann, J., & Robson, D. J. (1990) · IEEE Software · doi:10.1109/52.43047 · Cited by 103
Proceedings International Conference On Software Maintenance ICSM 2002
(2002) · International Conference on Software Maintenance · doi:10.1109/icsm.2002.1167741 · Cited by 94
Using the Case-Based Ranking Methodology for Test Case Prioritization
Tonella, P., Avesani, P., & Susi, A. (2006) · Proceedings/Proceedings - Conference on Software Maintenance · doi:10.1109/icsm.2006.74 · Cited by 91
Krishnamoorthi, R., & Mary, S. A. S. A. (2008) · Information and Software Technology · doi:10.1016/j.infsof.2008.08.007 · Cited by 90
Improving test suites via operational abstraction
Harder, M. P. F., Mellen, J., & Ernst, M. D. (2003) · Proceedings/Proceedings - International Conference on Software Engineering · doi:10.1109/icse.2003.1201188 · Cited by 89
Understanding the effects of changes on the cost‐effectiveness of regression testing techniques
Elbaum, S. G., Kallakuri, P., Malishevsky, A. G., Rothermel, G., & Kanduri, S. (2003) · Software Testing Verification and Reliability · doi:10.1002/stvr.263 · Cited by 88
An approach to software fault localization and revalidation based on incremental data flow analysis
Taha, A.-B., Thebaut, S. M., & Liu, S.-S. (2003) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/cmpsac.1989.65142 · Cited by 73
The impact of test suite granularity on the cost-effectiveness of regression testing
Rothermel, G., Elbaum, S. G., Malishevsky, A. G., et al. (2002) · Proceedings/Proceedings - International Conference on Software Engineering · doi:10.1145/581339.581358 · Cited by 70
Applying regression test selection for COTS-based applications
Zheng, J., Robinson, B., Williams, L. A., & Smiley, K. J. (2006) · Proceedings/Proceedings - International Conference on Software Engineering · doi:10.1145/1134285.1134357 · Cited by 59
Automated Session Data Repair for Web Application Regression Testing
Harman, M., & Alshahwan, N. (2008) · International Conference on Software Testing, Verification, and Validation · doi:10.1109/icst.2008.56 · Cited by 55
Self-citationSoftware architecture-based regression testing
Muccini, H., Dias, M., & Richardson, D. J. (2006) · Journal of Systems and Software · doi:10.1016/j.jss.2006.02.059 · Cited by 39
Fault Detection Probability Analysis for Coverage-Based Test Suite Reduction
McMaster, S., & Memon, A. M. (2007) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsm.2007.4362646 · Cited by 30
Test reuse in the spreadsheet paradigm
Ii, M. F., Jin, D., Rothermel, G., & Burnett, M. (2003) · Proceedings/Proceedings - International Symposium on Software Reliability Engineering · doi:10.1109/issre.2002.1173265 · Cited by 25
Cited by
The oracle problem in software testing: A survey
Barr, E. T., Harman, M., McMinn, P., Shahbaz, M., & Yoo, S. (2015) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2014.2372785 · Cited by 1,104
Self-citationSearch-based software engineering: Trends, techniques and applications
Harman, M., Mansouri, S. A., & Zhang, Y. (2012) · ACM Computing Surveys · doi:10.1145/2379776.2379787 · Cited by 777
Self-citationA Survey on Metamorphic Testing
Segura, S., Fraser, G., Sanchez, A. B., & Ruiz-Cortes, A. (2016) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2016.2532875 · Cited by 563
An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generation
Schäfer, M., Nadi, S., Eghbali, A., & Tip, F. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2023.3334955 · Cited by 431
Guiding Deep Learning System Testing Using Surprise Adequacy
Kim, J., Feldt, R., & Yoo, S. (2019) · 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2019.00108 · Cited by 430
Self-citation
Metadata
- Authors
- Yoo, S., & Harman, M.
- Venue
- Software Testing, Verification and Reliability
- Publisher
- Wiley
- Published
- 2012-02-23
- Volume
- 22
- Issue
- 2
- Pages
- 67-120
- ISSN
- 0960-0833 (ISSN-L)
- OpenAlex
- W2014515160
- Cited by
- 1,322
- References
- 56
- FWCI
- 351.07
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.