The DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.
Techniques for improving regression testing in continuous integration development environments
Abstract
In continuous integration development environments, software engineers frequently integrate new or changed code with the mainline codebase. This can reduce the amount of code rework that is needed as systems evolve and speed up development time. While continuous integration processes traditionally require that extensive testing be performed following the actual submission of code to the codebase, it is also important to ensure that enough testing is performed prior to code submission to avoid breaking builds and delaying the fast feedback that makes continuous integration desirable. In this work, we present algorithms that make continuous integration processes more cost-effective. In an initial pre-submit phase of testing, developers specify modules to be tested, and we use regression test selection techniques to select a subset of the test suites for those modules that render that phase more cost-effective. In a subsequent post-submit phase of testing, where dependent modules as well as changed modules are tested, we use test case prioritization techniques to ensure that failures are reported more quickly. In both cases, the techniques we utilize are novel, involving algorithms that are relatively inexpensive and do not rely on code coverage information -- two requirements for conducting testing cost-effectively in this context. To evaluate our approach, we conducted an empirical study on a large data set from Google that we make publicly available. The results of our study show that our selection and prioritization techniques can each lead to cost-effectiveness improvements in the continuous integration process.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Prioritizing test cases for regression testing
Rothermel, G., Untch, R. H., Chu, C., & Harrold, M. J. (2001) · IEEE Transactions on Software Engineering · doi:10.1109/32.962562 · Cited by 1,338
Self-citationRegression testing minimization, selection and prioritization: a survey
Yoo, S., & Harman, M. (2012) · Software Testing, Verification and Reliability · doi:10.1002/stv.430 · Cited by 1,322
Test case prioritization: a family of empirical studies
Elbaum, S., Malishevsky, A. G., & Rothermel, G. (2002) · IEEE Transactions on Software Engineering · doi:10.1109/32.988497 · Cited by 943
Self-citationA safe, efficient regression test selection technique
Rothermel, G., & Harrold, M. J. (1997) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/248233.248262 · Cited by 712
Self-citationAnalyzing regression test selection techniques
Rothermel, G., & Harrold, M. J. (1996) · IEEE Transactions on Software Engineering · doi:10.1109/32.536955 · Cited by 708
Self-citationContinuous Integration: Improving Software Quality and Reducing Risk
Duvall, P., Matyas, S., & Glover, A. (2007) · Cited by 510
TimeAware test suite prioritization
Walcott, K. R., Soffa, M. L., Kapfhammer, G. M., & Roos, R. S. (2006) · Proceedings of the 2006 international symposium on Software testing and analysis · doi:10.1145/1146238.1146240 · Cited by 340
On the Use of Mutation Faults in Empirical Assessments of Test Case Prioritization Techniques
Do, H., & Rothermel, G. (2006) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2006.92 · Cited by 287
Self-citationParallel symbolic execution for automated real-world software testing
Bucur, S., Ureche, V., Zamfir, C., & Candea, G. (2011) · Proceedings of the sixth conference on Computer systems · doi:10.1145/1966445.1966463 · Cited by 236
The Effects of Time Constraints on Test Case Prioritization: A Series of Controlled Experiments
Do, H., Mirarab, S., Tahvildari, L., & Rothermel, G. (2010) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2010.58 · Cited by 164
Self-citationTest Case Prioritization for Continuous Regression Testing: An Industrial Case Study
Marijan, D., Gotlieb, A., & Sen, S. (2013) · Proceedings/Proceedings - Conference on Software Maintenance · doi:10.1109/icsm.2013.91 · Cited by 155
Duvall, P., Matyas, S. M., & Glover, A. (2007) · Addison-Wesley Professional eBooks · Cited by 151
An experimental evaluation of continuous testing during development
Saff, D., & Ernst, M. D. (2004) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/1007512.1007523 · Cited by 140
Regression Testing Minimisation, Selection and Prioritisation - A Survey
Yoo, S., & Harman, M. (2009) · UCL Discovery (University College London) · Cited by 134
Whittaker, J. A. (2010) · doi:10.1145/1852666.1852675 · Cited by 90
How Well Do Test Case Prioritization Techniques Support Statistical Fault Localization
Jiang, B., Zhang, Z., Tse, T. H., & Chen, T. Y. (2009) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac.2009.23 · Cited by 70
A Scalable Distributed Concolic Testing Approach: An Empirical Evaluation
Kim, M., Kim, Y., & Rothermel, G. (2012) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst.2012.114 · Cited by 46
Self-citationOracle-Centric Test Case Prioritization
Staats, M., Loyola, P., & Rothermel, G. (2012) · Proceedings/Proceedings - International Symposium on Software Reliability Engineering · doi:10.1109/issre.2012.13 · Cited by 29
Self-citation
Cited by
Usage, costs, and benefits of continuous integration in open-source projects
Hilton, M., Tunnell, T., Huang, K., Marinov, D., & Dig, D. (2016) · Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering · doi:10.1145/2970276.2970358 · Cited by 342
Spieker, H., Gotlieb, A., Marijan, D., & Mossige, M. (2017) · Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis · doi:10.1145/3092703.3092709 · Cited by 283
Ali, S., Hafeez, Y., Hussain, S., & Yang, S. (2020) · Software Quality Journal · doi:10.1007/s11219-019-09463-4 · Cited by 43
Understanding Similarities and Differences in Software Development Practices Across Domains
Viggiato, M., Oliveira, J., Figueiredo, E., Jamshidi, P., & Kastner, C. (2019) · 2019 ACM/IEEE 14th International Conference on Global Software Engineering (ICGSE) · doi:10.1109/icgse.2019.00013 · Cited by 15
Metadata
- Authors
- Elbaum, S., Rothermel, G., & Penix, J.
- Venue
- Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering
- Publisher
- ACM
- Published
- 2014-11-11
- Pages
- 235-245
- OpenAlex
- W1991292921
- Cited by
- 327
- References
- 18
- FWCI
- 13.27
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.