The DOI matches its Crossref record.
Experimentation in software engineering
Abstract
Like other sciences and engineering disciplines, software engineering requires a cycle of model building, experimentation, and learning. Experiments are valuable tools for all software engineers who are involved in evaluating and choosing between different methods, techniques, languages and tools. The purpose of Experimentation in Software Engineering is to introduce students, teachers, researchers, and practitioners to empirical studies in software engineering, using controlled experiments. The introduction to experimentation is provided through a process perspective, and the focus is on the steps that we have to go through to perform an experiment. The book is divided into three parts. The first part provides a background of theories and methods used in experimentation. Part II then devotes one chapter to each of the five experiment steps: scoping, planning, execution, analysis, and result presentation. Part III completes the presentation with two examples. Assignments and statistical material are provided in appendixes. Overall the book provides indispensable information regarding empirical studies in particular for experiments, but also for case studies, systematic literature reviews, and surveys. It is a revision of the authors book, which was published in 2000. In addition, substantial new material, e.g. concerning systematic literature reviews and case study research, is introduced. The book is self-contained and it is suitable as a course book in undergraduate or graduate studies where the need for empirical studies in software engineering is stressed. Exercises and assignments are included to combine the more theoretical material with practical aspects. Researchers will also benefit from the book, learning more about how to conduct empirical studies, and likewise practitioners may use it as a cookbook when evaluating new methods or techniques before implementing them in their organization.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cited by
Guidelines for conducting systematic mapping studies in software engineering: An update
Petersen, K., Vakkalanka, S., & Kuzniarz, L. (2015) · Information and Software Technology · doi:10.1016/j.infsof.2015.03.007 · Cited by 2,669
Garousi, V., Felderer, M., & Mäntylä, M. V. (2019) · Information and Software Technology · doi:10.1016/j.infsof.2018.09.006 · Cited by 831
Collins, C., Dennehy, D., Conboy, K., & Mikalef, P. (2021) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2021.102383 · Cited by 783
The ABC of Software Engineering Research
Stol, K.-J., & Fitzgerald, B. (2018) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3241743 · Cited by 245
Quality, productivity and economic benefits of software reuse: a review of industrial studies
Mohagheghi, P., & Conradi, R. (2007) · Empirical Software Engineering · doi:10.1007/s10664-007-9040-x · Cited by 197
Mourão, E., Pimentel, J. F., Murta, L., Kalinowski, M., Mendes, E., & Wohlin, C. (2020) · Information and Software Technology · doi:10.1016/j.infsof.2020.106294 · Cited by 191
Cico, O., Jaccheri, L., Nguyen-Duc, A., & Zhang, H. (2021) · Journal of Systems and Software · doi:10.1016/j.jss.2020.110736 · Cited by 195
In-IDE Code Generation from Natural Language: Promise and Challenges
Xu, F. F., Vasilescu, B., & Neubig, G. (2022) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3487569 · Cited by 154
How Programmers Debug, Revisited: An Information Foraging Theory Perspective
Lawrance, J., Bogart, C., Burnett, M., Bellamy, R., Rector, K., & Fleming, S. D. (2013) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2010.111 · Cited by 142
Thirteen years of SysML: a systematic mapping study
Wolny, S., Mazak, A., Carpella, C., Geist, V., & Wimmer, M. (2020) · Software and Systems Modeling · doi:10.1007/s10270-019-00735-y · Cited by 140
Steidl, M., Felderer, M., & Ramler, R. (2023) · Journal of Systems and Software · doi:10.1016/j.jss.2023.111615 · Cited by 126
Accessibility in Software Practice: A Practitioner’s Perspective
Bi, T., Xia, X., Lo, D., Grundy, J., Zimmermann, T., & Ford, D. (2022) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3503508 · Cited by 109
Knowledge-based approaches in software documentation: A systematic literature review
Ding, W., Liang, P., Tang, A., & van Vliet, H. (2014) · Information and Software Technology · doi:10.1016/j.infsof.2014.01.008 · Cited by 107
A Survey of Performance Optimization for Mobile Applications
Hort, M., Kechagia, M., Sarro, F., & Harman, M. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2021.3071193 · Cited by 91
Duboc, L., Penzenstadler, B., Porras, J., Akinli Kocak, S., Betz, S., Chitchyan, R., Leifler, O., Seyff, N., & Venters, C. C. (2020) · Requirements Engineering · doi:10.1007/s00766-020-00336-y · Cited by 88
Model-based testing for software safety: a systematic mapping study
Gurbuz, H. G., & Tekinerdogan, B. (2018) · Software Quality Journal · doi:10.1007/s11219-017-9386-2 · Cited by 78
SYMBALS: A Systematic Review Methodology Blending Active Learning and Snowballing
van Haastrecht, M., Sarhan, I., Yigit Ozkan, B., Brinkhuis, M., & Spruit, M. (2021) · Frontiers in Research Metrics and Analytics · doi:10.3389/frma.2021.685591 · Cited by 80
Gamified Requirements Engineering: Model and Experimentation
Lombriser, P., Dalpiaz, F., Lucassen, G., & Brinkkemper, S. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-30282-9_12 · Cited by 73
Duboc, L., Betz, S., Penzenstadler, B., Akinli Kocak, S., Chitchyan, R., Leifler, O., Porras, J., Seyff, N., & Venters, C. C. (2019) · 2019 IEEE 27th International Requirements Engineering Conference (RE) · doi:10.1109/re.2019.00013 · Cited by 69
Tosun, A., Dieste, O., Fucci, D., Vegas, S., Turhan, B., Erdogmus, H., Santos, A., Oivo, M., Toro, K., Jarvinen, J., et al. (2017) · Empirical Software Engineering · doi:10.1007/s10664-016-9490-0 · Cited by 50
Examining decision characteristics & challenges for agile software development
Drury-Grogan, M. L., Conboy, K., & Acton, T. (2017) · Journal of Systems and Software · doi:10.1016/j.jss.2017.06.003 · Cited by 45
A mapping study on documentation in Continuous Software Development
Theunissen, T., van Heesch, U., & Avgeriou, P. (2022) · Information and Software Technology · doi:10.1016/j.infsof.2021.106733 · Cited by 40
Older adults and hackathons: a qualitative study
Kopeć, W., Balcerzak, B., Nielek, R., Kowalik, G., Wierzbicki, A., & Casati, F. (2018) · Empirical Software Engineering · doi:10.1007/s10664-017-9565-6 · Cited by 39
Wyrich, M., Preikschat, A., Graziotin, D., & Wagner, S. (2021) · 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE) · doi:10.1109/icse43902.2021.00055 · Cited by 24
Studying test-driven development and its retainment over a six-month time span
Baldassarre, M. T., Caivano, D., Fucci, D., Juristo, N., Romano, S., Scanniello, G., & Turhan, B. (2021) · Journal of Systems and Software · doi:10.1016/j.jss.2021.110937 · Cited by 15
Understanding Similarities and Differences in Software Development Practices Across Domains
Viggiato, M., Oliveira, J., Figueiredo, E., Jamshidi, P., & Kastner, C. (2019) · 2019 ACM/IEEE 14th International Conference on Global Software Engineering (ICGSE) · doi:10.1109/icgse.2019.00013 · Cited by 15
Kassab, M., DeFranco, J., & Graciano Neto, V. (2018) · 2018 IEEE International Professional Communication Conference (ProComm) · doi:10.1109/procomm.2018.00033 · Cited by 15
Test Automation Process Improvement in a DevOps Team: Experience Report
Wang, Y., Pyhajarvi, M., & Mantyla, M. V. (2020) · 2020 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) · doi:10.1109/icstw50294.2020.00057 · Cited by 12
Anvari, F., & Richards, D. (2015) · Proceedings of the 10th International Conference on Evaluation of Novel Approaches to Software Engineering · doi:10.5220/0005366600900101 · Cited by 9
A tale of three systems: Case studies on the application of architectural tactics for cyber-foraging
Lewis, G., Lago, P., Echeverría, S., & Simoens, P. (2019) · Future Generation Computer Systems · doi:10.1016/j.future.2019.01.052 · Cited by 4
Metadata
- Venue
- Choice Reviews Online
- Publisher
- American Library Association
- Published
- 2013-04-17
- Volume
- 50
- Issue
- 09
- Pages
- 50-5018-50-5018
- ISSN
- 0009-4978 (ISSN-L)
- 1523-8253
- OpenAlex
- W1842503650
- Cited by
- 1,517
- References
- 0
- Type
- book review
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
