The DOI matches its Crossref record; 45 of the 45 references it cites were found in a registry or an index.
Using mapping studies as the basis for further research – A participant-observer case study
Open accessAbstract
Context: We are strong advocates of evidence-based software engineering (EBSE) in general and systematic literature reviews (SLRs) in particular. We believe it is essential that the SLR methodology is used constructively to support software engineering research. Objective: This study aims to assess the value of mapping studies which are a form of SLR that aims to identify and categorise the available research on a broad software engineering topic. Method: We used a multi-case, participant-observer case study using five examples of studies that were based on preceding mapping studies. We also validated our results by contacting two other researchers who had undertaken studies based on preceding mapping studies and by assessing review comments related to our follow-on studies. Results: Our original case study identified 11 unique benefits that can accrue from basing research on a preceding mapping study of which only two were case specific. We also identified nine problems associated with using preceding mapping studies of which two were case specific. These results were consistent with the information obtained from the validation activities. We did not find an example of an independent research group making use of a mapping study produced by other researchers. Conclusion: Mapping studies can save time and effort for researchers and provide baselines to assist new research efforts. However, they must be of high quality in terms of completeness and rigour if they are to be a reliable basis for follow-on research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(1984) · Cited by 81,983
Case study research: Design and methods
McCorcle, M. D., & Bell, E. L. (1986) · Evaluation and Program Planning · doi:10.1016/0149-7189(86)90052-2 · Cited by 5,885
Systematic Literature Reviews in Software Engineering—A Systematic Literature Review
Kitchenham, B., Pearl Brereton, O., Budgen, D., Turner, M., Bailey, J., & Linkman, S. (2009) · Information and Software Technology · doi:10.1016/j.infsof.2008.09.009 · Cited by 4,537
Self-citationSystematic reviews in the social sciences: a practical guide
(2006) · Choice Reviews Online · doi:10.5860/choice.43-5664 · Cited by 4,089
Systematic Mapping Studies in Software Engineering
Petersen, K., Feldt, R., Mujtaba, S., & Mattsson, M. (2008) · Electronic Workshops in Computing · doi:10.14236/ewic/ease2008.8 · Cited by 3,239 · cited 3×
“Three recent papers have discussed the use of mapping studies in software engineering [9,13,31].”
BackgroundMethodInfluentialSystematic literature reviews in software engineering – A tertiary study
Kitchenham, B., Pretorius, R., Budgen, D., Pearl Brereton, O., Turner, M., Niazi, M., & Linkman, S. (2010) · Information and Software Technology · doi:10.1016/j.infsof.2010.03.006 · Cited by 1,230 · cited 3×
“In our experience , there are a relatively large number of mapping studies [10], but relatively little discussion of their value as a research tool (see Section 3).”
BackgroundResultInfluentialSelf-citationCase Study Research: Design and Methods, 3rd Edition
Yin, R. K. (2002) · Cited by 1,005
Requirements engineering paper classification and evaluation criteria: a proposal and a discussion
Wieringa, R., Maiden, N., Mead, N., & Rolland, C. (2006) · Requirements Engineering · doi:10.1007/s00766-005-0021-6 · Cited by 940
A systematic review of software development cost estimation studies
Jørgensen, M., & Shepperd, M. (2007) · Brunel University Research Archive (BURA) (Brunel University London) · doi:10.1109/tse.2007.3 · Cited by 742
A Systematic Review of Software Development Cost Estimation Studies
Jorgensen, M., & Shepperd, M. (2007) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2007.256943 · Cited by 729
A survey of controlled experiments in software engineering
Sjoeberg, D. I. K., Hannay, J. E., Hansen, O., Kampenes, V. B., Karahasanovic, A., Liborg, N.-K., & Rekdal, A. C. (2005) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2005.97 · Cited by 682
(2004) · doi:10.5555/998675.999432 · Cited by 656
Self-citationA review of studies on expert estimation of software development effort
Jørgensen, M. (2004) · Journal of Systems and Software · doi:10.1016/s0164-1212(02)00156-5 · Cited by 618
A systematic review of software fault prediction studies
Catal, C., & Diri, B. (2009) · Expert Systems with Applications · doi:10.1016/j.eswa.2008.10.027 · Cited by 530
Evidence-based software engineering for practitioners
Dyba, T., Kitchenham, B. A., & Jorgensen, M. (2005) · IEEE Software · doi:10.1109/ms.2005.6 · Cited by 463
Self-citationA systematic review of search-based testing for non-functional system properties
Afzal, W., Torkar, R., & Feldt, R. (2009) · Information and Software Technology · doi:10.1016/j.infsof.2008.12.005 · Cited by 447
Cross versus Within-Company Cost Estimation Studies: A Systematic Review
Kitchenham, B. A., Mendes, E., & Travassos, G. H. (2007) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2007.1001 · Cited by 424
Self-citationA systematic review of effect size in software engineering experiments
Kampenes, V. B., Dybå, T., Hannay, J. E., & Sjøberg, D. I. K. (2007) · Information and Software Technology · doi:10.1016/j.infsof.2007.02.015 · Cited by 370
Using Mapping Studies in Software Engineering.
Budgen, D., Turner, M., Pearl Brereton, A., & Kitchenham, B. A. (2008) · Cited by 290
Self-citationA systematic review of statistical power in software engineering experiments
Dybå, T., Kampenes, V. B., & Sjøberg, D. I. K. (2006) · Information and Software Technology · doi:10.1016/j.infsof.2005.08.009 · Cited by 289
What’s up with software metrics? – A preliminary mapping study
Kitchenham, B. (2010) · Journal of Systems and Software · doi:10.1016/j.jss.2009.06.041 · Cited by 253
Self-citationA Systematic Review of Theory Use in Software Engineering Experiments
Hannay, J. E., Sjoberg, D. I. K., & Dyba, T. (2007) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2007.12 · Cited by 226
Checklists for Software Engineering Case Study Research
Höst, M., & Runeson, P. (2007) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem.2007.46 · Cited by 157
The value of mapping studies – A participant-observer case study
Kitchenham, B. A., Budgen, D., & Brereton, O. P. (2010) · Electronic workshops in computing · doi:10.14236/ewic/ease2010.4 · Cited by 152
Self-citationSystematic Reviews to Support Evidence-Based Medicine: How to appraise, conduct and publish reviews
Khan, K. S., & Zamora, J. (2022) · doi:10.1201/9781003220039 · Cited by 151
Empirical evidence about the UML: a systematic literature review
Budgen, D., Burn, A. J., Brereton, O. P., Kitchenham, B. A., & Pretorius, R. (2011) · Software: Practice and Experience · doi:10.1002/spe.1009 · Cited by 138
Self-citationWhat Do We Know about the Effectiveness of Software Design Patterns?
Zhang, C., & Budgen, D. (2011) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2011.79 · Cited by 137
Self-citationEvidence relating to object-oriented software design: A survey
Bailey, J. W., Budgen, D., Turner, M., Kitchenham, B. A., Pearl Brereton, A., & Linkman, S. (2007) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem.2007.58 · Cited by 133
Self-citationRefining the systematic literature review process—two participant-observer case studies
Kitchenham, B. A., Pearl Brereton, A., Turner, M., Niazi, M., Linkman, S., Pretorius, R., & Budgen, D. (2010) · Empirical Software Engineering · doi:10.1007/s10664-010-9134-8 · Cited by 120
Self-citationA systematic review of quasi-experiments in software engineering
Kampenes, V. B., Dybå, T., Hannay, J. E., & Sjøberg, D. I. K. (2008) · Information and Software Technology · doi:10.1016/j.infsof.2008.04.006 · Cited by 119
Reflections on 10 Years of Software Process Simulation Modeling: A Systematic Review
Zhang, H., Kitchenham, B. A., & Pfahl, D. (2008) · Springer eBooks · doi:10.1007/978-3-540-79588-9_30 · Cited by 79
Self-citationKitchenham, B. A., Pearl Brereton, A., Turner, M., Niazi, M., Linkman, S., Pretorius, R., & Budgen, D. (2009) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem.2009.5314238 · Cited by 64
Self-citationSoftware Process Simulation Modeling: An Extended Systematic Review
Zhang, H., Kitchenham, B. A., & Pfahl, D. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-14347-2_27 · Cited by 58
Self-citationImproving Evidence about Software Technologies: A Look at Model-Based Testing
Neto, A. D., Subramanyan, R., Vieira, M., Travassos, G. H., & Shull, F. J. (2008) · IEEE Software · doi:10.1109/ms.2008.64 · Cited by 57
Evaluating logistic regression models to estimate software project outcomes
Cerpa, N., Bardeen, M., Kitchenham, B. A., & Verner, J. M. (2010) · Information and Software Technology · doi:10.1016/j.infsof.2010.03.011 · Cited by 55
Self-citationTeaching evidence-based software engineering to university students
Jørgensen, M., Dyba, T., & Kitchenham, B. A. (2005) · IEEE International Software Metrics Symposium (METRICS) · doi:10.1109/metrics.2005.46 · Cited by 54
Self-citationThe Role of Deliberate Artificial Design Elements in Software Engineering Experiments
Hannay, J. E., & Jørgensen, M. (2008) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2008.13 · Cited by 53
The educational value of mapping studies of software engineering literature
Kitchenham, B. A., Pearl Brereton, A., & Budgen, D. (2010) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/1806799.1806887 · Cited by 52
Self-citationA mapping study on empirical evidence related to the models and forms used in the uml
Pretorius, R., & Budgen, D. (2008) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1145/1414004.1414076 · Cited by 48
Self-citationSystematic Reviews to Support Evidence-based Medicine
Gray, H. (2003) · Physiotherapy · doi:10.1016/s0031-9406(05)60506-8 · Cited by 46
A systematic mapping study on non-functional search-based software testing
Afzal, W., Torkar, R., & FELDT, R. H. (2008) · Cited by 38
Software Process Simulation Modeling: Facts, Trends and Directions
Zhang, H., Kitchenham, B. A., & Pfahl, D. (2008) · Asia-Pacific Software Engineering Conference (APSEC) · doi:10.1109/apsec.2008.50 · Cited by 38
Self-citationMEASUREMENT IN SOFTWARE ENGINEERING: FROM THE ROADMAP TO THE CROSSROADS
Bellini, C. G. P., Pereira, R. D. C. D. F., & Becker, J. L. (2008) · International Journal of Software Engineering and Knowledge Engineering · doi:10.1142/s021819400800357x · Cited by 29
Preliminary Reporting Guidelines for Experience Papers
Budgen, D., & Zhang, C. (2009) · Electronic workshops in computing · doi:10.14236/ewic/ease2009.8 · Cited by 15
Self-citationCross-domain investigation of empirical practices
Budgen, D., Bailey, J., Turner, M., Kitchenham, B. A., Pearl Brereton, A., & Charters, S. (2009) · IET Software · doi:10.1049/iet-sen.2008.0106 · Cited by 12
Self-citation
Research works printed but not linked: 1
- Gómez, 2006, A systematic review measurement in software engineering: state-of-the-art in measures, ICSOFT 2006, CCIS, 10, 165
Cited by
Wohlin, C. (2014) · Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering · doi:10.1145/2601248.2601268 · Cited by 4,234
Guidelines for conducting systematic mapping studies in software engineering: An update
Petersen, K., Vakkalanka, S., & Kuzniarz, L. (2015) · Information and Software Technology · doi:10.1016/j.infsof.2015.03.007 · Cited by 2,669
Synthesizing information systems knowledge: A typology of literature reviews
Paré, G., Trudel, M.-C., Jaana, M., & Kitsiou, S. (2015) · Information & Management · doi:10.1016/j.im.2014.08.008 · Cited by 1,987
Collins, C., Dennehy, D., Conboy, K., & Mikalef, P. (2021) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2021.102383 · Cited by 783
The maturity of maturity model research: A systematic mapping study
Wendler, R. (2012) · Information and Software Technology · doi:10.1016/j.infsof.2012.07.007 · Cited by 693
Metadata
- Authors
- Kitchenham, B. A., Budgen, D., & Pearl Brereton, O.
- Venue
- Information and Software Technology
- Publisher
- Elsevier BV
- Published
- 2011-06
- Volume
- 53
- Issue
- 6
- Pages
- 638-651
- ISSN
- 0950-5849 (ISSN-L)
- OpenAlex
- W2072668705
- Cited by
- 621
- References
- 45
- FWCI
- 43.37
- Type
- journal article
- Language
- en
- Open access
- green
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.