The DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Preliminary guidelines for empirical research in software engineering
Open accessAbstract
Empirical software engineering research needs research guidelines to improve the research and reporting processes. We propose a preliminary set of research guidelines aimed at stimulating discussion among software researchers. They are based on a review of research guidelines developed for medical researchers and on our own experience in doing and reviewing software engineering research. The guidelines are intended to assist researchers, reviewers, and meta-analysts in designing, conducting, and evaluating empirical studies. Editorial boards of software engineering journals may wish to use our recommendations as a basis for developing guidelines for reviewers and for framing policies for dealing with the design, data collection, and analysis and reporting of empirical studies.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Coefficient Alpha and the Internal Structure of Tests
Cronbach, L. J. (1951) · Psychometrika · doi:10.1007/bf02310555 · Cited by 43,931
Statistical Power Analysis for the Behavioral Sciences
Cohen, J. (2013) · doi:10.4324/9780203771587 · Cited by 25,373
Statistical Analysis with Missing Data
Little, R. J. A., & Rubin, D. B. (2002) · Wiley Series in Probability and Statistics · doi:10.1002/9781119013563 · Cited by 8,473
Improving the Quality of Reporting of Randomized Controlled Trials: The CONSORT Statement
Begg, C. (1996) · JAMA · doi:10.1001/jama.1996.03540080059030 · Cited by 3,127
Statistical methods in psychology journals: Guidelines and explanations.
Wilkinson, L., & Task Force on Statistical Inference (1999) · American Psychologist · doi:10.1037/0003-066x.54.8.594 · Cited by 2,791
Heinsman, D. T., & Shadish, W. R. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.2.154 · Cited by 125
Cited by
Guidelines for conducting and reporting case study research in software engineering
Runeson, P., & Höst, M. (2009) · Empirical Software Engineering · doi:10.1007/s10664-008-9102-8 · Cited by 3,862
Empirical studies of agile software development: A systematic review
Dybå, T., & Dingsøyr, T. (2008) · Information and Software Technology · doi:10.1016/j.infsof.2008.01.006 · Cited by 2,668
(2004) · doi:10.5555/998675.999432 · Cited by 656
Applying Systematic Reviews to Diverse Study Types: An Experience Report
Dyba, T., Dingsoyr, T., & Hanssen, G. K. (2007) · First International Symposium on Empirical Software Engineering and Measurement (ESEM 2007) · doi:10.1109/esem.2007.59 · Cited by 391
Are Students Representatives of Professionals in Software Engineering Experiments?
Salman, I., Misirli, A. T., & Juristo, N. (2015) · 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering · doi:10.1109/icse.2015.82 · Cited by 304
(2015) · doi:10.1007/s10664-013-9279-3 · Cited by 273
The ABC of Software Engineering Research
Stol, K.-J., & Fitzgerald, B. (2018) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3241743 · Cited by 245
SEGRESS: Software Engineering Guidelines for REporting Secondary Studies
Kitchenham, B., Madeyski, L., & Budgen, D. (2023) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2022.3174092 · Cited by 219
Self-citationDeveloping search strategies for detecting relevant experiments
Dieste, O., Grimán, A., & Juristo, N. (2009) · Empirical Software Engineering · doi:10.1007/s10664-008-9091-7 · Cited by 200
Quality, productivity and economic benefits of software reuse: a review of industrial studies
Mohagheghi, P., & Conradi, R. (2007) · Empirical Software Engineering · doi:10.1007/s10664-007-9040-x · Cited by 197
Using an expert panel to validate a requirements process improvement model
Beecham, S., Hall, T., Britton, C., Cottee, M., & Rainer, A. (2005) · Journal of Systems and Software · doi:10.1016/j.jss.2004.06.004 · Cited by 171
Usability evaluation of mobile applications using ISO 9241 and ISO 25062 standards
Moumane, K., Idri, A., & Abran, A. (2016) · SpringerPlus · doi:10.1186/s40064-016-2171-z · Cited by 163
Measuring and predicting software productivity: A systematic map and review
Petersen, K. (2011) · Information and Software Technology · doi:10.1016/j.infsof.2010.12.001 · Cited by 128
Graziotin, D., Wang, X., & Abrahamsson, P. (2015) · Journal of Software: Evolution and Process · doi:10.1002/smr.1673 · Cited by 126
A systematic review of comparative evidence of aspect-oriented programming
Ali, M. S., Ali Babar, M., Chen, L., & Stol, K.-J. (2010) · Information and Software Technology · doi:10.1016/j.infsof.2010.05.003 · Cited by 89
SYMBALS: A Systematic Review Methodology Blending Active Learning and Snowballing
van Haastrecht, M., Sarhan, I., Yigit Ozkan, B., Brinkhuis, M., & Spruit, M. (2021) · Frontiers in Research Metrics and Analytics · doi:10.3389/frma.2021.685591 · Cited by 80
Sustainability in computing education: A systematic literature review
Peters, A.-K., Capilla, R., Coroamă, V. C., Heldal, R., Lago, P., Leifler, O., Moreira, A., Fernandes, J. P., Penzenstadler, B., Porras, J., et al. (2024) · ACM Transactions on Computing Education · doi:10.1145/3639060 · Cited by 78
(2007) · doi:10.1049/iet-sen:20060052 · Cited by 67
Empathic pedagogical conversational agents: A systematic literature review
Ortega‐Ochoa, E., Arguedas, M., & Daradoumis, T. (2024) · British Journal of Educational Technology · doi:10.1111/bjet.13413 · Cited by 61
Worse Than Spam: Issues In Sampling Software Developers
Baltes, S., & Diehl, S. (2016) · Proceedings of the 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement · doi:10.1145/2961111.2962628 · Cited by 54
Techniques and countermeasures for preventing insider threats
Alsowail, R. A., & Al-Shehari, T. (2022) · PeerJ Computer Science · doi:10.7717/peerj-cs.938 · Cited by 45
Kuttal, S. K., Ong, B., Kwasny, K., & Robe, P. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445659 · Cited by 44
A mapping study on documentation in Continuous Software Development
Theunissen, T., van Heesch, U., & Avgeriou, P. (2022) · Information and Software Technology · doi:10.1016/j.infsof.2021.106733 · Cited by 40
Bałdyga, M., Barański, K., Belter, J., Kalinowski, M., & Weichbroth, P. (2024) · Sensors · doi:10.3390/s24082633 · Cited by 40
Identifying contradictions in an incumbent–startup ecosystem–an activity theory approach
Baloutsos, S., Karagiannaki, A., & Pramatari, K. (2022) · European Journal of Innovation Management · doi:10.1108/ejim-04-2020-0114 · Cited by 35
The Emotional Roller Coaster of Responding to Requirements Changes in Software Engineering
Madampe, K., Hoda, R., & Grundy, J. (2023) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2022.3172925 · Cited by 33
A Framework for Emotion-Oriented Requirements Change Handling in Agile Software Engineering
Madampe, K., Hoda, R., & Grundy, J. (2023) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2023.3253145 · Cited by 32
Wyrich, M., Preikschat, A., Graziotin, D., & Wagner, S. (2021) · 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE) · doi:10.1109/icse43902.2021.00055 · Cited by 24
Studying test-driven development and its retainment over a six-month time span
Baldassarre, M. T., Caivano, D., Fucci, D., Juristo, N., Romano, S., Scanniello, G., & Turhan, B. (2021) · Journal of Systems and Software · doi:10.1016/j.jss.2021.110937 · Cited by 15
Understanding Similarities and Differences in Software Development Practices Across Domains
Viggiato, M., Oliveira, J., Figueiredo, E., Jamshidi, P., & Kastner, C. (2019) · 2019 ACM/IEEE 14th International Conference on Global Software Engineering (ICGSE) · doi:10.1109/icgse.2019.00013 · Cited by 15
(2020) · doi:10.1145/3382494.3422164 · Cited by 13
Automatic enforcement of constraints in real-time collaborative architectural decision making
Gaubatz, P., Lytra, I., & Zdun, U. (2015) · Journal of Systems and Software · doi:10.1016/j.jss.2015.01.056 · Cited by 11
Metadata
- Authors
- Kitchenham, B. A., Pfleeger, S. L., Pickard, L. M., Jones, P. W., Hoaglin, D. C., El Emam, K., & Rosenberg, J.
- Venue
- IEEE Transactions on Software Engineering
- Published
- 2002-08
- Volume
- 28
- Issue
- 8
- Pages
- 721-734
- ISSN
- 0098-5589 (ISSN-L)
- 1939-3520
- 2326-3881
- OpenAlex
- W2163851162
- Cited by
- 1,510
- References
- 6
- FWCI
- 122.67
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.