The DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index.
A Systematic Review of Theory Use in Software Engineering Experiments
Abstract
Empirically based theories are generally perceived as foundational to science. However, in many disciplines, the nature, role and even the necessity of theories remain matters for debate, particularly in young or practical disciplines such as software engineering. This article reports a systematic review of the explicit use of theory in a comprehensive set of 103 articles reporting experiments, from of a total of 5,453 articles published in major software engineering journals and conferences in the decade 1993-2002. Of the 103 articles, 24 use a total of 40 theories in various ways to explain the cause-effect relationship(s) under investigation. The majority of these use theory in the experimental design to justify research questions and hypotheses, some use theory to provide post hoc explanations of their results, and a few test or modify theory. A third of the theories are proposed by authors of the reviewed articles. The interdisciplinary nature of the theories used is greater than that of research in software engineering in general. We found that theory use and awareness of theoretical issues are present, but that theory-driven research is, as yet, not a major issue in empirical software engineering. Several articles comment explicitly on the lack of relevant theory. We call for an increased awareness of the potential benefits of involving theory, when feasible. To support software engineering researchers who wish to use theory, we show which of the reviewed articles on which topics use which theories for what purposes, as well as details of the theories' characteristics
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Tversky, A., & Kahneman, D. (1974) · Science · doi:10.1126/science.185.4157.1124 · Cited by 28,810
The magical number seven, plus or minus two: Some limits on our capacity for processing information.
Miller, G. A. (1956) · Psychological Review · doi:10.1037/h0043158 · Cited by 17,812
ASPECTS OF THE THEORY OF SYNTAX
Chomsky, N. (1964) · doi:10.21236/ad0616323 · Cited by 13,587
Organizational Information Requirements, Media Richness and Structural Design
Daft, R. L., & Lengel, R. H. (1986) · Management Science · doi:10.1287/mnsc.32.5.554 · Cited by 9,868
A spreading-activation theory of semantic processing.
Collins, A. M., & Loftus, E. F. (1975) · Psychological Review · doi:10.1037/0033-295x.82.6.407 · Cited by 8,141
Human Memory: A Proposed System and its Control Processes
Atkinson, R. C., & Shiffrin, R. M. (1968) · Psychology of Learning and Motivation · doi:10.1016/s0079-7421(08)60422-3 · Cited by 7,216
Fraassen, B. C. V. (1980) · doi:10.1093/0198244274.001.0001 · Cited by 4,682
The Nature of Theory in Information Systems1
Gregor, S. (2006) · MIS Quarterly · doi:10.2307/25148742 · Cited by 3,306 · cited 6×
“All these issues have a bearing on the nature and components of prospective theories.”
BackgroundMethodInfluentialDuncker, K. (1945) · Psychological Monographs · doi:10.1037/h0093599 · Cited by 2,938
Confirmation, disconfirmation, and information in hypothesis testing
Klayman, J., & Ha, Y.-W. (1987) · Psychological Review · doi:10.1037/0033-295x.94.2.211 · Cited by 2,094
Reconceptualizing the Determinants of Risk Behavior
Sitkin, S. B., & Pablo, A. L. (1992) · The Academy of Management Review · doi:10.2307/258646 · Cited by 2,057
The theoretical status of latent variables.
Borsboom, D., Mellenbergh, G. J., & van Heerden, J. (2003) · Psychological Review · doi:10.1037/0033-295x.110.2.203 · Cited by 1,566
Stimulus information as a determinant of reaction time.
Hyman, R. (1953) · Journal of Experimental Psychology · doi:10.1037/h0056940 · Cited by 1,443
Cognitive Fit: A Theory‐Based Analysis of the Graphs Versus Tables Literature*
Vessey, I. (1991) · Decision Sciences · doi:10.1111/j.1540-5915.1991.tb00344.x · Cited by 1,263
That's Interesting!: Towards a Phenomenology of Sociology and a Sociology of Phenomenology
Davis, M. S. (1971) · Philosophy of the Social Sciences · doi:10.1177/004839317100100211 · Cited by 1,219
Rosch, E. (1988) · Readings in Cognitive Science · doi:10.1016/b978-1-4832-1446-7.50028-5 · Cited by 946
Gigerenzer, G. (2004) · The Journal of Socio-Economics · doi:10.1016/j.socec.2004.09.033 · Cited by 890
Cognitive Fit: An Empirical Study of Information Acquisition
Vessey, I., & Galletta, D. (1991) · Information Systems Research · doi:10.1287/isre.2.1.63 · Cited by 705
Stimulus structures and mental representations in expert comprehension of computer programs
Pennington, N. (1987) · Cognitive Psychology · doi:10.1016/0010-0285(87)90007-7 · Cited by 526
Comments on “What theory is not”
DiMaggio, P. J. (1995) · Administrative Science Quarterly · doi:10.2307/2393790 · Cited by 488
Perspective-based Usability Inspection: An Empirical Validation of Efficacy
Zhang, Z., Basili, V., & Shneiderman, B. (1999) · Empirical Software Engineering · doi:10.1023/a:1009803214692 · Cited by 96
The Scope of General Theory: Methods for Linking Deductive and Inductive Comparative History
BOSWELL, T., & BROWN, C. (1999) · Sociological Methods & Research · doi:10.1177/0049124199028002002 · Cited by 45
Cited by
Systematic Literature Reviews in Software Engineering—A Systematic Literature Review
Kitchenham, B., Pearl Brereton, O., Budgen, D., Turner, M., Bailey, J., & Linkman, S. (2009) · Information and Software Technology · doi:10.1016/j.infsof.2008.09.009 · Cited by 4,537
Guidelines for conducting and reporting case study research in software engineering
Runeson, P., & Höst, M. (2009) · Empirical Software Engineering · doi:10.1007/s10664-008-9102-8 · Cited by 3,862
Systematic Mapping Studies in Software Engineering
Petersen, K., Feldt, R., Mujtaba, S., & Mattsson, M. (2008) · Electronic Workshops in Computing · doi:10.14236/ewic/ease2008.8 · Cited by 3,239
“The high number of categories for systematic reviews and their detail level is particularly visible for the reviews (Hannay et al. 2007, Sjøberg et al. 2005).”
Using mapping studies as the basis for further research – A participant-observer case study
Kitchenham, B. A., Budgen, D., & Pearl Brereton, O. (2011) · Information and Software Technology · doi:10.1016/j.infsof.2010.12.011 · Cited by 621
Identifying relevant studies in software engineering
Zhang, H., Babar, M. A., & Tell, P. (2011) · Information and Software Technology · doi:10.1016/j.infsof.2010.12.010 · Cited by 557
Metadata
- Authors
- Hannay, J. E., Sjoberg, D. I. K., & Dyba, T.
- Venue
- IEEE Transactions on Software Engineering
- Published
- 2007-02
- Volume
- 33
- Issue
- 2
- Pages
- 87-107
- ISSN
- 0098-5589 (ISSN-L)
- 1939-3520
- OpenAlex
- W2122626138
- Cited by
- 226
- References
- 22
- FWCI
- 7.65
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.