The DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
Play It Again, Sam! An Analysis of Constructive Replication in the Organizational Sciences
Open accessAbstract
Replication is an essential part of any science, confirming or adjusting our understanding of the world through repeated exploration of a phenomenon of interest. While there has been an increased interest in the role of replication studies, there also exists skepticism regarding the need for more replication. Our empirical analysis of 406 recent studies that use the term “replication” suggests that this criticism stems from a lack of appreciation of the different forms that replication can take, the prevalence (or lack thereof) of many of these forms, and the objectives that are met by one of the least common forms, constructive replication. As such, the purposes of our paper are (1) to explore the different forms that constructive replication can take and the objectives at which each can be directed, (2) to distinguish these forms from other forms of replication with which they are often confused, (3) to determine how common each form of replication is in our field, and (4) to provide concrete examples of different forms of constructiveness from published studies in order to pave the way towards more (and more useful) replications in the future.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
What is coefficient alpha? An examination of theory and applications
Cortina, J. M. (1993) · Journal of Applied Psychology · doi:10.1037/0021-9010.78.1.98 · Cited by 8,810 · cited 1×
“The number of studies that contain such substantive moderators has been growing steadily for decades (Aguinis, Edwards, & Bradley, 2017; Cortina, 1993).”
BackgroundSelf-citationThe preregistration revolution
Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1708274114 · Cited by 1,998
Pashler, H., & Wagenmakers, E. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612465253 · Cited by 1,321 · cited 1×
“These and other sources include explanations of the importance of replication (e.g., Anderson & Maxwell, 2016; Bettis, Helfat, & Shaver, 2016; Pashler & Wagenmakers, 2012; Simons, 2014; Vazire, 2018) and calls for change to the review process (e.g., Nosek & Lakens, 2014).”
BackgroundThat's Interesting!: Towards a Phenomenology of Sociology and a Sociology of Phenomenology
Davis, M. S. (1971) · Philosophy of the Social Sciences · doi:10.1177/004839317100100211 · Cited by 1,219
Registered Reports: A Method to Increase the Credibility of Published Results
Nosek, B. A., & Lakens, D. (2014) · Social Psychology · doi:10.1027/1864-9335/a000192 · Cited by 716 · cited 2×
“These and other sources include explanations of the importance of replication (e.g., Anderson & Maxwell, 2016; Bettis, Helfat, & Shaver, 2016; Pashler & Wagenmakers, 2012; Simons, 2014; Vazire, 2018) and calls for change to the review process (e.g., Nosek & Lakens, 2014).”
BackgroundAbusive supervision: A meta-analysis and empirical review
Mackey, J. D., Frieder, R. E., Brees, J. R., & Martinko, M. J. (2017) · Journal of Management · doi:10.1177/0149206315573997 · Cited by 706
The Value of Direct Replication
Simons, D. J. (2014) · Perspectives on Psychological Science · doi:10.1177/1745691613514755 · Cited by 630
Nelson, L. D., Simmons, J., & Simonsohn, U. (2018) · Annual Review of Psychology · doi:10.1146/annurev-psych-122216-011836 · Cited by 548 · cited 1×
“Nelson, Simmons, and Simonsohn (2018) and Johnson, Payne, Wang, Asher, and Mandal (2017) recently concluded that the prevalence of such strategies has resulted in a false positive rate likely to be even higher than suggested by the OSC.”
BackgroundShen (2016) · doi:10.1177/0149206314522300 · Cited by 543
Comment on “Estimating the reproducibility of psychological science.”
Gilbert, D. T., King, G., Pettigrew, S., & Wilson, T. D. (2016) · Science · doi:10.1126/science.aad7243 · Cited by 474 · cited 1×
“Borrowing a term from Lykken (1968) and C. W. Kelly, Chase, and Tucker (1979), we might say that Gilbert et al. (2016) criticized the studies in the OSC for not being literal replications, also referred to as direct replication (e.g., Simons, 2014) and exact replication (Stroebe & Strack, 2014).”
Method
Cited by
A perspective on using partial least squares structural equation modelling in data articles
Ringle, C. M., Sarstedt, M., Sinkovics, N., & Sinkovics, R. R. (2023) · Data in Brief · doi:10.1016/j.dib.2023.109074 · Cited by 767
Metacritiques of Upper Echelons Theory: Verdicts and Recommendations for Future Research
Neely, B. H., Lovelace, J. B., Cowen, A. P., & Hiller, N. J. (2020) · Journal of Management · doi:10.1177/0149206320908640 · Cited by 400
Survey response rates: Trends and a validity assessment framework
Holtom, B., Baruch, Y., Aguinis, H., & A Ballinger, G. (2022) · Human Relations · doi:10.1177/00187267211070769 · Cited by 380
Williams, R. I., Clark, L. A., Clark, W. R., & Raffo, D. M. (2021) · European Management Journal · doi:10.1016/j.emj.2020.09.007 · Cited by 300
Metadata
- Authors
- Köhler, T., & Cortina, J. M.
- Venue
- Journal of Management
- Publisher
- SAGE Publications
- Published
- 2019-04-24
- Volume
- 47
- Issue
- 2
- Pages
- 488-518
- ISSN
- 0149-2063 (ISSN-L)
- 1557-1211
- OpenAlex
- W2940938634
- Cited by
- 202
- References
- 17
- FWCI
- 8.94
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Play It Again, Sam! An Analysis of Constructive Replication in the Organizational Sciences. https://app.cochat.ai/papers/10.1177/0149206319843985
- The paper
- Köhler, T., & Cortina, J. M. (2021). Play It Again, Sam! An Analysis of Constructive Replication in the Organizational Sciences. Journal of Management. https://doi.org/10.1177/0149206319843985
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.