Seven principles of effective replication studies: strengthening the evidence base of management research
Open accessThe DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.
Abstract
Management research is often obsessed with novelty or originality.This is understandable and in line with most other academic disciplines.Novel findings are more interesting and can have a huge impact on management theory and practice.However, novelty as a sine qua non criterion is accompanied by serious threats to both the academic community and management practice.The academic community is increasingly concerned that many of these novel findings might be nonreplicable artifacts.Consequently, there is a plethora of results in the literature that might not constitute any knowledge, thus threatening the credibility and practical usefulness of management research.Replication studies, which we define as studies that put published empirical results to an additional empirical test, are needed for the discipline to develop in a meaningful way and close the theory practice gap.To be fair, the claim that most published management research is false or nonreplicable is probably an exaggeration.However, without replication in particular, the social sciences (like management research) are vulnerable to overestimating effect sizes (Camerer et al. 2018).While in management research it is uncommon to conduct explicit replication studies, many original findings are replicated implicitly.These findings may be shown as control variables models only before the authors proceed to their own empirical contribution, such as a moderation effect based on the (replicated) direct effect.Nevertheless, more explicit replications would be useful to aid efforts in synthesizing and aggregating available knowledge and will help to put the academic discussion on a solid foundation of empirical evidence.To improve this base of empirical evidence, Management Review Quarterly (MRQ) publishes structured literature reviews (Fisch and Block 2018), meta-analyses and, since 2018, replications.These instruments or submission types of evidence-based B Andreas Kuckertz
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
HARKing: Hypothesizing after the results are known
Kerr, N. L. (1998) · Personality and Social Psychology Review · doi:10.1207/s15327957pspr0203_4 · Cited by 1,989
Camerer, C. F., Dreber, A., Holzmeister, F., Ho, T.-H., Huber, J., Johannesson, M., Kirchler, M., Nave, G., Nosek, B. A., Pfeiffer, T., et al. (2018) · Nature Human Behaviour · doi:10.1038/s41562-018-0399-z · Cited by 1,521
Six tips for your (systematic) literature review in business and management research
Fisch, C., & Block, J. (2018) · Management Review Quarterly · doi:10.1007/s11301-018-0142-x · Cited by 516
Self-citation
Cited by
Walter, A.-T. (2021) · Management Review Quarterly · doi:10.1007/s11301-020-00186-6 · Cited by 331
Eight tips and questions for your bibliographic study in business and management research
Block, J. H., & Fisch, C. (2020) · Management Review Quarterly · doi:10.1007/s11301-020-00188-4 · Cited by 272
Self-citationHow to conduct a meta-analysis in eight steps: A practical guide
Hansen, C., Steinmetz, H., & Block, J. (2022) · Management Review Quarterly · doi:10.1007/s11301-021-00247-4 · Cited by 218
Self-citationReviewing systematic literature reviews: ten key questions and criteria for reviewers
Kuckertz, A., & Block, J. (2021) · Management Review Quarterly · doi:10.1007/s11301-021-00228-7 · Cited by 119
Self-citationNonprofit capacity and social performance: Mapping the field and future directions
Nordin, N., Khatibi, A., & Azam, S. M. F. (2024) · Management Review Quarterly · doi:10.1007/s11301-022-00297-2 · Cited by 52
“A bibliographic analysis is one type of systematic literature review, and it is widely applied for summarizing social science research (Block and Fisch 2020; Block and Kuckertz 2018).”
Background
Metadata
- Authors
- Block, J., & Kuckertz, A.
- Venue
- Management Review Quarterly
- Published
- 2018-09-27
- Volume
- 68
- Issue
- 4
- Pages
- 355-359
- ISSN
- 2198-1620 (ISSN-L)
- 2198-1639
- OpenAlex
- W2893952279
- Cited by
- 84
- References
- 3
- FWCI
- 6.60
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.