The DOI matches its Crossref record; 211 of the 211 references it cites were found in a registry or an index.
On doing relevant and rigorous experiments: Review and recommendations
Open accessAbstract
Although experiments are the gold standard for establishing causality, several threats can undermine the internal validity of experimental findings. In this article, we first discuss these threats, which include the lack of consequential decisions and outcomes, deception, demand effects and unfair comparisons, as well as issues concerning statistical validity (e.g., minimum sample size per cell, estimating variance correctly). We expose each problem, show potential solutions, and bring to the fore issues of relevance of the findings (i.e., external and ecological validity). Thereafter, we take stock of the state‐of‐the‐science regarding validity threats using a representative sample of 468 recent experiments from 258 articles published in top‐tier journals. We compare research practices in three fields of study—management, social psychology, and economics, which regularly use experimental research—to operations management, which has more recently begun to use the experimental paradigm. Our results underscore the importance for journals and authors to follow what we identify to be best‐practice methodological suggestions (i.e., the “ten commandments” of experimental research). We show that—on average—markers of methodological rigor and generalizability positively and significantly predict the citations received by published articles. Finally, given that experiments are infeasible in some settings, we conclude with a brief review of often overlooked quasi‐experimental designs, which are useful for generating strong counterfactuals and hence allow making causal claims in the field.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Items 201–211 of 211
Righting a wrong: Retaliation on a voodoo doll symbolizing an abusive supervisor restores justice
Liang, L. H., Brown, D. J., Lian, H., Hanig, S., Ferris, D. L., & Keeping, L. M. (2018) · The Leadership Quarterly · doi:10.1016/j.leaqua.2018.01.004 · Cited by 69
Manipulation checks: Advantage or disadvantage?
Kidd, R. F. (1976) · Cited by 69
Bøggild, T., & Laustsen, L. (2016) · The Leadership Quarterly · doi:10.1016/j.leaqua.2016.09.003 · Cited by 54
Remarks on Virtual World and Virtual Reality Experiments
Harrison, G. W., Haruvy, E. E., & Rutström, E. E. (2011) · Southern Economic Journal · doi:10.4284/0038-4038-78.1.87 · Cited by 45
Groupthink and Project Performance: The Influence of Personal Traits and Interpersonal Ties
Riccobono, F., Bruccoleri, M., & Größler, A. (2015) · Production and Operations Management · doi:10.1111/poms.12431 · Cited by 43
Bolton, G. E., & Ockenfels, A. (2013) · Journal of Economic Behavior & Organization · doi:10.1016/j.jebo.2013.11.003 · Cited by 41
Hopes for the future of operations management
Meredith, J. R. (2001) · Journal of Operations Management · doi:10.1016/s0272-6963(01)00056-0 · Cited by 36
Small sample failure of random assignment: A further examination.
Strube, M. (1991) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.59.2.346 · Cited by 31
Introduction to the Special Issue on Behavioral Operations
Gans, N., & Croson, R. T. A. (2008) · Manufacturing & Service Operations Management · doi:10.1287/msom.1080.0227 · Cited by 30
Reputation, Honesty, and Cheating in Informal Milk Markets in IndiaPreprint
Krrll, M., & Rustagi, D. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.2982365 · Cited by 23
Constructing useful theory: The case of Six Sigma
de Treville, S., Edelson, N. M., Kharkar, A. N., & Avanzi, B. (2008) · Operations Management Research · doi:10.1007/s12063-008-0005-z · Cited by 17
Cited by
Cooper, B., Eva, N., Zarea Fazlelahi, F., Newman, A., Lee, A., & Obschonka, M. (2020) · Journal of Vocational Behavior · doi:10.1016/j.jvb.2020.103472 · Cited by 339
Abusive supervision: A systematic review and fundamental rethink
Fischer, T., Tian, A. W., Lee, A., & Hughes, D. J. (2021) · The Leadership Quarterly · doi:10.1016/j.leaqua.2021.101540 · Cited by 318
Podsakoff, P. M., & Podsakoff, N. P. (2019) · The Leadership Quarterly · doi:10.1016/j.leaqua.2018.11.002 · Cited by 312
A critique of the Leader-Member Exchange construct: Back to square one
Gottfredson, R. K., Wright, S. L., & Heaphy, E. D. (2020) · The Leadership Quarterly · doi:10.1016/j.leaqua.2020.101385 · Cited by 205
Humble leadership: A review and synthesis of leader expressed humility
Kelemen, T. K., Matthews, S. H., Matthews, M. J., & Henry, S. E. (2023) · Journal of Organizational Behavior · doi:10.1002/job.2608 · Cited by 203
Metadata
- Authors
- Lonati, S., Quiroga, B. F., Zehnder, C., & Antonakis, J.
- Venue
- Journal of Operations Management
- Publisher
- Wiley
- Published
- 2018-11
- Volume
- 64
- Issue
- 1
- Pages
- 19-40
- ISSN
- 0272-6963 (ISSN-L)
- 1873-1317
- OpenAlex
- W2910905828
- Cited by
- 354
- References
- 211
- FWCI
- 16.99
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.