The DOI matches its Crossref record; 50 of the 50 references it cites were found in a registry or an index.
The Replication Recipe: What makes for a convincing replication?
Open accessAbstract
Psychological scientists have recently started to reconsider the importance of close replications in building a cumulative knowledge base; however, there is no consensus about what constitutes a convincing close replication study. To facilitate convincing close replication attempts we have developed a Replication Recipe, outlining standard criteria for a convincing close replication. Our Replication Recipe can be used by researchers, teachers, and students to conduct meaningful replication studies and integrate replications into their scholarly habits.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007) · Behavior Research Methods · doi:10.3758/bf03193146 · Cited by 65,971
Cohen, J. (1992) · Psychological Bulletin · doi:10.1037//0033-2909.112.1.155 · Cited by 8,771
A basic introduction to fixed-effect and random-effects models for meta-analysis
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2010) · Research Synthesis Methods · doi:10.1002/jrsm.12 · Cited by 6,793
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
Behavioral Study of obedience.
Milgram, S. (1963) · The Journal of Abnormal and Social Psychology · doi:10.1037/h0040525 · Cited by 4,918 · cited 1×
“Burger (2009) successfully replicated Milgram's famous obedience experiments (e.g., Milgram, 1963), suggesting that when well-conducted replications are successful they can provide us with greater con fi dence about the veracity of the predicted effect.”
BackgroundMood and judgment: The affect infusion model (AIM).
Forgas, J. P. (1995) · Psychological Bulletin · doi:10.1037/0033-2909.117.1.39 · Cited by 3,470
Social class, solipsism, and contextualism: How the rich are different from the poor.
Kraus, M. W., Piff, P. K., Mendoza-Denton, R., Rheinschmidt, M. L., & Keltner, D. (2012) · Psychological Review · doi:10.1037/a0028756 · Cited by 1,397
Nosek (2012) · doi:10.1177/1745691612459058 · Cited by 1,301
Shall we Really do it Again? The Powerful Concept of Replication is Neglected in the Social Sciences
Schmidt, S. (2009) · Review of General Psychology · doi:10.1037/a0015108 · Cited by 1,125
An Agenda for Purely Confirmatory Research
Wagenmakers, E.-J., Wetzels, R., Borsboom, D., van der Maas, H. L. J., & Kievit, R. A. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612463078 · Cited by 1,042 · cited 1×
“suspect ulterior motives of the replicator, and makes it more difficult to exercise liberty in choosing an analytic method to exploit the chances of declaring the findings in favor of (or against) the hypothesis (Simmons et al., 2011; Wagenmakers et al., 2012).”
Method
Cited by
Investigating Variation in Replicability: A “Many Labs” Replication Project
Klein, R. A., Ratliff, K. A., Vianello, M., Adams, R. B., Bahník, Š., Bernstein, M. J., Bocian, K., Brandt, M. J., Brooks, B., Brumbaugh, C. C., et al. (2014) · Social Psychology · doi:10.1027/1864-9335/a000178 · Cited by 1,208
Self-citationConstraints on Generality (COG): A Proposed Addition to All Empirical Papers
Simons, D. J., Shoda, Y., & Lindsay, D. S. (2017) · Perspectives on Psychological Science · doi:10.1177/1745691617708630 · Cited by 986
Is psychology suffering from a replication crisis? What does “failure to replicate” really mean?
Maxwell, S. E., Lau, M. Y., & Howard, G. S. (2015) · American Psychologist · doi:10.1037/a0039400 · Cited by 1,005
Westfall, J., Kenny, D. A., & Judd, C. M. (2014) · Journal of Experimental Psychology: General · doi:10.1037/xge0000014 · Cited by 928
Kleiman, E. M., Turner, B. J., Fedor, S., Beale, E. E., Huffman, J. C., & Nock, M. K. (2017) · Journal of Abnormal Psychology · doi:10.1037/abn0000273 · Cited by 831
“This specific replication paradigm (i.e., close replication; Brandt et al., 2014; Finkel, Eastwick, & Reis, 2015) is important because beyond the need to know if our findings replicate in general, it is important to know if our findings generalize across the spectrum of severity of suicide risk.”
Schäfer, T., & Schwarz, M. A. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.00813 · Cited by 774
Aquaporin-4-dependent glymphatic solute transport in the rodent brain
Mestre, H., Hablitz, L. M., Xavier, A. L., Feng, W., Zou, W., Pu, T., Monai, H., Murlidharan, G., Castellanos Rivera, R. M., Simon, M. J., et al. (2018) · eLife · doi:10.7554/elife.40070 · Cited by 738
Psychology, Science, and Knowledge Construction: Broadening Perspectives from the Replication Crisis
Shrout, P. E., & Rodgers, J. L. (2018) · Annual Review of Psychology · doi:10.1146/annurev-psych-122216-011845 · Cited by 725
Jusup, M., Holme, P., Kanazawa, K., Takayasu, M., Romić, I., Wang, Z., Geček, S., Lipić, T., Podobnik, B., Wang, L., et al. (2022) · Physics Reports · doi:10.1016/j.physrep.2021.10.005 · Cited by 643
The Value of Direct Replication
Simons, D. J. (2014) · Perspectives on Psychological Science · doi:10.1177/1745691613514755 · Cited by 630
Facts Are More Important Than Novelty: Replication in the Education Sciences
Makel, M. C., & Plucker, J. A. (2014) · Educational Researcher · doi:10.3102/0013189x14545513 · Cited by 609
“312 EDUCATIONAL RESEARCHER Although universal standards of conducting replications have (obviously) not yet been adopted, some have been proposed (e.g., Brandt et al., 2014; Open Science Collaboration, 2012).”
BackgroundNelson, L. D., Simmons, J., & Simonsohn, U. (2018) · Annual Review of Psychology · doi:10.1146/annurev-psych-122216-011836 · Cited by 548
Replication, falsification, and the crisis of confidence in social psychology
Earp, B. D., & Trafimow, D. (2015) · Frontiers in Psychology · doi:10.3389/fpsyg.2015.00621 · Cited by 472
Single-Paper Meta-Analysis: Benefits for Study Summary, Theory Testing, and Replicability
McShane, B. B., & Böckenholt, U. (2017) · Journal of Consumer Research · doi:10.1093/jcr/ucw085 · Cited by 457
Zwaan, R. A., Etz, A., Lucas, R. E., & Donnellan, M. B. (2018) · Behavioral and Brain Sciences · doi:10.1017/s0140525x17001972 · Cited by 444
Pre-registration in social psychology—A discussion and suggested template
van 't Veer, A. E., & Giner-Sorolla, R. (2016) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2016.03.004 · Cited by 439
Self-citationContextual sensitivity in scientific reproducibility
Van Bavel, J. J., Mende-Siedlecki, P., Brady, W. J., & Reinero, D. A. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1521897113 · Cited by 398
Moshontz, H., Campbell, L., Ebersole, C. R., IJzerman, H., Urry, H. L., Forscher, P. S., Grahe, J. E., McCarthy, R. J., Musser, E. D., Antfolk, J., et al. (2018) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245918797607 · Cited by 377
Self-citationEvidence that psychopathology symptom networks have limited replicability.
Forbes, M. K., Wright, A. G. C., Markon, K. E., & Krueger, R. F. (2017) · Journal of Abnormal Psychology · doi:10.1037/abn0000276 · Cited by 354
Freese, J., & Peterson, D. (2017) · Annual Review of Sociology · doi:10.1146/annurev-soc-060116-053450 · Cited by 310
Swedberg, R. (2020) · The Production of Knowledge · doi:10.1017/9781108762519.002 · Cited by 293
Bayesian tests to quantify the result of a replication attempt.
Verhagen, J., & Wagenmakers, E.-J. (2014) · Journal of Experimental Psychology: General · doi:10.1037/a0036731 · Cited by 291
Evaluating CloudResearch’s Approved Group as a solution for problematic data quality on MTurk
Hauser, D. J., Moss, A. J., Rosenzweig, C., Jaffe, S. N., Robinson, J., & Litman, L. (2022) · Behavior Research Methods · doi:10.3758/s13428-022-01999-x · Cited by 269
A Large Scale Test of the Effect of Social Class on Prosocial Behavior
Korndörfer, M., Egloff, B., & Schmukle, S. C. (2015) · PLOS ONE · doi:10.1371/journal.pone.0133193 · Cited by 255
Claesen, A., Gomes, S., Tuerlinckx, F., & Vanpaemel, W. (2021) · Royal Society Open Science · doi:10.1098/rsos.211037 · Cited by 197
Crowdsourcing hypothesis tests: Making transparent how design choices shape research results.
Landy, J. F., Jia, M. (., Ding, I. L., Viganola, D., Tierney, W., Dreber, A., Johannesson, M., Pfeiffer, T., Ebersole, C. R., Gronau, Q. F., et al. (2020) · Psychological Bulletin · doi:10.1037/bul0000220 · Cited by 168
Pre-Analysis Plans Have Limited Upside, Especially Where Replications Are Feasible
Coffman, L. C., & Niederle, M. (2015) · Journal of Economic Perspectives · doi:10.1257/jep.29.3.81 · Cited by 156
Ensuring the quality and specificity of preregistrations
Bakker, M., Veldkamp, C. L. S., van Assen, M. A. L. M., Crompvoets, E. A. V., Ong, H. H., Nosek, B. A., Soderberg, C. K., Mellor, D., & Wicherts, J. M. (2020) · PLOS Biology · doi:10.1371/journal.pbio.3000937 · Cited by 138
Cognitive control in media multitaskers: Two replication studies and a meta-Analysis
Wiradhany, W., & Nieuwenstein, M. R. (2017) · Attention, Perception, & Psychophysics · doi:10.3758/s13414-017-1408-4 · Cited by 121
I’ll see it when i believe it: motivated numeracy in perceptions of climate change risk
Nurse, M. S., & Grant, W. J. (2020) · Environmental Communication · doi:10.1080/17524032.2019.1618364 · Cited by 83
Modeling stimulus variation in three common implicit attitude tasks
Wolsiefer, K., Westfall, J., & Judd, C. M. (2017) · Behavior Research Methods · doi:10.3758/s13428-016-0779-0 · Cited by 73
Grimmelikhuijsen, S., John, P., Meijer, A., & Worthy, B. (2019) · Journal of Behavioral Public Administration · doi:10.30636/jbpa.12.34 · Cited by 52
Chatard, A., Hirschberger, G., & Pyszczynski, T. (2020) · doi:10.31234/osf.io/ejubn · Cited by 51
Hayes, J., & Schimel, J. (2018) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2017.10.005 · Cited by 45
Hardwicke, T. E., Szűcs, D., Thibault, R. T., Crüwell, S., van den Akker, O. R., Nuijten, M. B., & Ioannidis, J. P. A. (2021) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459211040837 · Cited by 44
Defending one's worldview under mortality salience: Testing the validity of an established idea
Schindler, S., Reinhardt, N., & Reinhard, M.-A. (2021) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2020.104087 · Cited by 40
The shooter bias: Replicating the classic effect and introducing a novel paradigm
Essien, I., Stelter, M., Kalbe, F., Koehler, A., Mangels, J., & Meliß, S. (2017) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2016.12.009 · Cited by 39
Harris, E. A., & Van Bavel, J. J. (2020) · doi:10.31234/osf.io/hfuas · Cited by 28
Moery, E., & Calin-Jageman, R. J. (2016) · Social Psychological and Personality Science · doi:10.1177/1948550616639649 · Cited by 16
Chandrashekar, S. P., Yeung, S. K., Yau, K. C., Cheung, C. Y., Agarwal, T. K., Wong, C. Y. J., Pillai, T., Thirlwell, T. N., Leung, W. N., Tse, C., et al. (2021) · Judgment and Decision Making · doi:10.1017/s1930297500008470 · Cited by 11
“Following the recent developments in psychological science and the growing recognition of the importance of reproducibility and replicability (e.g., Brandt et al., 2014; Open Science Collaboration, 2015; Van‘t Veer & Giner-Sorolla, 2016; Zwaan et al., 2018), we aimed to revisit the classic work on…”
Designators, descriptions, and artifact persistence
Marchak, K. A., & Hall, D. G. (2019) · Cognition · doi:10.1016/j.cognition.2019.06.011 · Cited by 4
Flynn, M. K., & Hebert, E. R. (2022) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2021.11.005 · Cited by 2
Metadata
- Authors
- Brandt, M. J., IJzerman, H., Dijksterhuis, A., Farach, F. J., Geller, J., Giner-Sorolla, R., Grange, J. A., Perugini, M., Spies, J. R., & van 't Veer, A.
- Venue
- Journal of Experimental Social Psychology
- Publisher
- Elsevier BV
- Published
- 2014-01
- Volume
- 50
- Pages
- 217-224
- ISSN
- 0022-1031 (ISSN-L)
- OpenAlex
- W3124580540
- Cited by
- 705
- References
- 50
- FWCI
- 49.00
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.