Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 40 of the 41 references it cites were found in a registry or an index.

Investigating Variation in Replicability: A “Many Labs” Replication Project

Open access

Published version of a preprint

Abstract

Although replication is a central tenet of science, direct replications are rare in psychology. This research tested variation in the replicability of 13 classic and contemporary effects across 36 independent samples totaling 6,344 participants. In the aggregate, 10 effects replicated consistently. One effect – imagined contact reducing prejudice – showed weak support for replicability. And two effects – flag priming influencing conservatism and currency priming influencing system justification – did not replicate. We compared whether the conditions such as lab versus online or US versus international sample predicted effect magnitudes. By and large they did not. The results of this small sample of effects suggest that replicability is more dependent on the effect itself than on the sample and setting used to investigate the effect.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1027/1864-9335/a000178

Cites

40 of 40 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 41 references

Research works printed but not linked: 2

  1. Hyman H. H., 1950, The teaching of contemporary affairs: 21st yearbook of the National Council of Social Studies, 11
  2. Klein, R. A. Ratliff, K. A. Vianello, M. Adams, R. B. Jr. Bahník, Š. Bernstein, M. J. … Nosek, B. A. (2014). Theory building through replication: Response to commentaries on the “Many Labs” replication project. Commentaries and rejoinder on Klein et al. (2014). Social Psychology . Advance online publication. doi: 10.1027/1864-9335/1000202 · doi:10.1027/1864-9335/1000202

Cited by

See all 1,208

Metadata

Authors
Klein, R. A., Ratliff, K. A., Vianello, M., Adams, R. B., Bahník, Š., Bernstein, M. J., Bocian, K., Brandt, M. J., Brooks, B., Brumbaugh, C. C., Cemalcilar, Z., Chandler, J., Cheong, W., Davis, W. E., Devos, T., Eisner, M., Frankowska, N., Furrow, D., Galliani, E. M., Hasselman, F., Hicks, J. A., Hovermale, J. F., Hunt, S. J., Huntsinger, J. R., IJzerman, H., John, M.-S., Joy-Gaba, J. A., Barry Kappes, H., Krueger, L. E., Kurtz, J., Levitan, C. A., Mallett, R. K., Morris, W. L., Nelson, A. J., Nier, J. A., Packard, G., Pilati, R., Rutchick, A. M., Schmidt, K., Skorinko, J. L., Smith, R., Steiner, T. G., Storbeck, J., Van Swol, L. M., Thompson, D., van ‘t Veer, A. E., Ann Vaughn, L., Vranka, M., Wichman, A. L., Woodzicka, J. A., & Nosek, B. A.
Affiliations
  • University of Florida (US)
  • University of Padua (IT)
  • Pennsylvania State University (US)
  • University of Würzburg (DE)
  • Uniwersytet SWPS (PL)
  • Tilburg University (NL)
  • City University of New York (US)
  • Koç University, Istanbul, Turkey
  • University of Michigan (US)
  • HELP University (MY)
  • Texas A&M University (US)
  • San Diego State University (US)
  • Mount Saint Vincent University (CA)
  • Radboud University Nijmegen (NL)
  • Virginia Commonwealth University (US)
  • East Texas A&M University (US)
  • Loyola University Chicago (US)
  • Worcester Polytechnic Institute (US)
  • London School of Economics and Political Science, London, UK
  • James Madison University (US)
  • Occidental College (US)
  • McDaniel College (US)
  • Connecticut College (US)
  • Wilfrid Laurier University (CA)
  • Universidade de Brasília (BR)
  • California State University, Northridge (US)
  • University of Virginia (US)
  • The Ohio State University (US)
  • University of Wisconsin-Madison, WI, USA
  • Tilburg University, The Netherlands
  • Ithaca College, NY, USA
  • Charles University (CZ)
  • Western Kentucky University (US)
  • Washington and Lee University (US)
  • Center for Open Science (US)
Venue
Social Psychology
Published
2014-05-01
Volume
45
Issue
3
Pages
142-152
ISSN
  • 1864-9335 (ISSN-L)
  • 2151-2590
OpenAlex
W1941783251
Cited by
1,208
References
41
Citations by year
2026: 62 · 2025: 62 · 2024: 73 · 2023: 97 · 2022: 103 · 2021: 143 · 2020: 151 · 2019: 135 · 2018: 114 · 2017: 86 · 2016: 88 · 2015: 62 · 2014: 28
FWCI
286.88
Type
journal article
Language
en
Open access
Locations
  • Social Psychology (publishedVersion, cc-by-nc-nd)
  • London School of Economics and Political Science Research Online (London School of Economics and Political Science) (acceptedVersion)
  • London School of Economics and Political Science Theses Online (London School of Economics and Political Science) (acceptedVersion)
  • Radboud Repository (Radboud University) (submittedVersion)
  • Tilburg University Research Portal (submittedVersion, other-oa)
  • Research portal (Tilburg University) (submittedVersion, other-oa)
  • Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
  • CSUN ScholarWorks (California State University, Northridge) (submittedVersion)
  • Padua Research Archive (University of Padova) (submittedVersion, cc-by-nc-nd)
  • Radboud Repository (Radboud University) (submittedVersion, other-oa)
Topics
  • Social Sciences › Sociology and Political Science › Social and Intergroup Psychology
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
Keywords
replicability, replication, direct replication, imagined contact, prejudice, conservatism, system justification

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Investigating Variation in Replicability: A “Many Labs” Replication Project. https://app.cochat.ai/papers/10.1027/1864-9335/a000178
The paper
Klein, R. A., Ratliff, K. A., Vianello, M., Adams, R. B., Bahník, Š., Bernstein, M. J., Bocian, K., Brandt, M. J., Brooks, B., Brumbaugh, C. C., Cemalcilar, Z., Chandler, J., Cheong, W., Davis, W. E., Devos, T., Eisner, M., Frankowska, N., Furrow, D., Galliani, E. M., . . . Nosek, B. A. (2014). Investigating Variation in Replicability: A “Many Labs” Replication Project. Social Psychology. https://doi.org/10.1027/1864-9335/a000178

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.