CoChat

Psychologists update their beliefs about effect sizes after replication studies

Open access

Published version of a preprint

The DOI matches its Crossref record; 56 of the 56 references it cites were found in a registry or an index.

Abstract

Self-correction-a key feature distinguishing science from pseudoscience-requires that scientists update their beliefs in light of new evidence. However, people are often reluctant to change their beliefs. We examined belief updating in action by tracking research psychologists' beliefs in psychological effects before and after the completion of four large-scale replication projects. We found that psychologists did update their beliefs; they updated as much as they predicted they would, but not as much as our Bayesian model suggests they should if they trust the results. We found no evidence that psychologists became more critical of replications when it would have preserved their pre-existing beliefs. We also found no evidence that personal investment or lack of expertise discouraged belief updating, but people higher on intellectual humility updated their beliefs slightly more. Overall, our results suggest that replication studies can contribute to self-correction within psychology, but psychologists may underweight their evidentiary value.

Overview

Integrity signals

Registry

Crossref · 10.1038/s41562-021-01220-7

Cites

56 of 56 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 56 references

Research works printed but not linked: 3

  1. Luchins, A. S. in The Order of Presentation in Persuasion (ed. C. Hovland) (Yale Univ. Press, 1957).
  2. Munro, G. D., Leary, S. P. & Lasane, T. P. Between a rock and a hard place: biased assimilation of scientific information in the face of commitment. North Am. J. Psychol. 6, 431–444 (2004).
  3. Jost, A. Die Assoziationsfestigkeit in ihrer Abha ̈ngigkeit von der Verteilung der Wiederholungen [The strength of associations in their dependence on the distribution of repetitions]. Z. Psychol. Physiol. Si. 16, 436–472 (1897).

Cited by

See all 22

Metadata

Authors
McDiarmid, A. D., Tullett, A. M., Whitt, C. M., Vazire, S., Smaldino, P. E., & Stephens, J. E.
Affiliations
  • University of Alabama (US)
  • The University of Melbourne (AU)
  • University of California, Merced (US)
Corresponding author
  • McDiarmid, Alex D.
  • Tullett, Alexa M.
Venue
Nature Human Behaviour
Published
2021-11-22
Volume
5
Issue
12
Pages
1663-1673
ISSN
2397-3374 (ISSN-L)
OpenAlex
W3215782000
Cited by
16
References
56
Citations by year
2026: 1 · 2025: 1 · 2023: 7 · 2022: 2 · 2021: 5
FWCI
2.02
Type
journal article
Language
en
Open access
Licence
Locations
  • Nature Human Behaviour (publishedVersion)
  • PubMed (publishedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • PsyArXiv (OSF Preprints) (submittedVersion)
Funders
National Science Foundation
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
  • Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
Keywords
belief updating, replication study, effect size, self-correction, Bayesian updating, intellectual humility, personal investing, trust in research, replication crisis
MeSH terms
Humans, Psychology, Research, Statistics as Topic

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Psychologists update their beliefs about effect sizes after replication studies. https://app.cochat.ai/papers/10.1038/s41562-021-01220-7
The paper
McDiarmid, A. D., Tullett, A. M., Whitt, C. M., Vazire, S., Smaldino, P. E., & Stephens, J. E. (2021). Psychologists update their beliefs about effect sizes after replication studies. Nature Human Behaviour. https://doi.org/10.1038/s41562-021-01220-7

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.