Commentary and Rejoinder on
The DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.
Abstract
Johnson, Cheung, and Donnellan (2014a) reported a failure to replicate Schnall, Benton, and Harvey (2008) ’s effect of cleanliness on moral judgment. However, inspection of the replication data shows that participants provided high numbers of severe moral judgments – a ceiling effect. In the original data percentage of extreme responses per moral dilemma correlated negatively with the effect of the manipulation. In contrast, this correlation was absent in the replications, due to almost all items showing a high percentage of extreme responses. Therefore the parametric statistics reported by Johnson et al. (2014a) are inconclusive regarding the reproducibility of the original effect. Direct replications are prone to error when reviewers only judge similarity of methods, but not resulting data and conclusions. It is my conclusion that preventable problems can arise if publication decisions are made without independent post-data peer evaluation.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“and failed replications are often disputed [50,51,52].”
From The natural selection of bad science (2016) · cites this paper as [51]
Citations
Cites
Disgust as Embodied Moral Judgment
Schnall, S., Haidt, J., Clore, G. L., & Jordan, A. H. (2008) · Personality and Social Psychology Bulletin · doi:10.1177/0146167208317771 · Cited by 1,350
Self-citationRegistered Reports: A Method to Increase the Credibility of Published Results
Nosek, B. A., & Lakens, D. (2014) · Social Psychology · doi:10.1027/1864-9335/a000192 · Cited by 716
With a Clean Conscience: Cleanliness Reduces the Severity of Moral Judgments
Schnall, S., Benton, J., & Harvey, S. (2008) · Psychological Science · doi:10.1111/j.1467-9280.2008.02227.x · Cited by 511
Self-citationGino, F., & Mogilner, C. (2014) · Psychological Science · doi:10.1177/0956797613506438 · Cited by 173
Does Cleanliness Influence Moral Judgments?: A Direct Replication of
Johnson, D. J., Cheung, F., & Donnellan, M. B. (2014) · Social Psychology · doi:10.1027/1864-9335/a000186 · Cited by 103
Self-citationCleanliness and godliness: Mutual association between two kinds of personal purity
Preston, J. L., & Ritter, R. S. (2012) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2012.05.015 · Cited by 97
Kay, A. C., Laurin, K., Fitzsimons, G. M., & Landau, M. J. (2014) · Journal of Experimental Psychology: General · doi:10.1037/a0034462 · Cited by 93
Clean Data: Statistical Artefacts Wash Out Replication Efforts
Schnall, S. (2014) · Apollo (University of Cambridge) · Cited by 17
Self-citation
Cited by
The natural selection of bad science
Smaldino, P. E., & McElreath, R. (2016) · Royal Society Open Science · doi:10.1098/rsos.160384 · Cited by 860
Replicability, Robustness, and Reproducibility in Psychological Science
Nosek, B. A., Hardwicke, T. E., Moshontz, H., Allard, A., Corker, K. S., Dreber, A., Fidler, F., Hilgard, J., Kline Struhl, M., Nuijten, M. B., et al. (2022) · Annual Review of Psychology · doi:10.1146/annurev-psych-020821-114157 · Cited by 770
Freese, J., & Peterson, D. (2017) · Annual Review of Sociology · doi:10.1146/annurev-soc-060116-053450 · Cited by 310
Robustness of statistical methods when measure is affected by ceiling and/or floor effect
Šimkovic, M., & Träuble, B. (2019) · PLOS ONE · doi:10.1371/journal.pone.0220889 · Cited by 162
Replication, Communication, and the Population Dynamics of Scientific Discovery
McElreath, R., & Smaldino, P. E. (2015) · PLOS ONE · doi:10.1371/journal.pone.0136088 · Cited by 117
Metadata
- Authors
- Schnall, S., Johnson, D. J., Cheung, F., & Brent Donnellan, M.
- Affiliations
- University of Cambridge (GB)
- Michigan State University (US)
- Department of Psychology, Michigan State University, East Lansing, MI, USA
- Venue
- Social Psychology
- Publisher
- Hogrefe Publishing Group
- Published
- 2014-05-01
- Volume
- 45
- Issue
- 4
- Pages
- 315-320
- ISSN
- 1864-9335 (ISSN-L)
- 2151-2590
- OpenAlex
- W1982995101
- Cited by
- 25
- References
- 8
- Citations by year
- 2026: 1 · 2021: 2 · 2019: 3 · 2018: 3 · 2017: 2 · 2016: 8 · 2015: 3 · 2014: 3
- FWCI
- 2.97
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Social Psychology (publishedVersion)
- The HKU Scholars Hub (University of Hong Kong) (submittedVersion)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Psychology of Moral and Emotional Judgment
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- replication failure, ceiling effect, moral judgment, cleanliness, ethical dilemmas, direct replication, reproducibility, publication decisions, extreme response
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Commentary and Rejoinder on. https://app.cochat.ai/papers/10.1027/1864-9335/a000204
- The paper
- Schnall, S., Johnson, D. J., Cheung, F., & Brent Donnellan, M. (2014). Commentary and Rejoinder on. Social Psychology. https://doi.org/10.1027/1864-9335/a000204
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.