The DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Questionable Research Practices Revisited
Abstract
The current discussion of questionable research practices (QRPs) is meant to improve the quality of science. It is, however, important to conduct QRP studies with the same scrutiny as all research. We note problems with overestimates of QRP prevalence and the survey methods used in the frequently cited study by John, Loewenstein, and Prelec. In a survey of German psychologists, we decomposed QRP prevalence into its two multiplicative components, proportion of scientists who ever committed a behavior and, if so, how frequently they repeated this behavior across all their research. The resulting prevalence estimates are lower by order of magnitudes. We conclude that inflated prevalence estimates, due to problematic interpretation of survey data, can create a descriptive norm (QRP is normal) that can counteract the injunctive norm to minimize QRPs and unwantedly damage the image of behavioral sciences, which are essential to dealing with many societal problems.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“the percentage of scientific literature that is actually affected by these practices is unknown, and evidence suggests that it is likely to be smaller, at least five times smaller according to a survey among psychologists (3).”
From Is science really facing a reproducibility crisis, and do we need it to? (2018) · cites this paper as [3]
Citations
Cites
Grice, H. P. (1975) · Speech Acts · doi:10.1163/9789004368811_003 · Cited by 23,221
The Constructive, Destructive, and Reconstructive Power of Social Norms
Schultz, P. W., Nolan, J. M., Cialdini, R. B., Goldstein, N. J., & Griskevicius, V. (2007) · Psychological Science · doi:10.1111/j.1467-9280.2007.01917.x · Cited by 3,643
Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling
John, L. K., Loewenstein, G., & Prelec, D. (2012) · Psychological Science · doi:10.1177/0956797611430953 · Cited by 2,239 · cited 24×
“However, the conspicuous convergence between self-referent admission judgments (in both studies) and the judgments reported by John et al. (2012) concerning others’ behaviors is hard to understand, given that these behaviors are neither precisely defined nor observable in others.”
BackgroundMethodResultInfluentialWillis, G. (2005) · doi:10.4135/9781412983655 · Cited by 1,778
Kahneman, D., & Tversky, A. (1982) · Judgment under Uncertainty · doi:10.1017/cbo9780511809477.015 · Cited by 1,476 · cited 1×
“…to determine the rate of psychologists with nonzero records and resorted to simpler heuristics, for example, a crude sense of familiarity with the topic of each QRP item (Hintzman & Curran, 1994) or some simulation heuristic used to judge the plausibility of behaviors (Kahneman & Tversky, 1982).”
MethodStewart, D. W., Schwarz, N., & Sudman, S. (1996) · Journal of Marketing Research · doi:10.2307/3152152 · Cited by 691 · cited 2×
“In standard survey practice, such ambiguities would have been identified at the questionnaire development stage (Schwarz, 1994; Schwarz & Sudman, 1996; Willis, 2005) and rectified.”
BackgroundSelf-citationDescriptive Social Norms as Underappreciated Sources of Social Control
Cialdini, R. B. (2007) · Psychometrika · doi:10.1007/s11336-006-1560-6 · Cited by 606 · cited 3×
“As numerous studies showed across different domains, communicating descriptive norms (It happens all the time) that are at odds with proscriptive norms (You should not do it) can undermine the influence of the proscriptive norm on individuals’ behavior (Cialdini, 2007; Schultz et al., 2014). In closing, we emphasize o…”
BackgroundThe Role of Representative Design in an Ecological Approach to Cognition.
Dhami, M. K., Hertwig, R., & Hoffrage, U. (2004) · Psychological Bulletin · doi:10.1037/0033-2909.130.6.959 · Cited by 408
Hintzman, D. L., & Curran, T. (1994) · Journal of Memory and Language · doi:10.1006/jmla.1994.1001 · Cited by 391 · cited 2×
“This category mistake is at the heart of most media reports about the John et al. (2012) results.”
BackgroundMethodJudgment in a Social Context: Biases, Shortcomings, and the Logic of Conversation
Schwarz, N. (1994) · Advances in Experimental Social Psychology · doi:10.1016/s0065-2601(08)60153-7 · Cited by 366 · cited 3×
“In standard survey practice, such ambiguities would have been identified at the questionnaire development stage (Schwarz, 1994; Schwarz & Sudman, 1996; Willis, 2005) and rectified.”
BackgroundSelf-citation
Research works printed but not linked: 1
- Wald A., 1947, Sequential analysis
Cited by
Replicability, Robustness, and Reproducibility in Psychological Science
Nosek, B. A., Hardwicke, T. E., Moshontz, H., Allard, A., Corker, K. S., Dreber, A., Fidler, F., Hilgard, J., Kline Struhl, M., Nuijten, M. B., et al. (2022) · Annual Review of Psychology · doi:10.1146/annurev-psych-020821-114157 · Cited by 770
Nelson, L. D., Simmons, J., & Simonsohn, U. (2018) · Annual Review of Psychology · doi:10.1146/annurev-psych-122216-011836 · Cited by 548
“…false-positive rate is 61%, these numbers are 6.6 and 13.1, respectively. www.annualreviews.org • Psychology’s Renaissance 517 Along the same lines, Fiedler & Schwarz (2016) criticized John et al.’s (2012) survey assessing the prevalence of questionable research practices on the grounds that…”
BackgroundInfluentialZwaan, R. A., Etz, A., Lucas, R. E., & Donnellan, M. B. (2018) · Behavioral and Brain Sciences · doi:10.1017/s0140525x17001972 · Cited by 444
The prevalence of statistical reporting errors in psychology (1985–2013)
Nuijten, M. B., Hartgerink, C. H. J., van Assen, M. A. L. M., Epskamp, S., & Wicherts, J. M. (2016) · Behavior Research Methods · doi:10.3758/s13428-015-0664-2 · Cited by 510
“This is in line with the finding of John et al. (2012) that social psychologists admit to more QRPs, find them more applicable to their field, and find them more defensible as compared to other subgroups in psychology (but see also Fiedler & Schwarz, 2015, on this issue).”
The past, present and future of Registered Reports
Chambers, C. D., & Tzavella, L. (2021) · Nature Human Behaviour · doi:10.1038/s41562-021-01193-7 · Cited by 499
Metadata
- Authors
- Fiedler, K., & Schwarz, N.
- Affiliations
- Heidelberg University (DE)
- University of Southern California (US)
- Corresponding author
- Fiedler, Klaus
- Venue
- Social Psychological and Personality Science
- Publisher
- SAGE Publications
- Published
- 2015-10-19
- Volume
- 7
- Issue
- 1
- Pages
- 45-52
- ISSN
- 1948-5506 (ISSN-L)
- 1948-5514
- OpenAlex
- W2201659447
- Cited by
- 279
- References
- 16
- Citations by year
- 2026: 12 · 2025: 26 · 2024: 22 · 2023: 22 · 2022: 47 · 2021: 29 · 2020: 30 · 2019: 29 · 2018: 29 · 2017: 20 · 2016: 10 · 2015: 2
- FWCI
- 59.23
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Social Psychological and Personality Science (publishedVersion)
- Topics
- Social Sciences › Education › Academic integrity and plagiarism
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Keywords
- questionable research practices, prevalence estimates, survey methodology critique, multiplicative decomposition, descriptive norms, injunctive norms, research integrity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Questionable Research Practices Revisited. https://app.cochat.ai/papers/10.1177/1948550615612150
- The paper
- Fiedler, K., & Schwarz, N. (2016). Questionable Research Practices Revisited. Social Psychological and Personality Science. https://doi.org/10.1177/1948550615612150
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.