Opportunistic biases: Their origins, effects, and an integrated solution.
The DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.
Abstract
Researchers commonly explore their data in multiple ways before deciding which analyses they will include in the final versions of their papers. While this improves the chances of researchers finding publishable results, it introduces an "opportunistic bias," such that the reported relations are stronger or otherwise more supportive of the researcher's theories than they would be without the exploratory process. The magnitudes of opportunistic biases can often be stronger than those of the effects being investigated, leading to invalid conclusions and a lack of clarity in research results. Authors typically do not report their exploratory procedures, so opportunistic biases are very difficult to detect just by reading the final version of a research report. In this article, we explain how a number of accepted research practices can lead to opportunistic biases, discuss the prevalence of these practices in psychology, consider the different effects that opportunistic biases have on psychological science, evaluate the strategies that methodologists have proposed to prevent or correct for the effects of these biases, and introduce an integrated solution to reduce the prevalence and influence of opportunistic biases. The recent prominence of articles discussing questionable research practices both in scientific journals and in the public media underscores the importance of understanding how opportunistic biases are created and how we might undo their effects.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Power failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641 · cited 1×
“Button et al. (2013) echo these concerns, noting that small studies tend to be accompanied by publication bias, selective reporting of outcomes, and worse design quality.”
BackgroundSimmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
The New Statistics: Why and How
Cumming, G. (2014) · Psychological Science · doi:10.1177/0956797613504966 · Cited by 3,200
Raise standards for preclinical cancer research
Begley, C. G., & Ellis, L. M. (2012) · Nature · doi:10.1038/483531a · Cited by 2,979 · cited 1×
“As the recent publicity of biased results in multiple fields (Begley & Ellis, 2012) suggests, if we are not able to reduce opportunistic biases in the practice of science, we risk losing continued scientific progress along with all of its practical benefits.”
BackgroundMeasuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling
John, L. K., Loewenstein, G., & Prelec, D. (2012) · Psychological Science · doi:10.1177/0956797611430953 · Cited by 2,239
HARKing: Hypothesizing after the results are known
Kerr, N. L. (1998) · Personality and Social Psychology Review · doi:10.1207/s15327957pspr0203_4 · Cited by 1,989
Group sequential methods in the design and analysis of clinical trials
POCOCK, S. J. (1977) · Biometrika · doi:10.1093/biomet/64.2.191 · Cited by 1,831 · cited 2×
“, Frick, 1998; Botella, Ximénez, Revuelta, & Suero, 2006), whereas others involve tests specially designed for sequential analysis (e.g., Wald, 1947; Pocock, 1977).”
MethodThe statistical power of abnormal-social psychological research: A review.
Cohen, J. (1962) · The Journal of Abnormal and Social Psychology · doi:10.1037/h0045186 · Cited by 1,824
P-curve: A key to the file-drawer.
Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2014) · Journal of Experimental Psychology: General · doi:10.1037/a0033242 · Cited by 1,500 · cited 1×
“The Center for Open Science has recently developed the Open Science Framework (OSF) to facilitate this process via open collaboration (http://openscienceframework .org, Spies & Nosek, 2014).”
BackgroundPashler, H., & Wagenmakers, E. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612465253 · Cited by 1,321 · cited 1×
“We echo one of Pashler and Wagenmakers’ (2012) conclusions about the problem of replicability, that “. . . it would be a mistake to try to rely upon any single solution to such a complex problem” (p. 529).”
Background
Cited by
Wicherts, J. M., Veldkamp, C. L. S., Augusteijn, H. E. M., Bakker, M., van Aert, R. C. M., & van Assen, M. A. L. M. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.01832 · Cited by 754
Goldberg, S. B., Riordan, K. M., Sun, S., & Davidson, R. J. (2022) · Perspectives on Psychological Science · doi:10.1177/1745691620968771 · Cited by 520
“As is increasingly discussed, selective reporting in particular allows opportunistic bias and can substantially reduce the scientific integrity of a body of literature (DeCoster et al., 2015).”
BackgroundMindfulness Meditation and Psychopathology
Wielgosz, J., Goldberg, S. B., Kral, T. R. A., Dunne, J. D., & Davidson, R. J. (2019) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-021815-093423 · Cited by 435
“For example, concerns about replicability and publication bias (CoronadoMontoya et al. 2016) are relevant across psychological (DeCoster et al. 2015) and medical research (Begley & Ellis 2012).”
BackgroundEnsuring the quality and specificity of preregistrations
Bakker, M., Veldkamp, C. L. S., van Assen, M. A. L. M., Crompvoets, E. A. V., Ong, H. H., Nosek, B. A., Soderberg, C. K., Mellor, D., & Wicherts, J. M. (2020) · PLOS Biology · doi:10.1371/journal.pbio.3000937 · Cited by 138
Giluk (2009) · doi:10.1017/s003329172000389x · Cited by 68
Metadata
- Authors
- DeCoster, J., Sparks, E. A., Sparks, J. C., Sparks, G. G., & Sparks, C. W.
- Venue
- American Psychologist
- Published
- 2015-09
- Volume
- 70
- Issue
- 6
- Pages
- 499-514
- ISSN
- 0003-066X (ISSN-L)
- 1935-990X
- OpenAlex
- W2397189785
- Cited by
- 51
- References
- 20
- FWCI
- 2.39
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Opportunistic biases: Their origins, effects, and an integrated solution. https://app.cochat.ai/papers/10.1037/a0039191
- The paper
- DeCoster, J., Sparks, E. A., Sparks, J. C., Sparks, G. G., & Sparks, C. W. (2015). Opportunistic biases: Their origins, effects, and an integrated solution. American Psychologist. https://doi.org/10.1037/a0039191
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.