Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Cited by 182

The DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.

When Does HARKing Hurt? Identifying When Different Types of Undisclosed Post Hoc Hypothesizing Harm Scientific Progress

Open access
Rubin, Mark · 2017 · Review of General Psychology

Published version of a preprint

Abstract

Hypothesizing after the results are known, or HARKing, occurs when researchers check their research results and then add or remove hypotheses on the basis of those results without acknowledging this process in their research report ( Kerr, 1998 ). In the present article, I discuss 3 forms of HARKing: (a) using current results to construct post hoc hypotheses that are then reported as if they were a priori hypotheses; (b) retrieving hypotheses from a post hoc literature search and reporting them as a priori hypotheses; and (c) failing to report a priori hypotheses that are unsupported by the current results. These 3 types of HARKing are often characterized as being bad for science and a potential cause of the current replication crisis. In the present article, I use insights from the philosophy of science to present a more nuanced view. Specifically, I identify the conditions under which each of these 3 types of HARKing is most and least likely to be bad for science. I conclude with a brief discussion about the ethics of each type of HARKing.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1037/gpr0000128

Cites

16 of 16 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 48

  1. 10.1371/journal.pone.0172792 · DOI · Paper page
  2. 10.1057/s41267-017-0081-0 · DOI · Paper page
  3. 10.5465/AMLE.2010.56659889 · DOI · Paper page
  4. Bem D. J., 1987, The compleat academic: A practical guide for the beginning social scientist, 171
  5. 10.1111/peps.12111 · DOI
  6. 10.5465/amle.2015.0201 · DOI
  7. 10.2307/2216049 · doi:10.2307/2216049
  8. 10.1016/j.actpsy.2014.02.001 · DOI · Paper page
  9. 10.1016/j.jmp.2015.10.003 · DOI · Paper page
  10. 10.1371/journal.pone.0010271 · DOI · Paper page
  11. 10.1207/s15327957pspr0203_1 · DOI
  12. 10.1093/bjps/axl004 · DOI
  13. 10.1207/s15327957pspr0203_2 · DOI · Paper page
  14. 10.1093/bjps/55.1.1 · DOI · Paper page
  15. 10.1177/0149206316679487 · DOI · Paper page
  16. HowsonC. (1988). Accommodation, prediction and Bayesian confirmation theory. Proceedings of the Biennial Meeting of the Philosophy of Science Association (Vol. 2, pp. 381–392). Philosophy of Science Association. · DOI
  17. 10.1177/0956797611430953 · DOI · Paper page
  18. 10.1111/iops.12045 · DOI
  19. 10.1093/acprof:osobl/9780199778188.003.0024 · DOI
  20. 10.1093/bjps/52.3.575 · DOI
  21. 10.1080/19312450701641409 · DOI
  22. 10.1037/a0039400 · DOI · Paper page
  23. 10.1086/289639 · DOI
  24. 10.1093/bjps/axn034 · DOI
  25. Mayo D. G., 2010, Error and inference: Recent exchanges on experimental reasoning, reliability, and the objectivity and rationality of science, 155
  26. 10.1016/j.shpsa.2013.10.005 · DOI
  27. 10.1093/bjps/axl003 · DOI · Paper page
  28. 10.1111/iops.12049 · DOI
  29. 10.1037/pspa0000084 · DOI
  30. 10.1177/1088868313496330 · DOI
  31. 10.1093/bjps/25.1.1 · DOI · Paper page
  32. O'Boyle E. H., 2017, Journal of Management, 43, 367
  33. ParkerC. G. (2015). Reframing the reproducibility crisis: Using an error-statistical account to inform the interpretation of replication results in psychological research. Doctoral dissertation, Virginia Polytechnic Institute and State University. Retrieved from https://vtechworks.lib.vt.edu/handle/10919/52963
  34. Popper K. R., 1979, Objective knowledge: An evolutionary approach
  35. 10.1037/gpr0000123 · DOI · Paper page
  36. 10.1080/00048408712342751 · doi:10.1080/00048408712342751
  37. 10.5465/amle.2016.0039 · DOI
  38. SimonsD. J., ShodaY. & LindsayD. S. (2017, April 17). Constraints on generality (COG): A proposed addition to all empirical papers. Retrieved from http://osf.io/preprints/psyarxiv/w9e3r · DOI
  39. 10.1016/j.jesp.2015.09.017 · DOI
  40. Ṡwiątkowski W., 2017, Revue Internationale de Psychologie Sociale, 30, 111
  41. 10.3389/fpsyg.2016.01444 · DOI
  42. 10.1027/1864-9335/a000270 · DOI
  43. 10.1016/j.jesp.2016.03.004 · DOI · Paper page
  44. Wagenmakers E. J., 2015, Extrasensory perception: Support, skepticism, and science:, 153
  45. 10.1037/a0022790 · DOI · Paper page
  46. 10.1007/978-94-009-6525-6_11 · DOI
  47. Worrall J., 2010, Error and inference: Recent exchanges on experimental reasoning, reliability, and the objectivity and rationality of science, 125
  48. 10.1093/bjps/24.2.95 · DOI · Paper page

Cited by

See all 182

Metadata

Authors
Rubin, M.
Venue
Review of General Psychology
Published
2017-12
Volume
21
Issue
4
Pages
308-320
ISSN
  • 1089-2680 (ISSN-L)
  • 1939-1552
OpenAlex
W2765220332
Cited by
182
References
16
FWCI
22.57
Type
journal article
Language
en
Open access
hybrid
Licence
journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.