HARKing: Hypothesizing After the Results are Known
The DOI matches its Crossref record; 65 of the 65 references it cites were found in a registry or an index.
Abstract
This article considers a practice in scientific communication termed HARKing (Hypothesizing After the Results are Known). HARKing is defined as presenting a post hoc hypothesis (i.e., one based on or informed by one's results) in one's research report as if it were, in fact, an a priori hypotheses. Several forms of HARKing are identified and survey data are presented that suggests that at least some forms of HARKing are widely practiced and widely seen as inappropriate. I identify several reasons why scientists might HARK. Then I discuss several reasons why scientists ought not to HARK. It is conceded that the question of whether HARKing's costs exceed its benefits is a complex one that ought to be addressed through research, open discussion, and debate. To help stimulate such discussion (and for those such as myself who suspect that HARKing's costs do exceed its benefits), I conclude the article with some suggestions for deterring HARKing.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“One consequence of this approach is that unexpected results are often reported as if they were hypothesized a priori (Kerr, 1998).”
From Acceptance and Commitment Therapy Processes and Mediation: Challenges and How to Address Them (2023) · cites this paper as Kerr, 1998
In the citing paper
“Of course, flexible theorizing allows researchers to deduce a potentially infinite number of hypotheses from a priori theory and evidence (Kerr, 1998, p. 210), and these hypotheses can then be used to explain any set of results.”
From Does preregistration improve the credibility of research findings? (2020) · cites this paper as Kerr, 1998
In the citing paper
“Preregistration's historical transparency is not necessary to identify relevant a priori hypotheses, because their relevance makes them obvious as alternative explanations during pre- or post-publication peer review (Kerr, 1998, p. 208; Rubin, 2019a).”
From Does preregistration improve the credibility of research findings? (2020) · cites this paper as Kerr, 1998
In the citing paper
“In fields such as psychology, neuroscience and medicine, practices that increase false discoveries remain not only common, but normative [5–11].”
From The natural selection of bad science (2016) · cites this paper as [8]
In the citing paper
“Poor research design and data analysis encourage false-positive findings.”
From The natural selection of bad science (2016) · cites this paper as [8]
5 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Baron, R. M., & Kenny, D. A. (1986) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.51.6.1173 · Cited by 91,532
Social Theory and Social Structure
Ross, E. J., & Merton, R. K. (1958) · The American Catholic Sociological Review · doi:10.2307/3709394 · Cited by 10,315
The Logic of Scientific Discovery
(2013) · A Realist Theory of Science · doi:10.4324/9780203090732-9 · Cited by 7,731
Journal of social behavior and personality
(1986) · Personality and Individual Differences · doi:10.1016/0191-8869(86)90025-5 · Cited by 6,706
Social Theory and Social Structure.
Bock, K. E., & Merton, R. K. (1969) · American Sociological Review · doi:10.2307/2091967 · Cited by 6,660
Causation, Prediction, and Search
Spirtes, P., Glymour, C., & Scheines, R. (1993) · Lecture Notes in Statistics · doi:10.1007/978-1-4612-2748-9 · Cited by 4,617
Scripts, Plans, Goals, and Understanding: An Inquiry into Human Knowledge Structures
Kronenfeld, D. B., Schank, R. C., & Abelson, R. P. (1978) · Language · doi:10.2307/412850 · Cited by 3,054
The Methodology of Scientific Research Programmes: Philosophical Papers
Lakatos, I. (1978) · doi:10.1017/cbo9780511621123 · Cited by 3,045
Scripts, Plans, Goals, and Understanding: An Inquiry into Human Knowledge Structures
Hunt, E., Schank, R., & Abelson, R. (1979) · The American Journal of Psychology · doi:10.2307/1421499 · Cited by 2,944
Hempel, C. G. (1966) · Medical Entomology and Zoology · Cited by 2,921
Platt, J. R. (1964) · Science · doi:10.1126/science.146.3642.347 · Cited by 2,907
On the Failure to Eliminate Hypotheses in a Conceptual Task
Wason, P. C. (1960) · Quarterly Journal of Experimental Psychology · doi:10.1080/17470216008416717 · Cited by 2,098
Research Methods in Social Relations
(1992) · Journal of Environmental Psychology · doi:10.1016/s0272-4944(05)80303-2 · Cited by 2,081
Science as a Process: An Evolutionary Account of the Social and Conceptual Development of Science
Hull, D. L. (1988) · DigitalGeorgetown (Georgetown University Library) · doi:10.7208/chicago/9780226360492.001.0001 · Cited by 2,061
Dawes, R. M. (1980) · Annual Review of Psychology · doi:10.1146/annurev.ps.31.020180.001125 · Cited by 2,022
The logic of scientific discovery
(1959) · Journal of the Franklin Institute · doi:10.1016/s0016-0032(59)90407-7 · Cited by 1,915
Hindsight is not equal to foresight: The effect of outcome knowledge on judgment under uncertainty.
Fischhoff, B. (1975) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037//0096-1523.1.3.288 · Cited by 1,837
Organizational behavior and human decision processes
Armstrong, J. S. (1988) · International Journal of Forecasting · doi:10.1016/0169-2070(88)90125-2 · Cited by 1,753
Model modifications in covariance structure analysis: The problem of capitalization on chance.
MacCallum, R. C., Roznowski, M., & Necowitz, L. B. (1992) · Psychological Bulletin · doi:10.1037//0033-2909.111.3.490 · Cited by 1,632
Koriat, A., Lichtenstein, S., & Fischhoff, B. (1980) · Journal of Experimental Psychology: Human Learning and Memory · doi:10.1037/0278-7393.6.2.107 · Cited by 1,477
Baron, J. (2023) · Cambridge University Press eBooks · doi:10.1017/9781009263672 · Cited by 1,422
That's Interesting!: Towards a Phenomenology of Sociology and a Sociology of Phenomenology
Davis, M. S. (1971) · Philosophy of the Social Sciences · doi:10.1177/004839317100100211 · Cited by 1,219
Foss, B. (1966) · Medical Entomology and Zoology · Cited by 1,157
Causal thinking and the representation of narrative events
Trabasso, T., & van den Broek, P. (1985) · Journal of Memory and Language · doi:10.1016/0749-596x(85)90049-x · Cited by 1,069
Bias in Human Reasoning: Causes and Consequences
Evans, J. S. B. T. (1990) · Medical Entomology and Zoology · Cited by 1,033
Consequences of prejudice against the null hypothesis.
Greenwald, A. G. (1975) · Psychological Bulletin · doi:10.1037/h0076157 · Cited by 1,022 · cited 2×
“2McGuire (1983a) and Greenwald, Pratkanis, Leippe, and Baumgardner (1986) describe another means to the same end.”
BackgroundHindsight: Biased judgments of past events after the outcomes are known.
Hawkins, S. A., & Hastie, R. (1990) · Psychological Bulletin · doi:10.1037//0033-2909.107.3.311 · Cited by 855
Bad Blood: The Tuskegee Syphilis Experiment
Kiple, K. F., & Jones, J. H. (1982) · The American Historical Review · doi:10.2307/1870318 · Cited by 814
Hindsight-foresight: The effect of outcome knowledge on judgment under uncertainty.: (459202004-001)
Fischhoff, B. (1974) · PsycEXTRA Dataset · doi:10.1037/e459202004-001 · Cited by 799
Varieties of Confirmation Bias
Klayman, J. (1995) · Psychology of Learning and Motivation · doi:10.1016/s0079-7421(08)60315-1 · Cited by 755
Under what conditions does theory obstruct research progress?
Greenwald, A. G., Pratkanis, A. R., Leippe, M. R., & Baumgardner, M. H. (1986) · Psychological Review · doi:10.1037/0033-295x.93.2.216 · Cited by 574
On the psychology of experimental surprises.
Slovic, P., & Fischhoff, B. (1977) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037//0096-1523.3.4.544 · Cited by 485
A century of psychology as science.
(1992) · American Psychological Association eBooks · doi:10.1037/10117-000 · Cited by 461
Horwich, P., & Ayer, A. J. (1973) · The Philosophical Review · doi:10.2307/2183724 · Cited by 414
Scientist as Subject: The Psychological Imperative.
Lewis, L. S., & Mahoney, M. J. (1978) · Contemporary Sociology · doi:10.2307/2064480 · Cited by 412
Perceived informativeness of facts.
Fischhoff, B. (1977) · Journal of Experimental Psychology: Human Perception and Performance · doi:10.1037/0096-1523.3.2.349 · Cited by 404 · cited 2×
“same bias that Slovic and Fischhoffs (1977) lay participants exhibited. The hindsight bias has been replicated many times and appears to be a fairly robust phenomenon (Hawkins & Hastie, 1990). And, although several studies have identified ways of attenuating the bias (Arkes, Faust, Guilmette, & Hart, 1988; Davies, 198…”
BackgroundEliminating the hindsight bias.
Arkes, H. R., Faust, D., Guilmette, T. J., & Hart, K. (1988) · Journal of Applied Psychology · doi:10.1037//0021-9010.73.2.305 · Cited by 396
Hindsight bias among physicians weighing the likelihood of diagnoses.
Arkes, H. R., Wortmann, R. L., Saville, P. D., & Harkness, A. R. (1981) · Journal of Applied Psychology · doi:10.1037//0021-9010.66.2.252 · Cited by 333
The story model for juror decision making
Pennington, N., & Hastie, R. (1993) · Inside the Juror · doi:10.1017/cbo9780511752896.010 · Cited by 304
A Preliminary Discourse on the Study of Natural Philosophy
Herschel, J. F. W. (1987) · University of Chicago Press eBooks · doi:10.7208/chicago/9780226327785.001.0001 · Cited by 289
McGuire, W. J. (1983) · Advances in Experimental Social Psychology · doi:10.1016/s0065-2601(08)60393-7 · Cited by 237
Outcome feedback: Hindsight and information.
Hoch, S. J., & Loewenstein, G. F. (1989) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037//0278-7393.15.4.605 · Cited by 199
Bad Blood: The Tuskegee Syphilis Experiment
Challenor, B., & Jones, J. H. (1982) · Journal of Public Health Policy · doi:10.2307/3342040 · Cited by 198
Strategies of information search in causal analysis
Shaklee, H., & Fischhoff, B. (1982) · Memory & Cognition · doi:10.3758/bf03202434 · Cited by 179
Testing the null hypothesis and the strategy and tactics of investigating theoretical models.
Grant, D. A. (1962) · Psychological Review · doi:10.1037/h0038813 · Cited by 169
Stealing into Print: Fraud, Plagiarism, and Misconduct in Scientific Publishing
Lafollette, M. C. (1992) · University of California Press eBooks · doi:10.1525/9780520917804 · Cited by 161
Hypothesis plausibility and hypothesis generation
Gettys, C. F., & Fisher, S. D. (1979) · Organizational Behavior and Human Performance · doi:10.1016/0030-5073(79)90018-7 · Cited by 147
I knew it all along: or, did I?
Hasher, L., Attig, M. S., & Alba, J. W. (1981) · Journal of Verbal Learning and Verbal Behavior · doi:10.1016/s0022-5371(81)90323-6 · Cited by 121
Ethics for psychologists: A commentary on the APA Ethics Code.
Canter, M. B., Bennett, B. E., Jones, S. E., & Nagy, T. F. (1994) · American Psychological Association eBooks · doi:10.1037/10162-000 · Cited by 110
Davies, M. F. (1987) · Organizational Behavior and Human Decision Processes · doi:10.1016/0749-5978(87)90005-7 · Cited by 83
Confirmatory Versus Comparative Approaches to Judging Theory Tests
Sternthal, B., Tybout, A. M., & Calder, B. J. (1987) · Journal of Consumer Research · doi:10.1086/209097 · Cited by 83
Responsible Science: Ensuring the Integrity of the Research Process
Arrison, T. (2014) · doi:10.2172/1126508 · Cited by 81
The Compleat academic : a practical guide for the beginning social scientist
mark-p-zanna-and-john-m-darley (1986) · Cited by 70
Gergen versus the mainstream: Are hypotheses in social psychology subject to empirical test?
Wallach, L., & Wallach, M. A. (1994) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.67.2.233 · Cited by 63
Horwich, P. (2016) · Cambridge University Press eBooks · doi:10.1017/cbo9781316494219 · Cited by 54
BAYESIANISM AND SUPPORT BY NOVEL FACTS
Howson, C. (1984) · The British Journal for the Philosophy of Science · doi:10.1093/bjps/35.3.245 · Cited by 52
Sharp, D. (1995) · The Lancet · doi:10.1016/s0140-6736(95)91342-4 · Cited by 52
Mathematical and theoretical systems
Keats, J. A. (1989) · Elsevier eBooks · Cited by 43
Heidgerken, L. E. (1954) · Nursing Research · doi:10.1097/00006199-195410000-00008 · Cited by 41
Hypothesized and Confounded Explanations in Theory Tests: A Bayesian Analysis
Brinberg, D., Lynch, Jr., & Sawyer, A. G. (1992) · Journal of Consumer Research · doi:10.1086/209293 · Cited by 39
The Premature Demise of the Solo Experiment
Wegner, D. M. (1992) · Personality and Social Psychology Bulletin · doi:10.1177/0146167292184017 · Cited by 30
Gergen versus the mainstream: Are hypotheses in social psychology subject to empirical test?
Wallach, L., & Wallach, M. A. (1994) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.67.2.233 · Cited by 24
Prediction and Hindsight as Confirmatory Evidence
Simon, H. A. (1955) · Philosophy of Science · doi:10.1086/287427 · Cited by 19
Are There Bellwether Electoral Districts?
Tufte, E. R., & Sun, R. A. (1975) · Public Opinion Quarterly · doi:10.1086/268196 · Cited by 14
How to Win Acceptances by Psychology Journals: 21 Tips for Better Writing
Sternberg, R. J. (2008) · PanPacific management review · Cited by 3
Research works printed but not linked: 18
- American Psychological Association, 1996, Task force on statistical inference initial report (draft) [posted on the World Wide Web]
- 10.1037/0021-9010.73.2.305 · doi:10.1037/0021-9010.73.2.305 · Paper page
- 10.1037/0021-9010.66.2.252 · doi:10.1037/0021-9010.66.2.252 · Paper page
- Baumeister, R., 1993, Dialogue: Newsletter of the Society for Personality and Social Psychology, 8, 10
- Dane, F. C., 1990, Research methods
- Doyle, A. C., 1953, The Complete Sherlock Holmes (Vol. II)
- Feyerabend, P. K., 1993, Against method
- 10.1037/0096-1523.1.3.288 · doi:10.1037/0096-1523.1.3.288 · Paper page
- 10.1037/0033-2909.107.3.311 · doi:10.1037/0033-2909.107.3.311 · Paper page
- Judd, C., 1991, Research methods in social relations
- 10.1037/0033-2909.111.3.490 · DOI · Paper page
- Messick, D. M., 1983, Annual review of personality and social psychology, 11
- Neuliep, J. W., 1993, Journal of Social Behavior and Personality, 8, 21
- Richardson, J. T. E., 1996, Handbook of qualitative research methodsfor psychology and the social sciences
- Seligman, M. E. P., 1991, Learned optimism
- 10.1037/0096-1523.3.4.544 · doi:10.1037/0096-1523.3.4.544 · Paper page
- Sternberg, R. J., 1992, The APS Observer, 5, 12
- Tesser, A., 1994, On becoming a social psychologist
Cited by
A manifesto for reproducible science
Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie du Sert, N., Simonsohn, U., Wagenmakers, E.-J., Ware, J. J., & Ioannidis, J. P. A. (2017) · Nature Human Behaviour · doi:10.1038/s41562-016-0021 · Cited by 3,397
Selective publication of antidepressant trials and its influence on apparent efficacy
Turner, E. H., Matthews, A. M., Linardatos, E., Tell, R. A., & Rosenthal, R. (2008) · New England Journal of Medicine · doi:10.1056/nejmsa065779 · Cited by 2,435
Ellis, P. D. (2010) · doi:10.1017/cbo9780511761676 · Cited by 2,461
The preregistration revolution
Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1708274114 · Cited by 1,998
Genome-wide association studies
Uffelmann, E., Huang, Q. Q., Munung, N. S., de Vries, J., Okada, Y., Martin, A. R., Martin, H. C., Lappalainen, T., & Posthuma, D. (2021) · Nature Reviews Methods Primers · doi:10.1038/s43586-021-00056-9 · Cited by 1,849
Metadata
- Authors
- Kerr, N. L.
- Affiliations
- Michigan State University (US)
- Corresponding author
- Kerr, Norbert L.
- ORCID
- Kerr, Norbert L.
- Venue
- Personality and Social Psychology Review
- Publisher
- SAGE Publications
- Published
- 1998-08
- Volume
- 2
- Issue
- 3
- Pages
- 196-217
- ISSN
- 1088-8683 (ISSN-L)
- 1532-7957
- OpenAlex
- W2118070418
- Cited by
- 1,989
- References
- 65
- Citations by year
- 2026: 98 · 2025: 132 · 2024: 178 · 2023: 182 · 2022: 254 · 2021: 207 · 2020: 208 · 2019: 186 · 2018: 137 · 2017: 126 · 2016: 84 · 2015: 40 · 2014: 39 · 2013: 38 · 2012: 18
- FWCI
- 1.94
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Personality and Social Psychology Review (publishedVersion)
- PubMed (publishedVersion)
- Kent Academic Repository (University of Kent) (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Education › Academic integrity and plagiarism
- Keywords
- HARKing, research reporting practices, scientific integrity, publication practices, research misconduct
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.