Priming, Replication, and the Hardest Science
Abstract
Concerns have been raised recently about the replicability of behavioral priming effects, and calls have been issued to identify priming methodologies with effects that can be obtained in any context and with any population. I argue that such expectations are misguided and inconsistent with evolutionary understandings of the brain as a computational organ. Rather, we should expect priming effects to be highly sensitive to variations in experimental features and subject populations. Such variation does not make priming effects frivolous or capricious but instead can be predicted a priori. However, absent theories specifying the precise contingencies that lead to such variation, failures to replicate another researcher’s findings will necessarily be ambiguous with respect to the inferences that can be made. Priming research is not yet at the stage where such theories exist, and therefore failures are uninformative at the current time. Ultimately, priming researchers themselves must provide direct replications of their own effects; researchers have been deficient in meeting this responsibility and have contributed to the current state of confusion. The recommendations issued in this article reflect concerns both with the practice of priming researchers and with the inappropriate expectations of researchers who have failed to replicate others’ priming effects.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Several prominent examples of behavioral priming have proven difficult to replicate.
“Several prominent examples of behavioral priming have proven difficult to replicate56,57”
From Accuracy prompts are a replicable and generalizable approach for reducing the spread of misinformation (2022) · cites this paper as [56]
Cites
The weirdest people in the world?
Henrich, J., Heine, S. J., & Norenzayan, A. (2010) · Behavioral and Brain Sciences · doi:10.1017/s0140525x0999152x · Cited by 12,641 · see citing works
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505 · see citing works
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567 · see citing works
Social psychology: handbook of basic principles
Higgins, E., & Kruglanski, A. (2008) · Choice Reviews Online · doi:10.5860/choice.45-3496 · Cited by 5,534 · see citing works
Social psychology: handbook of basic principles
(1997) · Choice Reviews Online · doi:10.5860/choice.34-3595 · Cited by 4,970 · see citing works
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · see citing works
Endostatin: an endogenous inhibitor of angiogenesis and tumor growth
O'Reilly, M. S., Boehm, T., Shing, Y., Fukai, N., Vasios, G., Lane, W. S., Flynn, E., Birkhead, J. R., Olsen, B. R., & Folkman, J. (1997) · Cell · doi:10.1016/s0092-8674(00)81848-6 · Cited by 4,486 · see citing works
WEIRD languages have misled us, too
Majid, A., & Levinson, S. C. (2010) · Behavioral and Brain Sciences · doi:10.1017/s0140525x1000018x · Cited by 4,169 · see citing works
Editorial Office (1997) · Obesity Surgery · doi:10.1381/096089297765555359 · Cited by 3,917 · see citing works
Facilitation in recognizing pairs of words: Evidence of a dependence between retrieval operations.
Meyer, D. E., & Schvaneveldt, R. W. (1971) · Journal of Experimental Psychology · doi:10.1037/h0031564 · Cited by 2,974 · see citing works
Neely, J. H. (1977) · Journal of Experimental Psychology General · doi:10.1037/0096-3445.106.3.226 · Cited by 2,826 · see citing works
Bargh, J. A., Chen, M., & Burrows, L. (1996) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.71.2.230 · Cited by 2,736 · see citing works
Bargh, J. A., Chen, M. I.-C., & Burrows, L. (1996) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.71.2.230 · Cited by 2,637 · see citing works
The Handbook of evolutionary psychology
(2006) · Choice Reviews Online · doi:10.5860/choice.44-0622 · Cited by 2,128 · see citing works
Knowledge activation: Accessibility, applicability, and salience.
Higgins, E. T. (1996) · Cited by 1,970 · see citing works
Meehl, P. E. (1978) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.46.4.806 · Cited by 1,952 · see citing works
Pashler, H., & Wagenmakers, E. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612465253 · Cited by 1,321 · see citing works
Judd, C. M., Westfall, J., & Kenny, D. A. (2012) · Journal of Personality and Social Psychology · doi:10.1037/a0028347 · Cited by 1,236 · see citing works
Formidability and the logic of human anger
Sell, A. N., Tooby, J., & Cosmides, L. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0904312106 · Cited by 826 · see citing works
Recommendations for Increasing Replicability in Psychology
Asendorpf, J. B., Conner, M. T., Fruyt, F. D., Houwer, J. D., Denissen, J. J. A., Fiedler, K., Fiedler, S., Funder, D. C., Kliegl, R. M., Nosek, B. A., et al. (2013) · European Journal of Personality · doi:10.1002/per.1919 · Cited by 818 · see citing works
Conceptual Foundations of Evolutionary Psychology
Tooby, J., & Cosmides, L. (2015) · Wiley eBooks · doi:10.1002/9780470939376.ch1 · Cited by 648 · see citing works
Stereotypes as Judgmental Heuristics: Evidence of Circadian Variations in Discrimination
Bodenhausen, G. V. (1990) · Psychological Science · doi:10.1111/j.1467-9280.1990.tb00226.x · Cited by 614 · see citing works
Roaring and numerical assessment in contests between groups of female lions, Panthera leo
McComb, K., Packer, C., & Pusey, A. E. (1994) · Animal Behaviour · doi:10.1006/anbe.1994.1052 · Cited by 603 · see citing works
Positive Affect as Implicit Motivator: On the Nonconscious Operation of Behavioral Goals.
Ruud Custers, A., & Aarts, H. (2005) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.89.2.129 · Cited by 563 · see citing works
Wilson, M. L., HAUSER, M. D., & Wrangham, R. W. (2001) · Animal Behaviour · doi:10.1006/anbe.2000.1706 · Cited by 478 · see citing works
Bos, P. A., Panksepp, J., Bluthé, R.-M., & Honk, J. V. (2012) · Frontiers in Neuroendocrinology · doi:10.1016/j.yfrne.2011.01.002 · Cited by 429 · see citing works
Defensive behaviors of laboratory and wild Rattus norvegicus..
Blanchard, R. J., Flannelly, K. J., & Blanchard, D. C. (1986) · Journal of comparative psychology · doi:10.1037/0735-7036.100.2.101 · Cited by 405 · see citing works
Understanding the Role of the Self in Prime-to-Behavior Effects: The Active-Self Account
Wheeler, S. C., DeMarree, K. G., & Petty, R. E. (2007) · Personality and Social Psychology Review · doi:10.1177/1088868307302223 · Cited by 365 · see citing works
Blanchard, D. C., Hynd, A. L., Minke, K. A., Minemoto, T., & Blanchard, R. J. (2001) · Neuroscience & Biobehavioral Reviews · doi:10.1016/s0149-7634(01)00056-2 · Cited by 346 · see citing works
Loersch, C., & Payne, B. K. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691611406921 · Cited by 278 · see citing works
Automatic social behavior as motivated preparation to interact.
Cesario, J., Plaks, J. E., & Higgins, E. T. (2006) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.90.6.893 · Cited by 253 · see citing works
Self-citationCooperation in male lions: kinship, reciprocity or mutualism?
Grinnell, J., Packer, C., & Pusey, A. E. (1995) · Animal Behaviour · doi:10.1016/0003-3472(95)80157-x · Cited by 248 · see citing works
The Automaticity of Race and Afrocentric Facial Features in Social Judgments.
Blair, I. V., Judd, C. M., & Fallman, J. L. (2004) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.87.6.763 · Cited by 243 · see citing works
On the Relation between Associative Strength and Automatic Behavior
Dijksterhuis, A., Aarts, H., Bargh, J. A., & van Knippenberg, A. (2000) · Journal of Experimental Social Psychology · doi:10.1006/jesp.2000.1427 · Cited by 202 · see citing works
DeMarree, K. G., Wheeler, S. C., & Petty, R. E. (2005) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.89.5.657 · Cited by 168 · see citing works
The affective meanings of automatic social behaviors: Three mechanisms that explain priming.
Schröder, T., & Thagard, P. R. (2012) · Psychological Review · doi:10.1037/a0030972 · Cited by 155 · see citing works
Chimpanzees and the mathematics of battle
Wilson, M. L., Britton, N. F., & Franks, N. R. (2002) · Proceedings of the Royal Society B Biological Sciences · doi:10.1098/rspb.2001.1926 · Cited by 154 · see citing works
Shantz, A. D., & Latham, G. P. (2009) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2009.01.001 · Cited by 138 · see citing works
Numerical assessment and individual call discrimination by wild spotted hyaenas, Crocuta crocuta
Benson‐Amram, S., Heinen, V. K., Dryer, S. L., & Holekamp, K. E. (2011) · Animal Behaviour · doi:10.1016/j.anbehav.2011.07.004 · Cited by 135 · see citing works
The Connection Between Varying Treatment Effects and the Crisis of Unreplicable Research
Gelman, A. (2015) · Journal of Management · doi:10.1177/0149206314525208 · Cited by 125 · see citing works
Cesario, J., Plaks, J. E., Hagiwara, N., Navarrete, C. D., & Higgins, E. T. (2010) · Psychological Science · doi:10.1177/0956797610378685 · Cited by 125 · see citing works
Self-citationThe sex → aggression link: A perception-behavior dissociation.
Mussweiler, T., & Förster, J. (2000) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.79.4.507 · Cited by 124 · see citing works
Fessler, D. M. T., & Holbrook, C. (2013) · Psychological Science · doi:10.1177/0956797612461508 · Cited by 117 · see citing works
BODIES IN CONTEXT: POWER POSES AS A COMPUTATION OF ACTION POSSIBILITY
Cesario, J., & Mcdonald, M. M. (2013) · Social Cognition · doi:10.1521/soco.2013.31.2.260 · Cited by 91 · see citing works
Self-citationThe influence of social groups on goal contagion
Loersch, C., Aarts, H., Payne, B. K., & Jefferis, V. E. (2008) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2008.07.009 · Cited by 89 · see citing works
Mussweiler, T., & Neumann, R. (2000) · Journal of Experimental Social Psychology · doi:10.1006/jesp.1999.1415 · Cited by 86 · see citing works
The independent effects of skin tone and facial features on Whites' affective reactions to Blacks
Hagiwara, N., Kashy, D. A., & Cesario, J. (2012) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2012.02.001 · Cited by 83 · see citing works
Self-citationTarget prototypicality moderates racial bias in the decision to shoot
Ma, D. S., & Correll, J. (2010) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2010.11.002 · Cited by 76 · see citing works
Knowing how to react: Automatic response priming from social categories.
Jonas, K. J., & Sassenberg, K. (2006) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.90.5.709 · Cited by 66 · see citing works
Blonde like me: When self-construals moderate stereotype priming effects on intellectual performance
Bry, C., Follenfant, A., & Meyer, T. (2007) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2007.06.005 · Cited by 42 · see citing works
When Chronic Isn't Chronic: The Moderating Role of Active Self-Aspects
Brown, C. M., & McConnell, A. R. (2008) · Personality and Social Psychology Bulletin · doi:10.1177/0146167208324675 · Cited by 31 · see citing works
Timing and selection effects within a psychology subject pool: Personality and sex matter
Witt, E. A., Donnellan, M. B., & Orlando, M. J. (2010) · Personality and Individual Differences · doi:10.1016/j.paid.2010.10.019 · Cited by 30 · see citing works
Perceptual Bias in Threat Distance
Cesario, J., & Navarrete, C. (2014) · Social Psychological and Personality Science · doi:10.1177/1948550613485605 · Cited by 18 · see citing works
Self-citationBehind the Headlines of Endostatin's Ups and Downs
Cohen, J. D. (1999) · Science · doi:10.1126/science.283.5406.1250 · Cited by 17 · see citing works
Katzko, M. W. (2006) · Review of General Psychology · doi:10.1037/1089-2680.10.3.210 · Cited by 17 · see citing works
What is all the hullabaloo about endostatin?
Rowe, P. M. (1999) · The Lancet · doi:10.1016/s0140-6736(05)76101-8 · Cited by 11 · see citing works
Introduction to the Special Section on Advancing Science
Spellman, B. A. (2013) · Perspectives on Psychological Science · doi:10.1177/1745691613493245 · Cited by 10 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Siddaway, A. P., Wood, A. M., & Hedges, L. V. (2019) · Annual Review of Psychology · doi:10.1146/annurev-psych-010418-102803 · Cited by 2,550 · see citing works
Investigating Variation in Replicability: A “Many Labs” Replication Project
Klein, R. A., Ratliff, K. A., Vianello, M., Adams, R. B., Bahník, Š., Bernstein, M. J., Bocian, K., Brandt, M. J., Brooks, B., Brumbaugh, C. C., et al. (2014) · Social Psychology · doi:10.1027/1864-9335/a000178 · Cited by 1,208 · see citing works
Replicability, Robustness, and Reproducibility in Psychological Science
Nosek, B. A., Hardwicke, T. E., Moshontz, H., Allard, A., Corker, K. S., Dreber, A., Fidler, F., Hilgard, J., Kline Struhl, M., Nuijten, M. B., et al. (2022) · Annual Review of Psychology · doi:10.1146/annurev-psych-020821-114157 · Cited by 770 · see citing works
The Value of Direct Replication
Simons, D. J. (2014) · Perspectives on Psychological Science · doi:10.1177/1745691613514755 · Cited by 630 · see citing works
Facts Are More Important Than Novelty: Replication in the Education Sciences
Makel, M. C., & Plucker, J. A. (2014) · Educational Researcher · doi:10.3102/0013189x14545513 · Cited by 609 · see citing works
Many Labs 3: Evaluating participant pool quality across the academic semester via replication
Ebersole, C. R., Atherton, O. E., Belanger, A. L., Skulborstad, H. M., Allen, J. M., Banks, J. B., Baranski, E., Bernstein, M. J., Bonfiglio, D. B. V., Boucher, L., et al. (2016) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2015.10.012 · Cited by 468 · see citing works
Contextual sensitivity in scientific reproducibility
Van Bavel, J. J., Mende-Siedlecki, P., Brady, W. J., & Reinero, D. A. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1521897113 · Cited by 398 · see citing works
Does Oxytocin Increase Trust in Humans? A Critical Review of Research
Nave, G., Camerer, C., & McCullough, M. (2015) · Perspectives on Psychological Science · doi:10.1177/1745691615600138 · Cited by 309 · see citing works
Pennycook, G., & Rand, D. G. (2022) · Nature Communications · doi:10.1038/s41467-022-30073-5 · Cited by 270 · see citing works
Understanding Priming Effects in Social Psychology: An Overview and Integration
Molden, D. C. (2014) · Social Cognition · doi:10.1521/soco.2014.32.supp.243 · Cited by 93 · see citing works
Encouraging pro-environmental behaviors through children-based appeals: a kin selection perspective
Palomo-Vélez, G., Buczny, J., & Van Vugt, M. (2020) · Sustainability · doi:10.3390/su12020748 · Cited by 23 · see citing works
Does Power Reduce Temporal Discounting? Commentary on Joshi and Fast (2013)
Zhang, M., & Smith, P. K. (2018) · Psychological Science · doi:10.1177/0956797617754219 · Cited by 13 · see citing works
Schiavone, S. R. (2018) · University of Kentucky Libraries · doi:10.13023/etd.2018.276 · Cited by 1 · see citing works
- Authors
- Cesario, J.
- Affiliations
- Michigan State University (US)
- Corresponding author
- Cesario, Joseph
- ORCID
- Cesario, Joseph
- Venue
- Perspectives on Psychological Science
- Publisher
- SAGE Publications
- Published
- 2014-01
- Volume
- 9
- Issue
- 1
- Pages
- 40-48
- ISSN
- 1745-6916 (ISSN-L)
- 1745-6924
- OpenAlex
- W2012428153
- Cited by
- 320
- References
- 57
- Citations by year
- 2026: 8 · 2025: 17 · 2024: 19 · 2023: 20 · 2022: 25 · 2021: 23 · 2020: 29 · 2019: 22 · 2018: 35 · 2017: 39 · 2016: 32 · 2015: 27 · 2014: 24
- FWCI
- 132.45
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Perspectives on Psychological Science (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Sociology and Political Science › Psychology of Social Influence
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Keywords
- behavioral priming, replication, replicability, direct replication, evolutionary psychology, experimental variability, replication failure, research practices
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
