The DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Using Nonnaive Participants Can Reduce Effect Sizes
Abstract
Although researchers often assume their participants are naive to experimental materials, this is not always the case. We investigated how prior exposure to a task affects subsequent experimental results. Participants in this study completed the same set of 12 experimental tasks at two points in time, first as a part of the Many Labs replication project and again a few days, a week, or a month later. Effect sizes were markedly lower in the second wave than in the first. The reduction was most pronounced when participants were assigned to a different condition in the second wave. We discuss the methodological implications of these findings.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Repeated exposure to stimuli can blunt effect sizes in various experimental paradigms [103].”
From Not so terrifying after all? A set of failed replications of the mortality salience effects of Terror Management Theory (2023) · cites this paper as [103]
In the citing paper
“The high rate of non-naivety among MTurk participants has recently been shown to have the potential to significantly reduce the effect sizes of known research findings (Chandler et al., 2015).”
From Beyond the Turk: Alternative platforms for crowdsourcing behavioral research (2017) · cites this paper as Chandler et al., 2015
Citations
Cites
Statistical power analyses using G∗Power 3.1: tests for correlation and regression analyses
Faul, F., Erdfelder, E., Buchner, A., & Lang, A.-G. (2009) · Behavior Research Methods · doi:10.3758/brm.41.4.1149 · Cited by 37,615
The Framing of Decisions and the Psychology of Choice
Tversky, A., & Kahneman, D. (1981) · Science · doi:10.1126/science.7455683 · Cited by 18,497 · cited 1×
“In the gain-versus-loss-framing task (Tversky & Kahneman, 1981), participants read a disease scenario framed in terms of either gains or losses.”
MethodMeasuring individual differences in implicit cognition: the implicit association test
Greenwald, A. G., McGhee, D. E., & Schwartz, J. L. K. (1998) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.74.6.1464 · Cited by 9,210
Instructional manipulation checks: Detecting satisficing to increase statistical power
Oppenheimer, D. M., Meyvis, T., & Davidenko, N. (2009) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2009.03.009 · Cited by 3,493
Johnson, M. K., Hashtroudi, S., & Lindsay, D. S. (1993) · Psychological Bulletin · doi:10.1037/0033-2909.114.1.3 · Cited by 3,392
Inside the Turk: Understanding Mechanical Turk as a Participant Pool
Paolacci, G., & Chandler, J. (2014) · Current Directions in Psychological Science · doi:10.1177/0963721414531598 · Cited by 2,136
One Hundred Years of Social Psychology Quantitatively Described
Richard, F. D., Bond, C. F., & Stokes-Zoota, J. J. (2003) · Review of General Psychology · doi:10.1037/1089-2680.7.4.331 · Cited by 1,558
Experimental methods: Between-subject and within-subject design
Charness, G., Gneezy, U., & Kuhn, M. A. (2012) · Journal of Economic Behavior & Organization · doi:10.1016/j.jebo.2011.08.009 · Cited by 1,469
Investigating Variation in Replicability: A “Many Labs” Replication Project
Klein, R. A., Ratliff, K. A., Vianello, M., Adams, R. B., Bahník, Š., Bernstein, M. J., Bocian, K., Brandt, M. J., Brooks, B., Brumbaugh, C. C., et al. (2014) · Social Psychology · doi:10.1027/1864-9335/a000178 · Cited by 1,208
Ambiguity Aversion and Comparative Ignorance
Fox, C. R., & Tversky, A. (1995) · The Quarterly Journal of Economics · doi:10.2307/2946693 · Cited by 1,072
Chandler, J., Mueller, P., & Paolacci, G. (2014) · Behavior Research Methods · doi:10.3758/s13428-013-0365-7 · Cited by 950
Intuitive Cooperation and the Social Heuristics Hypothesis: Evidence from 15 Time Constraint Studies
Rand, D. G., Peysakhovich, A., Kraft‐Todd, G., Newman, G. E., Wurzbacher, O., Nowak, M. A., & Greene, J. D. (2013) · Nature Communications · doi:10.1038/ncomms4677 · Cited by 772
Measures of Anchoring in Estimation Tasks
Jacowitz, K. E., & Kahneman, D. J. (1995) · Personality and Social Psychology Bulletin · doi:10.1177/01461672952111004 · Cited by 753
Within-subjects designs: To use or not to use?
Greenwald, A. G. (1976) · Psychological Bulletin · doi:10.1037/0033-2909.83.2.314 · Cited by 529
On the self-erasing nature of errors of prediction.
Sherman, S. J. (1980) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.39.2.211 · Cited by 525
Health of the Implicit Association Test at age 3.
Greenwald, A., & Nosek, B. A. (2001) · Experimental Psychology · doi:10.1026//0949-3946.48.2.85 · Cited by 488
Weber, S. J., & Cook, T. D. (1972) · Psychological Bulletin · doi:10.1037/h0032351 · Cited by 385
Response Scales: Effects of Category Range on Reported Behavior and Comparative Judgments
Schwarz, N., Hippler, H., Deutsch, B., & Strack, F. (1985) · Public Opinion Quarterly · doi:10.1086/268936 · Cited by 372
How to show that 9 > 221: Collect judgments in a between-subjects design.
Birnbaum, M. H. (1999) · Psychological Methods · doi:10.1037/1082-989x.4.3.243 · Cited by 230
Elaboration enhances the imagined contact effect
Husnu, S., & Crisp, R. J. (2010) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2010.05.014 · Cited by 219
Prospect theory's reflection hypothesis: A critical examination
Hershey, J. C., & Schoemaker, P. J. H. (1980) · Organizational Behavior and Human Performance · doi:10.1016/0030-5073(80)90037-9 · Cited by 215
Experiments in Wording Questions: II
Rugg, D. (1941) · Public Opinion Quarterly · doi:10.1086/265467 · Cited by 153
Professional respondents in nonprobability online panels
Hillygus, D. S., Jackson, N., & Young, M. (2014) · Wiley eBooks · doi:10.1002/9781118763520.ch10 · Cited by 138
Surveys on surveys: limitations and potentialities
Goyder, J. (1986) · Public Opinion Quarterly · doi:10.1086/268957 · Cited by 123
Prestige, Suggestion, and Attitudes
Lorge, I. D., & Curtiss, C. (1936) · The Journal of Social Psychology · doi:10.1080/00224545.1936.9919891 · Cited by 118
Attitudes Over Time: The Psychology of Panel Conditioning
Sturgis, P., Allum, N., & Brunton‐Smith, I. (2009) · Wiley eBooks · doi:10.1002/9780470743874.ch7 · Cited by 110
The retrospective gambler’s fallacy: Unlikely events, constructing the past, and multiple universes
Oppenheimer, D. M., & Monin, B. (2009) · Judgment and Decision Making · doi:10.1017/s1930297500001170 · Cited by 85
The role of prior knowledge in explaining hypothetical events
Hirt, E. R., & Sherman, S. J. (1985) · Journal of Experimental Social Psychology · doi:10.1016/0022-1031(85)90023-x · Cited by 56
Long-run learning in games of cooperation
Mason, W., Suri, S., & Watts, D. J. (2014) · ACM Conference on Economics and Computation (EC) · doi:10.1145/2600057.2602892 · Cited by 14
Flagged first, then by citations
Research works printed but not linked: 2
- Hyman H. H., 1950, The teaching of contemporary affairs, 11
- Lipsey M. W., 2001, Applied Social Research Methods Series: Vol. 49. Practical meta-analysis
Cited by
Sorted by citations
Beyond the Turk: Alternative platforms for crowdsourcing behavioral research
Peer, E., Brandimarte, L., Samat, S., & Acquisti, A. (2017) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2017.01.006 · Cited by 2,957
TurkPrime.com: A versatile crowdsourcing data acquisition platform for the behavioral sciences
Litman, L., Robinson, J., & Abberbock, T. (2016) · Behavior Research Methods · doi:10.3758/s13428-016-0727-z · Cited by 2,122
Data quality of platforms and panels for online behavioral research
Peer, E., Rothschild, D., Gordon, A., Evernden, Z., & Damer, E. (2021) · Behavior Research Methods · doi:10.3758/s13428-021-01694-3 · Cited by 1,718
Understanding and managing the threat of common method bias: Detection, prevention and control
Kock, F., Berbekova, A., & Assaf, A. G. (2021) · Tourism Management · doi:10.1016/j.tourman.2021.104330 · Cited by 1,692
Conducting Clinical Research Using Crowdsourced Convenience Samples
Chandler, J., & Shapiro, D. (2016) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-021815-093623 · Cited by 1,072
Self-citation
Metadata
- Authors
- Chandler, J., Paolacci, G., Peer, E., Mueller, P., & Ratliff, K. A.
- Affiliations
- University of Michigan (US)
- Erasmus University Rotterdam (NL)
- Graduate School of Business Administration, Bar-Ilan University
- Princeton University (US)
- University of Florida (US)
- Venue
- Psychological Science
- Publisher
- SAGE Publications
- Published
- 2015-06-10
- Volume
- 26
- Issue
- 7
- Pages
- 1131-1139
- ISSN
- 0956-7976 (ISSN-L)
- 1467-9280
- OpenAlex
- W2162475434
- Cited by
- 185
- References
- 29
- Citations by year
- 2026: 6 · 2025: 20 · 2024: 10 · 2023: 15 · 2022: 10 · 2021: 23 · 2020: 20 · 2019: 21 · 2018: 14 · 2017: 27 · 2016: 15 · 2015: 3
- FWCI
- 29.35
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Science (publishedVersion)
- PubMed (publishedVersion)
- EUR Research Repository (Erasmus University Rotterdam) (publishedVersion)
- EUR Research Repository (Erasmus University Rotterdam) (submittedVersion)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory Processes and Influences
- Keywords
- replication, task familiarity, within-participant design
- MeSH terms
- Adult, Female, Humans, Male, Patient Participation, Patient Selection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
