Cited by 185Psychology Research and BibliometricsMeta-analysis and systematic reviewsMemory Processes and Influences

The DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.

Using Nonnaive Participants Can Reduce Effect Sizes

Chandler, Jesse, Paolacci, Gabriele, Peer, Eyal, Mueller, Pam, Ratliff, Kate A.·2015·Psychological Science

Open this paper in CoChat — free account

Abstract

Although researchers often assume their participants are naive to experimental materials, this is not always the case. We investigated how prior exposure to a task affects subsequent experimental results. Participants in this study completed the same set of 12 experimental tasks at two points in time, first as a part of the Many Labs replication project and again a few days, a week, or a month later. Effect sizes were markedly lower in the second wave than in the first. The reduction was most pronounced when participants were assigned to a different condition in the second wave. We discuss the methodological implications of these findings.

Integrity signals

Registry

Crossref · 10.1177/0956797615585115

Cites

29 of 29 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Statistical power analyses using G∗Power 3.1: tests for correlation and regression analyses
Faul, F., Erdfelder, E., Buchner, A., & Lang, A.-G. (2009) · Behavior Research Methods · doi:10.3758/brm.41.4.1149 · Cited by 37,615
The Framing of Decisions and the Psychology of Choice
Tversky, A., & Kahneman, D. (1981) · Science · doi:10.1126/science.7455683 · Cited by 18,497 · cited 1×
“In the gain-versus-loss-framing task (Tversky & Kahneman, 1981), participants read a disease scenario framed in terms of either gains or losses.”
Method
Measuring individual differences in implicit cognition: the implicit association test
Greenwald, A. G., McGhee, D. E., & Schwartz, J. L. K. (1998) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.74.6.1464 · Cited by 9,210
Instructional manipulation checks: Detecting satisficing to increase statistical power
Oppenheimer, D. M., Meyvis, T., & Davidenko, N. (2009) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2009.03.009 · Cited by 3,493
Source monitoring.
Johnson, M. K., Hashtroudi, S., & Lindsay, D. S. (1993) · Psychological Bulletin · doi:10.1037/0033-2909.114.1.3 · Cited by 3,392
Inside the Turk: Understanding Mechanical Turk as a Participant Pool
Paolacci, G., & Chandler, J. (2014) · Current Directions in Psychological Science · doi:10.1177/0963721414531598 · Cited by 2,136
One Hundred Years of Social Psychology Quantitatively Described
Richard, F. D., Bond, C. F., & Stokes-Zoota, J. J. (2003) · Review of General Psychology · doi:10.1037/1089-2680.7.4.331 · Cited by 1,558
Experimental methods: Between-subject and within-subject design
Charness, G., Gneezy, U., & Kuhn, M. A. (2012) · Journal of Economic Behavior & Organization · doi:10.1016/j.jebo.2011.08.009 · Cited by 1,469
Investigating Variation in Replicability: A “Many Labs” Replication Project
Klein, R. A., Ratliff, K. A., Vianello, M., Adams, R. B., Bahník, Š., Bernstein, M. J., Bocian, K., Brandt, M. J., Brooks, B., Brumbaugh, C. C., et al. (2014) · Social Psychology · doi:10.1027/1864-9335/a000178 · Cited by 1,208
Ambiguity Aversion and Comparative Ignorance
Fox, C. R., & Tversky, A. (1995) · The Quarterly Journal of Economics · doi:10.2307/2946693 · Cited by 1,072

Show all 29 references

Research works printed but not linked: 2

Hyman H. H., 1950, The teaching of contemporary affairs, 11
Lipsey M. W., 2001, Applied Social Research Methods Series: Vol. 49. Practical meta-analysis

Sorted by citations

Beyond the Turk: Alternative platforms for crowdsourcing behavioral research
Peer, E., Brandimarte, L., Samat, S., & Acquisti, A. (2017) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2017.01.006 · Cited by 2,957
TurkPrime.com: A versatile crowdsourcing data acquisition platform for the behavioral sciences
Litman, L., Robinson, J., & Abberbock, T. (2016) · Behavior Research Methods · doi:10.3758/s13428-016-0727-z · Cited by 2,122
Data quality of platforms and panels for online behavioral research
Peer, E., Rothschild, D., Gordon, A., Evernden, Z., & Damer, E. (2021) · Behavior Research Methods · doi:10.3758/s13428-021-01694-3 · Cited by 1,718
Understanding and managing the threat of common method bias: Detection, prevention and control
Kock, F., Berbekova, A., & Assaf, A. G. (2021) · Tourism Management · doi:10.1016/j.tourman.2021.104330 · Cited by 1,692
Conducting Clinical Research Using Crowdsourced Convenience Samples
Chandler, J., & Shapiro, D. (2016) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-021815-093623 · Cited by 1,072
Self-citation
MTurk Research: Review and Recommendations
Aguinis, H., Villamor, I., & Ramani, R. S. (2020) · Journal of Management · doi:10.1177/0149206320969787 · Cited by 1,098
Validating the demographic, political, psychological, and experimental results obtained from a new source of online survey respondents
Coppock, A., & McClellan, O. A. (2019) · Research & Politics · doi:10.1177/2053168018822174 · Cited by 987
Online panels in social science research: Expanding sampling methods beyond Mechanical Turk
Chandler, J., Rosenzweig, C., Moss, A. J., Robinson, J., & Litman, L. (2019) · Behavior Research Methods · doi:10.3758/s13428-019-01273-7 · Cited by 668
Investigating an alternate form of the cognitive reflection test
Thomson, K. S., & Oppenheimer, D. M. (2016) · Judgment and Decision Making · doi:10.1017/s1930297500007622 · Cited by 612
Crowdsourcing Consumer Research
Goodman, J. K., & Paolacci, G. (2017) · Journal of Consumer Research · doi:10.1093/jcr/ucx047 · Cited by 597
Demand Effects in Survey Experiments: An Empirical Assessment
MUMMOLO, J., & PETERSON, E. (2019) · American Political Science Review · doi:10.1017/s0003055418000837 · Cited by 548
Generalizing from Survey Experiments Conducted on Mechanical Turk: A Replication Approach
Coppock, A. (2019) · Political Science Research and Methods · doi:10.1017/psrm.2018.10 · Cited by 515
Attention by design: Using attention checks to detect inattentive respondents and improve data quality
Abbey, J. D., & Meloy, M. G. (2017) · Journal of Operations Management · doi:10.1016/j.jom.2017.06.001 · Cited by 522
The shape of and solutions to the MTurk quality crisis
Kennedy, R., Clifford, S., Burleigh, T., Waggoner, P. D., Jewell, R., & Winter, N. J. G. (2020) · Political Science Research and Methods · doi:10.1017/psrm.2020.6 · Cited by 506
Amazon mechanical Turk in organizational psychology: an evaluation and practical recommendations
Cheung, J. H., Burns, D. K., Sinclair, R. R., & Sliter, M. (2017) · Journal of Business and Psychology · doi:10.1007/s10869-016-9458-5 · Cited by 474
Crowdsourcing research: Data collection with Amazon’s Mechanical Turk
Sheehan, K. B. (2018) · Communication Monographs · doi:10.1080/03637751.2017.1342043 · Cited by 408
Cooperation, Fast and Slow
Rand, D. G. (2016) · Psychological Science · doi:10.1177/0956797616654455 · Cited by 461
Conducting interactive experiments online
Arechar, A. A., Gächter, S., & Molleman, L. (2017) · Experimental Economics · doi:10.1007/s10683-017-9527-2 · Cited by 370
Implications of the Credibility Revolution for Productivity, Creativity, and Progress
Vazire, S. (2018) · Perspectives on Psychological Science · doi:10.1177/1745691617751884 · Cited by 398
The Use of Online Panel Data in Management Research: A Review and Recommendations
Porter, C. O. L. H., Outlaw, R., Gale, J., & Cho, T. S. (2018) · Journal of Management · doi:10.1177/0149206318811569 · Cited by 368
MTurk Character Misrepresentation: Assessment and Solutions
Sharpe Wessling, K., Huber, J., & Netzer, O. (2017) · Journal of Consumer Research · doi:10.1093/jcr/ucx053 · Cited by 330
Systems perspective of Amazon mechanical Turk for organizational research: review and recommendations
Keith, M. G., Tay, L., & Harms, P. D. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.01359 · Cited by 319
The average laboratory samples a population of 7,300 Amazon Mechanical Turk workers
Stewart, N., Ungemach, C., Harris, A. J. L., Bartels, D. M., Newell, B. R., Paolacci, G., & Chandler, J. (2015) · Judgment and Decision Making · doi:10.1017/s1930297500005611 · Cited by 304
“Participants previously exposed to an experiment tend to show smaller effect sizes the second time (Chandler, Paolacci, Peer, Mueller, & Ratliff, 2015).”
Data Collection via Online Platforms: Challenges and Recommendations for Future Research
Newman, A., Bavik, Y. L., Mount, M. P., & Shao, B. (2020) · Applied Psychology · doi:10.1111/apps.12302 · Cited by 360
Lie for a Dime
Chandler, J., & Paolacci, G. (2017) · Social Psychological and Personality Science · doi:10.1177/1948550617698203 · Cited by 279
Crowdsourcing Samples in Cognitive Science
Stewart, N., Chandler, J., & Paolacci, G. (2017) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2017.06.007 · Cited by 233
Tapped out or barely tapped? Recommendations for how to harness the vast and largely unused potential of the Mechanical Turk participant pool
Robinson, J., Rosenzweig, C., Moss, A. J., & Litman, L. (2019) · PLoS ONE · doi:10.1371/journal.pone.0226394 · Cited by 203
The quality of data collected online: An investigation of careless responding in a crowdsourced sample
Brühlmann, F., Petralito, S., Aeschbach, L. F., & Opwis, K. (2020) · Methods in Psychology · doi:10.1016/j.metip.2020.100022 · Cited by 168
Comparing attentional disengagement between prolific and MTurk samples
Albert, D. A., & Smilek, D. (2023) · Scientific Reports · doi:10.1038/s41598-023-46048-5 · Cited by 166
Measuring the Prevalence of Problematic Respondent Behaviors among MTurk, Campus, and Community Participants
Necka, E. A., Cacioppo, S., Norman, G. J., & Cacioppo, J. T. (2016) · PLoS ONE · doi:10.1371/journal.pone.0157732 · Cited by 183
An Identity Perspective on Ethical Leadership to Explain Organizational Citizenship Behavior: The Interplay of Follower Moral Identity and Leader Group Prototypicality
Gerpott, F. H., Quaquebeke, N. V., Schlamp, S., & Voelpel, S. C. (2017) · Journal of Business Ethics · doi:10.1007/s10551-017-3625-0 · Cited by 156
Correcting misperceptions of exponential coronavirus growth increases support for social distancing
Lammers, J., Crusius, J., & Gast, A. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2006048117 · Cited by 137
Increasing altruistic and cooperative behaviour with simple moral nudges
Capraro, V., Jagfeld, G., Klein, R., Mul, M., & de Pol, I. V. (2019) · Scientific Reports · doi:10.1038/s41598-019-48094-4 · Cited by 135
Statistical Discrimination and the Rationalization of Stereotypes
Tilcsik, A. (2020) · American Sociological Review · doi:10.1177/0003122420969399 · Cited by 139
Intertemporal Differences Among MTurk Workers: Time-Based Sample Variations and Implications for Online Data Collection
Casey, L. S., Chandler, J., Levine, A. S., Proctor, A., & Strolovitch, D. Z. (2017) · SAGE Open · doi:10.1177/2158244017712774 · Cited by 132
Past-focused environmental comparisons promote proenvironmental outcomes for conservatives
Baldwin, M. W., & Lammers, J. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1610834113 · Cited by 119
Is it ethical to use Mechanical Turk for behavioral research? Relevant data from a representative survey of MTurk participants and wages
Moss, A. J., Rosenzweig, C., Robinson, J., Jaffe, S. N., & Litman, L. (2023) · Behavior Research Methods · doi:10.3758/s13428-022-02005-0 · Cited by 129
“Keep on Turkin’”?
Hulland, J. S., & Miller, J. (2018) · Journal of the Academy of Marketing Science · doi:10.1007/s11747-018-0587-4 · Cited by 107
Consumers care and firms should too: On the benefits of disclosing supplier monitoring activities
Duan, Y., Hofer, C., & Aloysius, J. A. (2020) · Journal of Operations Management · doi:10.1002/joom.1129 · Cited by 104
A Note on Increases in Inattentive Online Survey-Takers Since 2020
Ternovski, J., Orr, L., Kalla, J., & Aronow, P. M. (2022) · Journal of Quantitative Description Digital Media · doi:10.51685/jqd.2022.002 · Cited by 103
The Shape of and Solutions to the MTurk Quality CrisisPreprint
Kennedy, R., Clifford, S., Burleigh, T. J., Jewell, R., & Waggoner, P. (2018) · SSRN Electronic Journal · doi:10.2139/ssrn.3272468 · Cited by 99
Moving experimental psychology online: How to obtain high quality data when we can’t see our participants
Rodd, J. M. (2023) · Journal of Memory and Language · doi:10.1016/j.jml.2023.104472 · Cited by 117
Can we reduce facial biases? Persistent effects of facial trustworthiness on sentencing decisions
Jaeger, B., Todorov, A. T., Evans, A. M., & van Beest, I. (2020) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2020.104004 · Cited by 95
The Political Domain Appears Simpler to the Politically Extreme Than to Political Moderates
Lammers, J., Koch, A. S., Conway, P. J., & Brandt, M. J. (2016) · Social Psychological and Personality Science · doi:10.1177/1948550616678456 · Cited by 92
Sharing a work team with robots: The negative effect of robot co-workers on in-group identification with the work team
Savela, N., Kaakinen, M., Ellonen, N., & Oksanen, A. (2020) · Computers in Human Behavior · doi:10.1016/j.chb.2020.106585 · Cited by 96
Turking overtime: how participant characteristics and behavior vary over time and day on Amazon Mechanical Turk
Arechar, A. A., Kraft‐Todd, G., & Rand, D. G. (2017) · Journal of the Economic Science Association · doi:10.1007/s40881-017-0035-0 · Cited by 94
A beginner’s guide and best practices for using crowdsourcing platforms for survey research: The Case of Amazon Mechanical Turk (MTurk)
Çobanoğlu, C., Çavuşoğlu, M., & Türktarhan, G. (2021) · Journal of Global Business Insights · doi:10.5038/2640-6489.6.1.1177 · Cited by 98
Drivers of Data Quality in Advertising Research: Differences across MTurk and Professional Panel Samples
Berry, C., Kees, J., & Burton, S. (2022) · Journal of Advertising · doi:10.1080/00913367.2022.2079026 · Cited by 90
Participant Nonnaiveté and the reproducibility of cognitive psychology
Zwaan, R. A., Pecher, D., Paolacci, G., Bouwmeester, S., Verkoeijen, P. P. J. L., Katinka Dijkstra, B., & Zeelenberg, R. (2017) · Psychonomic Bulletin & Review · doi:10.3758/s13423-017-1348-y · Cited by 89
Wise deliberation sustains cooperation
Grossmann, I., Brienza, J. P., & Bobocel, D. R. (2017) · Nature Human Behaviour · doi:10.1038/s41562-017-0061 · Cited by 87
Rethinking spontaneous giving: Extreme time pressure and ego-depletion favor self-regarding reactions
Capraro, V., & Cococcioni, G. (2016) · Scientific Reports · doi:10.1038/srep27219 · Cited by 90
Time pressure and honesty in a deception game
Capraro, V., Schulz, J., & Rand, D. G. (2019) · Journal of Behavioral and Experimental Economics · doi:10.1016/j.socec.2019.01.007 · Cited by 77
Moving outside the lab: The viability of conducting sensorimotor learning studies online
Tsay, J. S., Lee, A., Ivry, R. B., & Avraham, G. (2021) · Neurons Behavior Data analysis and Theory · doi:10.51628/001c.26985 · Cited by 73
Do imposters threaten data quality? An examination of worker misrepresentation and downstream consequences in Amazon's Mechanical Turk workforce
Kan, I. P., & Drummey, A. B. (2018) · Computers in Human Behavior · doi:10.1016/j.chb.2018.02.005 · Cited by 62
Deliberation favours social efficiency by making people disregard their relative shares: evidence from USA and India
Capraro, V., Corgnet, B., Espín, A. M., & Hérnan-Gonzalez, R. (2017) · Royal Society Open Science · doi:10.1098/rsos.160605 · Cited by 55
“Put Yourself in Their Shoes”: Testing Empathy's Ability to Motivate Cosmopolitan Behavior
Faulkner, N. (2017) · Political Psychology · doi:10.1111/pops.12411 · Cited by 56
Exposing omitted moderators: Explaining why effect sizes differ in the social sciences
Krefeld-Schwalb, A., Sugerman, E. R., & Johnson, E. J. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2306281121 · Cited by 60
Conducting Online Behavioral Research Using Crowdsourcing Services in Japan
Majima, Y., Nishiyama, K., Nishihara, A., & Hata, R. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.00378 · Cited by 51
Foundational Tests of the Need-Support Model
Vaughn, L. A. (2017) · Personality and Social Psychology Bulletin · doi:10.1177/0146167216684132 · Cited by 49
Improving the Replicability of Psychological Science Through Pedagogy
Hawkins, R. X. D., Smith, E. N., Au, C., Arias, J. M., Catapano, R., Hermann, E., Keil, M., Lampinen, A. K., Raposo, S., Reynolds, J., et al. (2018) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245917740427 · Cited by 46
Media effects on the perceptions of robots
Savela, N., Turja, T., Latikka, R., & Oksanen, A. (2021) · Human Behavior and Emerging Technologies · doi:10.1002/hbe2.296 · Cited by 47
Conducting Web-Based Experiments for Numerical Cognition Research
Kochari, A. R. (2019) · Journal of Cognition · doi:10.5334/joc.85 · Cited by 40
Caution, Preprint! Brief Explanations Allow Nonscientists to Differentiate Between Preprints and Peer-Reviewed Journal Articles
Wingen, T., Berkessel, J. B., & Dohle, S. (2022) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459211070559 · Cited by 40
Biomicroscopy and scanning electron microscopy of early opacities in the aging human lens.
Vrensen, G. F. J. M., & Willekens, B. (1990) · PubMed · Cited by 38
On the priming of risk preferences: The role of fear and general affect
Alempaki, D., Starmer, C., & Tufano, F. (2019) · Journal of Economic Psychology · doi:10.1016/j.joep.2018.12.011 · Cited by 38
Creativity on Paid Crowdsourcing Platforms
Oppenlaender, J., Milland, K., Visuri, A., Ipeirotis, P. G., & Hosio, S. J. (2020) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376677 · Cited by 42
Analytic cognitive style predicts paranormal explanations of anomalous experiences but not the experiences themselves: Implications for cognitive theories of delusions
Ross, R. M., Hartig, B., & McKay, R. (2016) · Journal of Behavior Therapy and Experimental Psychiatry · doi:10.1016/j.jbtep.2016.08.018 · Cited by 39
Emotional reactions to robot colleagues in a role-playing experiment
Savela, N., Oksanen, A., Pellert, M., & García, D. (2021) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2021.102361 · Cited by 37
Snooping on Mobile Phones: Prevalence and Trends
Marques, D., Muslukhov, I., Guerreiro, T., Beznosov, K., & Carriço, L. M. (2016) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.3264715 · Cited by 30
Tighter nets for smaller fishes? Mapping the development of statistical practices in consumer research between 2008 and 2020
Krefeld-Schwalb, A., & Scheibehenne, B. (2022) · Marketing Letters · doi:10.1007/s11002-022-09662-3 · Cited by 28
Can You Really Study an Army on the Internet? Comparing How Status Tasks Perform in the Laboratory and Online Settings
Manago, B. N., Mize, T. D., & Doan, L. (2021) · Sociological Methodology · doi:10.1177/00811750211014242 · Cited by 28
Turkers of the World Unite: Multilevel In-Group Bias Among Crowdworkers on Amazon Mechanical Turk
Almaatouq, A., Krafft, P. M., Dunham, Y. C., Rand, D. G., & Pentland, A. (2019) · Social Psychological and Personality Science · doi:10.1177/1948550619837002 · Cited by 26
Not so terrifying after all? A set of failed replications of the mortality salience effects of Terror Management Theory
Treger, S., Benau, E. M., & Timko, C. A. (2023) · PLOS ONE · doi:10.1371/journal.pone.0285267 · Cited by 25
Gain-loss framing effects in dilemmas of trust and reciprocity
Evans, A. M., & van Beest, I. (2017) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2017.06.012 · Cited by 32
Reducing the number of non-naïve participants in Mechanical Turk samples
Meyers, E. A., Walker, A. C., Fugelsang, J. A., & Koehler, D. J. (2020) · Methods in Psychology · doi:10.1016/j.metip.2020.100032 · Cited by 29
Crowdsourcing in Cognitive and Systems Neuroscience
Johnson, B. P., Dayan, E., Censor, N., & Cohen, L. G. (2021) · The Neuroscientist · doi:10.1177/10738584211017018 · Cited by 23
Individuals fearing positivity do not perceive positive affect treatments as strong fits: A novel experimental finding and replication
Bryant, J. S., Gallagher, M. R., Collins, A. C., & Winer, E. S. (2022) · Journal of Behavior Therapy and Experimental Psychiatry · doi:10.1016/j.jbtep.2022.101830 · Cited by 22
A View from the Cockpit: Exploring Pilot Reactions to Attacks on Avionic Systems
Smith, M., Strohmeier, M., Harman, J., Lenders, V., & Martinović, I. (2020) · Network and Distributed System Security Symposium · doi:10.14722/ndss.2020.23022 · Cited by 20
Stressing the Advantages of Female Leadership Can Place Women at a Disadvantage
Lammers, J., & Gast, A. (2017) · Social Psychology · doi:10.1027/1864-9335/a000292 · Cited by 21
Upon Repeated Reflection: Consequences of Frequent Exposure to the Cognitive Reflection Test for Mechanical Turk Participants
Woike, J. K. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.02646 · Cited by 23
Ideas are cheap: When and why adults value labor over ideas.
Burgmer, P., Forstmann, M., & Stavrova, O. (2018) · Journal of Experimental Psychology General · doi:10.1037/xge0000473 · Cited by 20
Ambiguity and expectation-neglect in dilemmas of interpersonal trust
Evans, A. M., & Krueger, J. I. (2017) · Judgment and Decision Making · doi:10.1017/s1930297500006719 · Cited by 15
Dispersion matters: Diagnostics and control data computer simulation in Concealed Information Test studies
Lukács, G., & Specker, E. (2020) · PLoS ONE · doi:10.1371/journal.pone.0240259 · Cited by 16
Racial attention deficit
Levine, S. S., Reypens, C., & Stark, D. (2021) · Science Advances · doi:10.1126/sciadv.abg9508 · Cited by 17
When the agency wants too much: Experimental evidence on unfair audits and tax compliance
Lancee, B. S., Rossel, L., & Kasper, M. (2023) · Journal of Economic Behavior & Organization · doi:10.1016/j.jebo.2023.08.011 · Cited by 24
Use of Crowdsourced Online Surveys to Study the Impact of Architectural and Design Choices on Wellbeing
Altaf, B., Bianchi, E., Douglas, I. P., Douglas, K., Byers, B. S., Paredes, P. E., Ardoin, N. M., Markus, H. R., Murnane, E. L., Bencharit, L. Z., et al. (2022) · Frontiers in Sustainable Cities · doi:10.3389/frsc.2022.780376 · Cited by 13
The Company She Seeks: How the Prismatic Effects of Ties to High-Status Network Contacts Can Reduce Status for Women in Groups
Yu, S., & Shea, C. T. (2023) · Organization Science · doi:10.1287/orsc.2020.14640 · Cited by 14
Distinguishing Between Need Support and Regulatory Focus with LIWC
Vaughn, L. A. (2019) · Collabra Psychology · doi:10.1525/collabra.185 · Cited by 15
Why language matters: Shaping public risk tolerance during deterrence crises
Berejikian, J., & Zwald, Z. (2020) · Contemporary Security Policy · doi:10.1080/13523260.2020.1729496 · Cited by 15
Conducting Interactive Experiments OnlinePreprint
Arechar, A. A., Gächter, S., & Molleman, L. (2016) · SSRN Electronic Journal · doi:10.2139/ssrn.2884409 · Cited by 13
Comprehension in economic games
Koppel, L., Andersson, D., Johannesson, M., Strømland, E., & Tinghög, G. (2025) · Journal of Economic Behavior & Organization · doi:10.1016/j.jebo.2025.107039 · Cited by 17
Replications with MTurkers who are naïve versus experienced with academic studies: A comment on Connors, Khamitov, Moroz, Campbell, and Henderson (2015)
DeVoe, S. E., & House, J. (2016) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2015.11.004 · Cited by 12
Appreciative Joy Meditation Enhances Acceptance of Unfair Offer in Ultimatum Game
Ng, G. T. T., Lai, D. C. K., Zeng, X., & Oei, T. P. S. (2019) · Mindfulness · doi:10.1007/s12671-019-01139-1 · Cited by 13
Cold Side-Effect Effect: Affect Does Not Mediate the Influence of Moral Considerations in Intentionality Judgments
Díaz, R., Viciana, H., & Gomila, A. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.00295 · Cited by 12
Why People Hate Congress but Love Their Own Congressperson: An Information Processing Explanation
Lammers, J., Pauels, E., Fleischmann, A., & Galinsky, A. D. (2021) · Personality and Social Psychology Bulletin · doi:10.1177/01461672211002336 · Cited by 12
Caution, preprint! Brief explanations allow non-scientists to differentiate between preprints and peer-reviewed journal articlesPreprint
Wingen, T., Berkessel, J. B., & Dohle, S. (2021) · doi:10.31219/osf.io/7n3mj · Cited by 9
Systematic review and meta-analysis of educational approaches to reduce cognitive biases among students
Swaryandini, G., Graham, J., Griffith, S., Grilo, V., Ruzzante, F., Zhang, X., Yeung, S. K., Mangiarulo, M., Basarkod, G., Ng, C., et al. (2025) · Nature Human Behaviour · doi:10.1038/s41562-025-02253-y · Cited by 11
Prolific: Crowdsourcing Academic Online Research
Péer, E. (2024) · Cambridge University Press eBooks · doi:10.1017/9781009000796.005 · Cited by 10
The Daily Lives of Crowdsourced U.S. Respondents: A Time Use Comparison of MTurk, Prolific, and ATUS
Rinderknecht, R. G., Doan, L., & Sayer, L. C. (2025) · Sociological Methodology · doi:10.1177/00811750241312226 · Cited by 13
Fast, Cheap, and Unethical? The Interplay of Morality and Methodology in Crowdsourced Survey Research
Haug, M. C. (2017) · Review of Philosophy and Psychology · doi:10.1007/s13164-017-0374-z · Cited by 10

Authors: Chandler, J., Paolacci, G., Peer, E., Mueller, P., & Ratliff, K. A.
Affiliations: University of Michigan (US)
Erasmus University Rotterdam (NL)
Graduate School of Business Administration, Bar-Ilan University
Princeton University (US)
University of Florida (US)
ORCID: Chandler, Jesse
Paolacci, Gabriele
Ratliff, Kate A.
Venue: Psychological Science
Publisher: SAGE Publications
Published: 2015-06-10
Volume: 26
Issue: 7
Pages: 1131-1139
ISSN: 0956-7976 (ISSN-L)
1467-9280
DOI: 10.1177/0956797615585115
OpenAlex: W2162475434
Cited by: 185
References: 29
Citations by year: 2026: 6 · 2025: 20 · 2024: 10 · 2023: 15 · 2022: 10 · 2021: 23 · 2020: 20 · 2019: 21 · 2018: 14 · 2017: 27 · 2016: 15 · 2015: 3
FWCI: 29.35
Type: journal article
Language: en
Open access: closed
Licence: journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
Locations: Psychological Science (publishedVersion)
PubMed (publishedVersion)
EUR Research Repository (Erasmus University Rotterdam) (publishedVersion)
EUR Research Repository (Erasmus University Rotterdam) (submittedVersion)
Topics: Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Life Sciences › Neuroscience › Cognitive Neuroscience › Memory Processes and Influences
Keywords: replication, task familiarity, within-participant design
MeSH terms: Adult, Female, Humans, Male, Patient Participation, Patient Selection

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Using Nonnaive Participants Can Reduce Effect Sizes

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks