The DOI matches its Crossref record; 65 of the 65 references it cites were found in a registry or an index.
Testing the Waters: Behavior across Participant Pools
Abstract
We leverage a large-scale incentivized survey eliciting behaviors from (almost) an entire undergraduate university student population, a representative sample of the US population, and Amazon Mechanical Turk (MTurk) to address concerns about the external validity of experiments with student participants. Behavior in the student population offers bounds on behaviors in other populations, and correlations between behaviors are similar across samples. Furthermore, non-student samples exhibit higher levels of noise. Adding historical lab participation data, we find a small set of attributes over which lab participants differ from non-lab participants. An additional set of lab experiments shows no evidence of observer effects. (JEL C83, D90, D91)
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Measuring individual differences in implicit cognition: the implicit association test
Greenwald, A. G., McGhee, D. E., & Schwartz, J. L. K. (1998) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.74.6.1464 · Cited by 9,210
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515
Understanding and using the Implicit Association Test: I. An improved scoring algorithm.
Greenwald, A. G., Nosek, B. A., & Banaji, M. R. (2003) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.85.2.197 · Cited by 5,815
Gender Differences in Preferences
Croson, R., & Gneezy, U. (2009) · Journal of Economic Literature · doi:10.1257/jel.47.2.448 · Cited by 5,694
Cognitive Reflection and Decision Making
Frederick, S. (2005) · Journal of Economic Perspectives · doi:10.1257/089533005775196732 · Cited by 5,256
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702
Orne, M. T. (1962) · American Psychologist · doi:10.1037/h0043424 · Cited by 4,041
Evaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical Turk
Berinsky, A. J., Huber, G. A., & Lenz, G. S. (2012) · Political Analysis · doi:10.1093/pan/mpr057 · Cited by 4,008
INDIVIDUAL RISK ATTITUDES: MEASUREMENT, DETERMINANTS, AND BEHAVIORAL CONSEQUENCES
Dohmen, T., Falk, A., Huffman, D., Sunde, U., Schupp, J., & Wagner, G. G. (2011) · Journal of the European Economic Association · doi:10.1111/j.1542-4774.2011.01015.x · Cited by 3,896
Cited by
A Single-Item Visual Analogue Scale (VAS) Measure for Assessing Depression among College Students
Huang, Z., Kohler, I. V., & Kämpfen, F. (2020) · Community Mental Health Journal · doi:10.1007/s10597-019-00469-7 · Cited by 51
Looming Large or Seeming Small? Attitudes Towards Losses in a Representative Sample
Chapman, J., Snowberg, E., Wang, S. W., & Camerer, C. (2025) · Review of Economic Studies · doi:10.1093/restud/rdae093 · Cited by 10
Self-citationKang, B., & Wu, S. (2023) · Journal of Experimental Criminology · doi:10.1007/s11292-022-09512-2 · Cited by 6
Metadata
- Authors
- Snowberg, E., & Yariv, L.
- Venue
- American Economic Review
- Publisher
- American Economic Association
- Published
- 2021-02-01
- Volume
- 111
- Issue
- 2
- Pages
- 687-719
- ISSN
- 0002-8282 (ISSN-L)
- OpenAlex
- W2887346961
- Cited by
- 273
- References
- 65
- FWCI
- 58.39
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Testing the Waters: Behavior across Participant Pools. https://app.cochat.ai/papers/10.1257/aer.20181065
- The paper
- Snowberg, E., & Yariv, L. (2021). Testing the Waters: Behavior across Participant Pools. American Economic Review. https://doi.org/10.1257/aer.20181065
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.