The DOI matches its Crossref record; 59 of the 59 references it cites were found in a registry or an index.
The viability of crowdsourcing for survey research
Open accessAbstract
Online contract labor portals (i.e., crowdsourcing) have recently emerged as attractive alternatives to university participant pools for the purposes of collecting survey data for behavioral research. However, prior research has not provided a thorough examination of crowdsourced data for organizational psychology research. We found that, as compared with a traditional university participant pool, crowdsourcing respondents were older, were more ethnically diverse, and had more work experience. Additionally, the reliability of the data from the crowdsourcing sample was as good as or better than the corresponding university sample. Moreover, measurement invariance generally held across these groups. We conclude that the use of these labor portals is an efficient and appropriate alternative to a university participant pool, despite small differences in personality and socially desirable responding across the samples. The risks and advantages of crowdsourcing are outlined, and an overview of practical and ethical guidelines is provided.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Statistical Power Analysis for the Behavioral Sciences
Muller, K. (1989) · Technometrics · doi:10.1080/00401706.1989.10488618 · Cited by 99,167
Statistical power analysis for the behavioral sciences
(1990) · Computers, Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,269
The weirdest people in the world?
Henrich, J., Heine, S. J., & Norenzayan, A. (2010) · Behavioral and Brain Sciences · doi:10.1017/s0140525x0999152x · Cited by 12,641
A new scale of social desirability independent of psychopathology.
Crowne, D. P., & Marlowe, D. (1960) · Journal of Consulting Psychology · doi:10.1037/h0047358 · Cited by 8,686
(2000) · Cited by 4,106
WEIRD languages have misled us, too
Majid, A., & Levinson, S. C. (2010) · Behavioral and Brain Sciences · doi:10.1017/s0140525x1000018x · Cited by 4,076
Henrich, J., Heine, S. J., & Norenzayan, A. (2010) · Nature · doi:10.1038/466029a · Cited by 3,441
(1999) · Medical Entomology and Zoology · Cited by 3,151
ESTIMATION OF LATENT ABILITY USING A RESPONSE PATTERN OF GRADED SCORES1
Samejima, F. (1968) · ETS Research Bulletin Series · doi:10.1002/j.2333-8504.1968.tb00153.x · Cited by 2,587
Personality Psychology in Europe
Vera-Bermudez, J., Raad, D. B., de Vries, J., García, A. M. P., Sanchez-Elvita, A., & Heck, G. V. (1997) · University of Groningen research database (University of Groningen / Centre for Information Technology) · Cited by 2,462
Research works printed but not linked: 2
- Cole, F., Sanik, K., DeCarlo, D., Finkelstein, A., Funkhouser, T., Rusinkiewicz, S., et al. (2009). How well do line drawings depict shape? ACM Transactions on Graphics, 28, 1–9. doi: 10.1145/1531326.1531334 · doi:10.1145/1531326.1531334
- Turk and rescue. (2007). Economist, 384(8547), 97.
Cited by
Inside the Turk: Understanding Mechanical Turk as a Participant Pool
Paolacci, G., & Chandler, J. (2014) · Current Directions in Psychological Science · doi:10.1177/0963721414531598 · Cited by 2,136
Data quality of platforms and panels for online behavioral research
Peer, E., Rothschild, D., Gordon, A., Evernden, Z., & Damer, E. (2021) · Behavior Research Methods · doi:10.3758/s13428-021-01694-3 · Cited by 1,718
Conducting Clinical Research Using Crowdsourced Convenience Samples
Chandler, J., & Shapiro, D. (2016) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-021815-093623 · Cited by 1,072
An MTurk Crisis? Shifts in Data Quality and the Impact on Study Results
Chmielewski, M., & Kucker, S. C. (2020) · Social Psychological and Personality Science · doi:10.1177/1948550619875149 · Cited by 1,027
Coppock, A., & McClellan, O. A. (2019) · Research & Politics · doi:10.1177/2053168018822174 · Cited by 987
Metadata
- Authors
- Behrend, T. S., Sharek, D. J., Meade, A. W., & Wiebe, E. N.
- Venue
- Behavior Research Methods
- Published
- 2011-03-25
- Volume
- 43
- Issue
- 3
- Pages
- 800-813
- ISSN
- 1554-351X (ISSN-L)
- 1554-3528
- OpenAlex
- W2086676398
- Cited by
- 1,018
- References
- 59
- FWCI
- 64.03
- Type
- journal article
- Language
- en
- Open access
- hybrid
- Licence
- www.springer.com/tdm (tdm)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The viability of crowdsourcing for survey research. https://app.cochat.ai/papers/10.3758/s13428-011-0081-0
- The paper
- Behrend, T. S., Sharek, D. J., Meade, A. W., & Wiebe, E. N. (2011). The viability of crowdsourcing for survey research. Behavior Research Methods. https://doi.org/10.3758/s13428-011-0081-0
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
