Using Mechanical Turk to recruit participants for internet intervention research: experience from recruitment for four trials targeting hazardous alcohol consumption
Open accessThe DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Mechanical Turk (MTurk) is an online portal operated by Amazon where 'requesters' (individuals or businesses) can submit jobs for 'workers.' MTurk is used extensively by academics as a quick and cheap means of collecting questionnaire data, including information on alcohol consumption, from a diverse sample of participants. We tested the feasibility of recruiting for alcohol Internet intervention trials through MTurk. METHODS: Participants, 18 years or older, who drank at least weekly were recruited for four intervention trials (combined sample size, N = 11,107). The same basic recruitment strategy was employed for each trial - invite participants to complete a survey about alcohol consumption (less than 15 min in length, US$1.50 payment), identify eligible participants who drank in a hazardous fashion, invite those eligible to complete a follow-up survey ($10 payment), randomize participants to be sent or not sent information to access an online intervention for hazardous alcohol use. Procedures where put in place to optimize the chances that participants could only complete the baseline survey once. RESULTS: There was a substantially slower rate of recruitment by the fourth trial compared to the earlier trials. Demographic characteristics also varied across trials (age, sex, employment and marital status). Patterns of alcohol consumption, while displaying some differences, did not appear to vary in a linear fashion between trials. CONCLUSIONS: It is possible to recruit large (but not inexhaustible) numbers of people who drink in a hazardous fashion. Issues for online intervention research when employing this sample are discussed.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reputation as a sufficient condition for data quality on Amazon Mechanical Turk
Peer, E., Vosgerau, J., & Acquisti, A. (2014) · Behavior Research Methods · doi:10.3758/s13428-013-0434-y · Cited by 1,618 · cited 9×
“Advantageous details of the MTurk platform when conducting multiple recruitments When conducting multiple studies from the same account, the MTurk platform allows the requestor (the customer hiring the workers) to assign workers who have completed a HIT a code (a ‘qualification’; specific to one requestor’s account).”
BackgroundInfluentialUsing Mechanical Turk to Study Clinical Populations
Shapiro, D. N., Chandler, J., & Mueller, P. A. (2013) · Clinical Psychological Science · doi:10.1177/2167702612469015 · Cited by 1,139
Conducting Clinical Research Using Crowdsourced Convenience Samples
Chandler, J., & Shapiro, D. (2016) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-021815-093623 · Cited by 1,072
Riper, H., Blankers, M., Hadiwijaya, H., Cunningham, J., Clarke, S., Wiers, R., Ebert, D., & Cuijpers, P. (2014) · PLoS ONE · doi:10.1371/journal.pone.0099912 · Cited by 265 · cited 1×
“We were also able to recruit participants more quickly using MTurk than in our other trials using alternate advertising methods (although it is important to note that other researchers have been able to obtain study samples quickly using Facebook or Google AdWords and that the recruitment on MTurk was substantially sl…”
ResultSelf-citationKim, H. S., & Hodgins, D. C. (2017) · Psychology of Addictive Behaviors · doi:10.1037/adb0000219 · Cited by 206
A randomized controlled trial of an internet‐based intervention for alcohol abusers
Cunningham, J. A., Wild, T. C., Cordingley, J., Van Mierlo, T., & Humphreys, K. (2009) · Addiction · doi:10.1111/j.1360-0443.2009.02726.x · Cited by 182 · cited 1×
“Our primary observation is that the characteristics of the MTurk recruited population more closely match the drinking patterns of participants recruited through general population surveys [18] as compared to recruitment through paper newspapers [19] or through online recruitment (e.”
BackgroundSelf-citationComparison of two Internet-based interventions for problem drinkers: randomized controlled trial
Cunningham, J. A. (2012) · Journal of Medical Internet Research · doi:10.2196/jmir.2090 · Cited by 68 · cited 1×
“Our primary observation is that the characteristics of the MTurk recruited population more closely match the drinking patterns of participants recruited through general population surveys [18] as compared to recruitment through paper newspapers [19] or through online recruitment (e.”
BackgroundTreating alcohol problems with self-help materials: a population study.
Cunningham, J. A., Koski-Jännes, A., Wild, T. C., & Cordingley, J. (2002) · Journal of Studies on Alcohol · doi:10.15288/jsa.2002.63.649 · Cited by 37 · cited 1×
“This does have the advantage of allowing for designs where some participants are not provided with an intervention at all (and further, had no expectation of receiving an intervention) [25].”
BackgroundSelf-citation
Cited by
Rundle, S. M., Cunningham, J. A., & Hendershot, C. S. (2021) · Drug and Alcohol Review · doi:10.1111/dar.13244 · Cited by 61
Self-citationExploring the Use of Large Language Models for Improving the Awareness of Mindfulness
Kumar, H., Wang, Y., Shi, J., Musabirov, I., Farb, N. A. S., & Williams, J. J. (2023) · Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544549.3585614 · Cited by 50
Bertholet, N., Godinho, A., & Cunningham, J. A. (2019) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2018.12.002 · Cited by 40
Self-citation
Metadata
- Authors
- Cunningham, J. A., Godinho, A., & Kushnir, V.
- Venue
- BMC Medical Research Methodology
- Published
- 2017-12
- Volume
- 17
- Issue
- 1
- Pages
- 156-156
- ISSN
- 1471-2288 (ISSN-L)
- OpenAlex
- W2775716135
- Cited by
- 57
- References
- 8
- FWCI
- 9.53
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Using Mechanical Turk to recruit participants for internet intervention research: experience from recruitment for four trials targeting hazardous alcohol consumption. https://app.cochat.ai/papers/10.1186/s12874-017-0440-3
- The paper
- Cunningham, J. A., Godinho, A., & Kushnir, V. (2017). Using Mechanical Turk to recruit participants for internet intervention research: experience from recruitment for four trials targeting hazardous alcohol consumption. BMC Medical Research Methodology. https://doi.org/10.1186/s12874-017-0440-3
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.