Using different Facebook advertisements to recruit men for an online mental health study: Engagement and selection bias
Open accessThe DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
Abstract
A growing number of researchers are using Facebook to recruit for a range of online health, medical, and psychosocial studies. There is limited research on the representativeness of participants recruited from Facebook, and the content is rarely mentioned in the methods, despite some suggestion that the advertisement content affects recruitment success. This study explores the impact of different Facebook advertisement content for the same study on recruitment rate, engagement, and participant characteristics. Five Facebook advertisement sets ("resilience", "happiness", "strength", "mental fitness", and "mental health") were used to recruit male participants to an online mental health study which allowed them to find out about their mental health and wellbeing through completing six measures. The Facebook advertisements recruited 372 men to the study over a one month period. The cost per participant from the advertisement sets ranged from $0.55 to $3.85 Australian dollars. The "strength" advertisements resulted in the highest recruitment rate, but participants from this group were least engaged in the study website. The "strength" and "happiness" advertisements recruited more younger men. Participants recruited from the "mental health" advertisements had worse outcomes on the clinical measures of distress, wellbeing, strength, and stress. This study confirmed that different Facebook advertisement content leads to different recruitment rates and engagement with a study. Different advertisement also leads to selection bias in terms of demographic and mental health characteristics. Researchers should carefully consider the content of social media advertisements to be in accordance with their target population and consider reporting this to enable better assessment of generalisability.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
KESSLER, R. C., ANDREWS, G., COLPE, L. J., HIRIPI, E., MROCZEK, D. K., NORMAND, S.-L. T., WALTERS, E. E., & ZASLAVSKY, A. M. (2002) · Psychological Medicine · doi:10.1017/s0033291702006074 · Cited by 10,327
The brief resilience scale: Assessing the ability to bounce back
Smith, B. W., Dalen, J., Wiggins, K., Tooley, E., Christopher, P., & Bernard, J. (2008) · International Journal of Behavioral Medicine · doi:10.1080/10705500802222972 · Cited by 6,439 · cited 3×
“The BRSwas used as the Resilience measure.”
InfluentialFURUKAWA, T. A., KESSLER, R. C., SLADE, T., & ANDREWS, G. (2003) · Psychological Medicine · doi:10.1017/s0033291702006700 · Cited by 1,570 · cited 1×
“The K6 is a measure of nonspecific psychological distress validated for use among the Australian population (Furukawa et al., 2003).”
BackgroundSocial research 2.0: virtual snowball sampling method using Facebook
Baltar, F., & Brunet, I. (2012) · Internet Research · doi:10.1108/10662241211199960 · Cited by 1,284 · cited 2×
“, 2014) or minority ethnic groups (Baltar and Brunet, 2012), but also people from the general population (Batterham et al.”
BackgroundRyan, T., & Xenos, S. (2011) · Computers in Human Behavior · doi:10.1016/j.chb.2011.02.004 · Cited by 1,260
Ramo, D. E., & Prochaska, J. J. (2012) · Journal of Medical Internet Research · doi:10.2196/jmir.1878 · Cited by 404
Recruiting for health, medical or psychosocial research using Facebook: Systematic review
Thornton, L., Batterham, P. J., Fassnacht, D. B., Kay-Lambkin, F., Calear, A. L., & Hunt, S. (2016) · Internet Interventions · doi:10.1016/j.invent.2016.02.001 · Cited by 326 · cited 7×
“Indeed, a recent systematic review reported over 100 studies have used Facebook to recruit participants for health, medical or mental health research, all within the last 8 years (Thornton et al., 2016).”
BackgroundInfluentialLindner, P., Nyström, M. B. T., Hassmén, P., Andersson, G., & Carlbring, P. (2015) · Internet Interventions · doi:10.1016/j.invent.2015.04.002 · Cited by 51 · cited 1×
“Lindner et al. (2015) found that participants seeking internet-administered cognitive behavioural therapy for depression differed in demographic and clinical characteristics depending on the source of recruitment.”
BackgroundBatterham, P. J., Calear, A. L., Sunderland, M., Carragher, N., & Brewer, J. L. (2016) · BJPsych Open · doi:10.1192/bjpo.bp.115.001552 · Cited by 41
Ellis, L. A., McCabe, K. L., Rahilly, K. A., Nicholas, M. A., Davenport, T. A., Burns, J. M., & Hickie, I. B. (2014) · Clinical Investigation · doi:10.4155/cli.14.61 · Cited by 38
Cited by
Apolinário-Hagen, J., Fritsche, L., Bierhals, C., & Salewski, C. (2018) · Internet Interventions · doi:10.1016/j.invent.2017.12.002 · Cited by 58
Deady, M., Johnston, D. A., Glozier, N., Milne, D., Choi, I., Mackinnon, A., Mykletun, A., Calvo, R. A., Gayed, A., Bryant, R., et al. (2018) · BMJ Open · doi:10.1136/bmjopen-2017-020510 · Cited by 14
“Social media advertising targeted at employed people will also be used to recruit beyond the partner organisations using a similar approach to previous studies.(37) The study will also be promoted via members of the research team presenting at partner worksites.”
ResultSelf-citationDuPont, C. M., Pressman, S. D., Reed, R. G., Manuck, S. B., Marsland, A. L., & Gianaros, P. J. (2023) · Affective Science · doi:10.1007/s42761-022-00148-z · Cited by 13
Metadata
- Authors
- Choi, I., Milne, D. N., Glozier, N., Peters, D., Harvey, S. B., & Calvo, R. A.
- Venue
- Internet Interventions
- Publisher
- Elsevier BV
- Published
- 2017-06
- Volume
- 8
- Pages
- 27-34
- ISSN
- 2214-7829 (ISSN-L)
- OpenAlex
- W2593891322
- Cited by
- 101
- References
- 10
- FWCI
- 9.10
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Using different Facebook advertisements to recruit men for an online mental health study: Engagement and selection bias. https://app.cochat.ai/papers/10.1016/j.invent.2017.02.002
- The paper
- Choi, I., Milne, D. N., Glozier, N., Peters, D., Harvey, S. B., & Calvo, R. A. (2017). Using different Facebook advertisements to recruit men for an online mental health study: Engagement and selection bias. Internet Interventions. https://doi.org/10.1016/j.invent.2017.02.002
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.