CoChat
Sign Up
Cited by 56LGBTQ Health, Identity, and PolicySurvey Methodology and NonresponseGender, Sexuality, and Education

Bias From Potentially Mischievous Responders on Large-Scale Estimates of Lesbian, Gay, Bisexual, or Questioning (LGBQ)–Heterosexual Youth Health Disparities

Open access
Cimpian, Joseph R., Timmer, Jennifer D., Birkett, Michelle A., Marro, Rachel L., Turner, Blair C., Phillips, Gregory L.·2018·American Journal of Public Health
DOI 10.2105/ajph.2018.304407PMC PMC6215371
Open this paper in CoChat — free account

Abstract

Objectives. To determine how sensitive estimates of lesbian, gay, bisexual, or questioning (LGBQ)–heterosexual youth health disparities are to the presence of potentially mischievous responders. Methods. We used US data from the 2015 Youth Risk Behavior Survey, pooled across jurisdictions that included a question about sexual identity for a total sample of 148 960 students. We used boosted regressions (a machine-learning technique) to identify unusual patterns of responses to 7 screener items presumably unrelated to LGBQ identification, which generated an index of suspected mischievousness. We estimated LGBQ–heterosexual youth disparities on 20 health outcomes; then we removed 1% of suspected mischievous responders at a time and re-estimated disparities to assess the robustness of original estimates. Results. Accounting for suspected mischievousness reduced estimates of the average LGBQ–heterosexual youth health disparity by up to 46% for boys and 23% for girls; however, screening did not affect all outcomes equally. Drug- and alcohol-related disparities were most affected, particularly among boys, but bullying and suicidal ideation were unaffected. Conclusions. Including screener items in public health data sets and performing rigorous sensitivity analyses can support the validity of youth health estimates.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.2105/ajph.2018.304407

Cites

5 of 5 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Results were stable across a wide range of screening levels for mischievous responders.

    “Because my results were stable across a wide range of screening levels, I present all results using the full analytic sample.”

    From Homophobic Bullying as Gender Policing: Population-Based Evidence (2023) · cites this paper as Cimpian et al. 2018

    The cited paper’s Results (‘Screening Does Not Affect All Outcomes Equally’; page not provided) report substantial variation across outcomes, including an 87% reduction in the estimated male ecstasy-use disparity after screening. It does not support a general claim that results stayed stable across screening levels. The citing sentence describes the citing author’s own results, which the source cannot establish. (LLM, conf=0.87)

    Checked by a reasoning model — may be wrong.

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 25

  1. [1]Fanelli D, Ioannidis JP. US studies may overestimate effect sizes in softer research. Proc Natl Acad Sci U S A. 2013;110(37):15031–15036. doi: 10.1073/pnas.1302997110. [DOI] [PMC free article] [PubMed] [Google Scholar] · DOI
  2. [3]Fan X, Miller BC, Park K-E et al. An exploratory study about inaccuracy and invalidity in adolescent self-report surveys. Field Methods. 2006;18(3):223–244. [Google Scholar]
  3. [4]Furlong MJ, Sharkey JD, Bates MP, Smith DC. An examination of the reliability, data screening procedures, and extreme response patterns for the Youth Risk Behavior Surveillance Survey. J Sch Violence. 2008;3(2-3):109–130. [Google Scholar]
  4. [5]Robinson-Cimpian JP. Inaccurate estimation of disparities due to mischievous responders: several suggestions to assess conclusions. Educ Res. 2014;43(4):171–185. [Google Scholar]
  5. [6]Cimpian JR. Classification errors and bias regarding research on sexual minority youths. Educ Res. 2017;46(9):517–529. [Google Scholar]
  6. [7]Groves RM, Jr, Fowler FJ, Couper MP, Lepkowski JM, Tourangeau R. Survey Methodology. Hoboken, NJ: John Wiley and Sons; 2011. [Google Scholar]
  7. [8]Badgett L. Best Practices for Asking Questions About Sexual Orientation on Surveys. Los Angeles, CA: Williams Institute; 2009. [Google Scholar]
  8. [9]Saewyc EM, Bauer GR, Skay CL et al. Measuring sexual orientation in adolescent health surveys; evaluation of eight school-based surveys. J Adolesc Health. 2004;35(4):345e1–345e15. doi: 10.1016/j.jadohealth.2004.06.002. [DOI] [PubMed] [Google Scholar] · DOI
  9. [11]Turner CF, Ku L, Rogers SM, Lindberg LD, Pleck JH, Sonenstein FL. Adolescent sexual behavior, drug use, and violence: increased reporting with computer survey technology. Science. 1998;280(5365):867–873. doi: 10.1126/science.280.5365.867. [DOI] [PubMed] [Google Scholar] · DOI
  10. [12]Kann L, Olsen EO, McManus T et al. Sexual identity, sex of sexual contacts, and health-related behaviors among students in grades 9–12—United States and selected sites, 2015. MMWR Surveill Summ. 2016;65(9):1–202. doi: 10.15585/mmwr.ss6509a1. [DOI] [PubMed] [Google Scholar] · DOI · Paper page
  11. [13]Espelage DL, Aragon SR, Birkett M, Koenig BW. Homophobic teasing, psychological outcomes, and sexual orientation among high school students: what influence do parents and schools have? School Psych Rev. 2008;37(2):202–216. [Google Scholar]
  12. [14]Robinson JP, Espelage DL. Inequities in educational and psychological outcomes between LGBTQ and straight students in middle and high school. Educ Res. 2011;40(7):315–330. [Google Scholar]
  13. [15]Russell ST, Sinclair KO, Poteat VP, Koenig BW. Adolescent health and harassment based on discriminatory bias. Am J Public Health. 2012;102(3):493–495. doi: 10.2105/AJPH.2011.300430. [DOI] [PMC free article] [PubMed] [Google Scholar] · DOI
  14. [17]Jia Y, Konold TR, Cornell D, Huang F. The impact of validity screening on associations between self-reports of bullying victimization and student outcomes. Educ Psychol Meas. 2018;78(1):80–102. doi: 10.1177/0013164416671767. [DOI] [PMC free article] [PubMed] [Google Scholar] · DOI
  15. [18]Shukla K, Konold T. A two-step latent profile method for identifying invalid respondents in self-reported survey data. J Exp Educ. 2017;86(3):473–488. [Google Scholar]
  16. [19]Cornell D, Klein J, Konold T, Huang F. Effects of validity screening items on adolescent survey data. Psychol Assess. 2012;24(1):21–35. doi: 10.1037/a0024824. [DOI] [PubMed] [Google Scholar] · DOI
  17. [21]Clayton HB, Lowry R, August E, Jones SE. Nonmedical use of prescription drugs and sexual risk behaviors. Pediatrics. 2016;137(1):e20152480. doi: 10.1542/peds.2015-2480. [DOI] [PMC free article] [PubMed] [Google Scholar] · DOI
  18. [22]Raifman J, Moscoe E, Austin SB, McConnell M. Difference-in-differences analysis of the association between state same-sex marriage policies and adolescent suicide attempts. JAMA Pediatr. 2017;171(4):350–356. doi: 10.1001/jamapediatrics.2016.4529. [DOI] [PMC free article] [PubMed] [Google Scholar] · DOI · Paper page
  19. [23]Vagi KJ, O’Malley Olsen E, Basile KC, Vivolo-Kantor AM. Teen dating violence (physical and sexual) among US high school students: findings from the 2013 national Youth Risk Behavior Survey. JAMA Pediatr. 2015;169(5):474–482. doi: 10.1001/jamapediatrics.2014.3577. [DOI] [PMC free article] [PubMed] [Google Scholar] · DOI · Paper page
  20. [24]Zaza S, Kann L, Barrios LC. Lesbian, gay, and bisexual adolescents population estimate and prevalence of health behaviors. JAMA. 2016;316(22):2355–2356. doi: 10.1001/jama.2016.11683. [DOI] [PMC free article] [PubMed] [Google Scholar] · DOI
  21. [26]Friedman JH. Greedy function approximation: a gradient boosting machine. Ann Stat. 2001;29(5):1189–1232. [Google Scholar]
  22. [28]Athey S, Imbens GW. The state of applied econometrics: causality and policy evaluation. J Econ Perspect. 2017;31(2):3–32. [Google Scholar]
  23. [29]Mullainathan S, Spiess J. Machine learning: an applied econometric approach. J Econ Perspect. 2017;31(2):87–106. [Google Scholar]
  24. [31]Fish JN, Russell ST. Have mischievous responders misidentified sexual minority youth disparities in the National Longitudinal Study of Adolescent to Adult Health? Arch Sex Behav. 2017 doi: 10.1007/s10508-017-0993-6. Epub ahead of print. [DOI] [PMC free article] [PubMed] [Google Scholar] · DOI
  25. [32]Savin-Williams RC, Joyner K. The dubious assessment of gay, lesbian, and bisexual adolescents of Add Health. Arch Sex Behav. 2014;43(3):413–422. doi: 10.1007/s10508-013-0219-5. [DOI] [PubMed] [Google Scholar] · DOI

Cited by

Sorted by citations

Authors
Cimpian, J. R., Timmer, J. D., Birkett, M. A., Marro, R. L., Turner, B. C., & Phillips, G. L.
Affiliations
Northwestern University (US)
ORCID
Venue
American Journal of Public Health
Publisher
American Public Health Association
Published
2018-11
Volume
108
Issue
S4
Pages
S258-S265
ISSN
  • 0090-0036 (ISSN-L)
  • 1541-0048
DOI
10.2105/ajph.2018.304407
PMC
PMC6215371
OpenAlex
W2899316941
Cited by
56
References
5
Citations by year
2026: 1 · 2025: 14 · 2024: 8 · 2023: 12 · 2022: 5 · 2021: 4 · 2020: 9 · 2019: 2 · 2018: 1
FWCI
5.42
Type
journal article
Language
en
Open access
Locations
  • American Journal of Public Health (publishedVersion)
  • PubMed Central (publishedVersion)
  • PubMed (publishedVersion)
Funders
  • Spencer Foundation
  • National Academy of Education
  • National Institutes of Health
  • National Institute on Alcohol Abuse and Alcoholism (R01AA024409)
  • National Institute on Drug Abuse (K08DA037825)
  • National Institute on Minority Health and Health Disparities
Topics
  • Social Sciences › Psychology › Social Psychology › LGBTQ Health, Identity, and Policy
  • Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
  • Social Sciences › Gender Studies › Gender, Sexuality, and Education
Keywords
LGBQ youth, sexual minority youth, health disparities, survey response bias, Youth Risk Behavior Survey, bullying, suicidal ideation, sensitivity analysis, data validity
MeSH terms
Adolescent, Adult, Bisexuality, Child, Data Interpretation, Statistical, Female, Homosexuality, Humans, Male, Surveys and Questionnaires, Risk-Taking, Young Adult

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review