Loss to follow-up in cohort studies: how much is too much?
Abstract
Loss to follow-up is problematic in most cohort studies and often leads to bias. Although guidelines suggest acceptable follow-up rates, the authors are unaware of studies that test the validity of these recommendations. The objective of this study was to determine whether the recommended follow-up thresholds of 60-80% are associated with biased effects in cohort studies. A simulation study was conducted using 1000 computer replications of a cohort of 500 observations. The logistic regression model included a binary exposure and three confounders. Varied correlation structures of the data represented various levels of confounding. Differing levels of loss to follow-up were generated through three mechanisms: missing completely at random (MCAR), missing at random (MAR) and missing not at random (MNAR). The authors found no important bias with levels of loss that varied from 5 to 60% when loss to follow-up was related to MCAR or MAR mechanisms. However, when observations were lost to follow-up based on a MNAR mechanism, the authors found seriously biased estimates of the odds ratios with low levels of loss to follow-up. Loss to follow-up in cohort studies rarely occurs randomly. Therefore, when planning a cohort study, one should assume that loss to follow-up is MNAR and attempt to achieve the maximum follow-up rate possible.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Participants rarely leave a trial due to random reasons.
“because participants rarely leave a trial due to random reasons [25].”
From Patient Engagement with Conversational Agents in Health Applications 2016–2022: A Systematic Review and Meta-Analysis (2024) · cites this paper as [25]
Loss of data of more than 20% can have a profound impact on the validity of a study, if this loss is not of a random nature.
“Loss of data of more than 20% can have a profound impact on the validity of a study, if this loss is not of a random nature (Kristman et al., 2004).”
From The relationship between improvement in insomnia severity and long-term outcomes in the treatment of chronic fatigue (2018) · cites this paper as Kristman et al., 2004
Cites
Statistical Analysis With Missing Data
Lahiff, M., Little, R. J. A., & Rubin, D. B. (1989) · Journal of the American Statistical Association · doi:10.2307/2289883 · Cited by 17,384 · see citing works
Missing data: our view of the state of the art
Schafer, J. L., & Graham, J. W. (2002) · Psychological Methods · doi:10.1037/1082-989x.7.2.147 · Cited by 11,638 · see citing works
(1990) · Computers Environment and Urban Systems · doi:10.1016/0198-9715(90)90039-v · Cited by 2,468 · see citing works
Ziegel, E. R., & Lohr, S. L. (2000) · Technometrics · doi:10.2307/1271491 · Cited by 2,450 · see citing works
A comparison of inclusive and restrictive strategies in modern missing data procedures.
Collins, L. M., Schafer, J. L., & Kam, C.-M. (2001) · Psychological Methods · doi:10.1037/1082-989x.6.4.330 · Cited by 2,389 · see citing works
Attrition in longitudinal studies
Twisk, J., & de Vente, W. (2002) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00476-0 · Cited by 527 · cited 4× · see citing works
“Under this assumption, the probability of being lost to follow-up depends on the outcome to be measured and cannot be completely explained by the covariates [12].”
InfluentialRESPONSE AND FOLLOW-UP BIAS IN COHORT STUDIES
Greenland, S. (1977) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a112451 · Cited by 264 · see citing works
Statistics in medical journals: some recent trends
Altman, D. G. (2000) · Statistics in Medicine · doi:10.1002/1097-0258(20001215)19:23 · Cited by 252 · see citing works
A high response is not essential to prevent selection bias
Wiel, A. B. D., van Exel, E. R., de Craen, A. J. M., Gussekloo, J., Lagaay, A. M., Knook, D. L., & Westendorp, R. G. J. (2002) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(02)00505-x · Cited by 203 · see citing works
Mortality and Cancer Rates in Nonrespondents to a Prospective Study of Older Women: 5-Year Follow-up
Bisgard, K. M., Folsom, A. R., Hong, C.-P., & Sellers, T. A. (1994) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a116948 · Cited by 181 · see citing works
SELECTION BIAS IN EPIDEMIOLOGIC STUDIES12
Kleinbaum, D. G., Morgenstern, H., & Kupper, L. L. (1981) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a113113 · Cited by 153 · see citing works
Testing for Random Dropouts in Repeated Measurement Data
Ridout, M. S., & Diggle, P. J. (1991) · Biometrics · doi:10.2307/2532413 · Cited by 145 · see citing works
Factors Affecting Attrition in a Longitudinal Smoking Prevention Study
Siddiqui, O. I., Flay, B. R., & Hu, F. B. (1996) · Preventive Medicine · doi:10.1006/pmed.1996.0089 · Cited by 117 · see citing works
A comparison of analytic methods for non-random missingness of outcome data
Crawford, S. L. (1995) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(94)00124-9 · Cited by 110 · see citing works
Deeg, D. J. H. (2002) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00472-3 · Cited by 97 · see citing works
Kempen, G. I. J. M., & VANSONDEREN, E. (2002) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00474-7 · Cited by 80 · see citing works
Hollen, P. J., Gralla, R. J., Cox, C., Eberly, S. W., & Kris, M. G. (1997) · Lung Cancer · doi:10.1016/s0169-5002(97)00059-7 · Cited by 59 · see citing works
McLean, R. R., Hannan, M. T., Epstein, B. E., et al. (2000) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a010263 · Cited by 39 · see citing works
The importance of critically interpreting simulation studies.
Maldonado, G., & Greenland, S. (1997) · PubMed · Cited by 31 · see citing works
Butler, C. W., Snyder, M., Wood, D. E., Curtis, J. R., Albert, R. K., & Benditt, J. O. (2001) · CHEST Journal · doi:10.1378/chest.119.4.1056 · Cited by 24 · see citing works
Two-Sample Problems for a Dichotomous Variable with Missing Data
ELASHOFF, J. D., & Elashoff, R. M. (1974) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.2307/2347050 · Cited by 15 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Long-term Follow-up After Bariatric Surgery: A Systematic Review
Puzziferri, N., Roshek, T. B., Mayo, H. G., Gallagher, R., Belle, S. H., & Livingston, E. H. (2014) · JAMA · doi:10.1001/jama.2014.10706 · Cited by 819 · see citing works
Association between sleep duration and cognitive decline
Ma, Y., Liang, L., Zheng, F., Shi, L., Zhong, B., & Xie, W. (2020) · JAMA Network Open · doi:10.1001/jamanetworkopen.2020.13573 · Cited by 535 · see citing works
Systematic Review of Multivariable Prognostic Models for Mild Traumatic Brain Injury
Silverberg, N. D., Gardner, A. J., Brubacher, J. R., Panenka, W. J., Li, J. J., & Iverson, G. L. (2015) · Journal of Neurotrauma · doi:10.1089/neu.2014.3600 · Cited by 393 · see citing works
Lee, Y., Doumouras, A. G., Yu, J., Brar, K., Banfield, L., Gmora, S., Anvari, M., & Hong, D. (2019) · Clinical Gastroenterology and Hepatology · doi:10.1016/j.cgh.2018.10.017 · Cited by 371 · see citing works
Bärnighausen, T., Chaiyachati, K., Chimbindi, N., Peoples, A., Haberer, J., & Newell, M.-L. (2011) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(11)70181-5 · Cited by 246 · see citing works
The impact of childhood cancer on parents’ socio-economic situation—A systematic review
Roser, K., Erdmann, F., Michel, G., Winther, J. F., & Mader, L. (2019) · Psycho-Oncology · doi:10.1002/pon.5088 · Cited by 110 · see citing works
Matcham, F., Leightley, D., Siddi, S., Lamers, F., White, K. M., Annas, P., de Girolamo, G., Difrancesco, S., Haro, J. M., Horsfall, M., et al. (2022) · BMC Psychiatry · doi:10.1186/s12888-022-03753-1 · Cited by 110 · see citing works
“…number of participants not indicated as symptomatic. b total number of symptomatic: participants meeting criteria for MDD on the CIDI-SF and scoring > 25 “unacceptable” dropout, follow-up levels of 50, 60 and 70% have previously been described as adequate, good and very good respectively [44].”
Schulz, A. J., Israel, B. A., Mentz, G. B., Bernal, C., Caver, D., DeMajo, R., Diaz, G., Gamboa, C., Gaines, C., Hoston, B., et al. (2015) · Health Education & Behavior · doi:10.1177/1090198114560015 · Cited by 77 · see citing works
Physical activity of moderate intensity in leisure time and the risk of all cause mortality
Bucksch, J. (2005) · British Journal of Sports Medicine · doi:10.1136/bjsm.2004.015768 · Cited by 67 · see citing works
Cevasco, K. E., Morrison Brown, R. E., Woldeselassie, R., & Kaplan, S. (2024) · Journal of Medical Systems · doi:10.1007/s10916-024-02059-x · Cited by 35 · see citing works
“We expected an increase in chatbot user engagement would appear as a positive effect on LTFU rates because participants rarely leave a trial due to random reasons [25 ].”
Singh, L. J., Gaye, F., Cole, A. M., Chan, E. S. M., & Kofler, M. J. (2022) · Neuropsychology · doi:10.1037/neu0000798 · Cited by 32 · see citing works
Garzón, M., Pereira-da-Silva, L., Seixas, J., Papoila, A., & Alves, M. (2018) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph15040688 · Cited by 28 · see citing works
Sensitivity analysis in multiple imputation in effectiveness studies of psychotherapy
Crameri, A., von Wyl, A., Koemeda, M., Schulthess, P., & Tschuschke, V. (2015) · Frontiers in Psychology · doi:10.3389/fpsyg.2015.01042 · Cited by 24 · see citing works
Vethe, D., Kallestad, H., Jacobsen, H. B., Landrø, N. I., Borchgrevink, P. C., & Stiles, T. C. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.01764 · Cited by 16 · see citing works
Epidemiological fallacies of modern psychiatric research
Newson, R. S., Karlsson, H., & Tiemeier, H. (2011) · Nordic Journal of Psychiatry · doi:10.3109/08039488.2010.539268 · Cited by 12 · see citing works
DEMİR, M., & KÖSKÜN, T. (2023) · Psikiyatride Güncel Yaklaşımlar · doi:10.18863/pgy.1192543 · Cited by 7 · see citing works
Chan, E. S. M., Gaye, F., Cole, A. M., Singh, L. J., & Kofler, M. J. (2023) · Neuropsychology · doi:10.1037/neu0000918 · Cited by 6 · see citing works
- Authors
- Kristman, V., Manno, M., & Côté, P.
- Affiliations
- University of Toronto (CA)
- Mount Sinai Hospital (CA)
- Venue
- European Journal of Epidemiology
- Published
- 2004-08
- Volume
- 19
- Issue
- 8
- Pages
- 751-760
- ISSN
- 0393-2990 (ISSN-L)
- 1573-7284
- OpenAlex
- W2013188809
- Cited by
- 740
- References
- 21
- Citations by year
- 2026: 31 · 2025: 51 · 2024: 50 · 2023: 50 · 2022: 68 · 2021: 77 · 2020: 52 · 2019: 47 · 2018: 57 · 2017: 55 · 2016: 39 · 2015: 28 · 2014: 21 · 2013: 30 · 2012: 17
- FWCI
- 5.43
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- European Journal of Epidemiology (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Keywords
- loss to follow-up, cohort studies, follow-up rates, missing data mechanisms, missing not at random, missing at random, missing completely at random, selection bias, confounding, simulation study, odds ratio bias
- MeSH terms
- Canada, Epidemiologic Methods, Follow-Up Studies, Humans, Sensitivity and Specificity, Cohort Studies, Bias, Confounding Factors, Epidemiologic, Confidence Intervals, Odds Ratio
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
