Attrition in longitudinal studies
Abstract
The purpose of this paper was to illustrate the influence of missing data on the results of longitudinal statistical analyses [i.e., MANOVA for repeated measurements and Generalised Estimating Equations (GEE)] and to illustrate the influence of using different imputation methods to replace missing data. Besides a complete dataset, four incomplete datasets were considered: two datasets with 10% missing data and two datasets with 25% missing data. In both situations missingness was considered independent and dependent on observed data. Imputation methods were divided into cross-sectional methods (i.e., mean of series, hot deck, and cross-sectional regression) and longitudinal methods (i.e., last value carried forward, longitudinal interpolation, and longitudinal regression). Besides these, also the multiple imputation method was applied and discussed. The analyses were performed on a particular (observational) longitudinal dataset, with particular missing data patterns and imputation methods. The results of this illustration shows that when MANOVA for repeated measurements is used, imputation methods are highly recommendable (because MANOVA as implemented in the software used, uses listwise deletion of cases with a missing value). Applying GEE analysis, imputation methods were not necessary. When imputation methods were used, longitudinal imputation methods were often preferable above cross-sectional imputation methods, in a way that the point estimates and standard errors were closer to the estimates derived from the complete dataset. Furthermore, this study showed that the theoretically more valid multiple imputation method did not lead to different point estimates than the more simple (longitudinal) imputation methods. However, the estimated standard errors appeared to be theoretically more adequate, because they reflect the uncertainty in estimation caused by missing values.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
GEE allows for not using imputation methods for missing data because participants with missing data are not excluded from the analysis.
“GEE allows for not using imputation methods for the missing data, because the participants with missing data are not excluded from the analysis [46].”
From Fibrates as drugs with senolytic and autophagic activity for osteoarthritis therapy (2019) · cites this paper as [46]
GEE models adjust for dependency of repeated measures within one participant and can handle non-normally distributed data and missing data.
“We chose GEE models because EMA data have a hierarchical structure; this type of model adjusts for dependency of repeated measures within one participant and can handle non-normally distributed data. GEE models are also suitable for dealing with missing data; it is not required to exclude participants with missing questionnaires, nor should missing questionnaires be imputed beforehand.”
From Experienced burden of and adherence to smartphone-based ecological momentary assessment in persons with affective disorders (2020) · cites this paper as [29]
Cites
Multiple Imputation for Nonresponse in Surveys
Rubin, D. B. (1987) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316696 · Cited by 20,701 · see citing works
Statistical Analysis With Missing Data
Lahiff, M., Little, R. J. A., & Rubin, D. B. (1989) · Journal of the American Statistical Association · doi:10.2307/2289883 · Cited by 17,384 · see citing works
Applied Regression Analysis and Other Multivariable Methods
Walker, E., Kleinbaum, D. G., Kupper, L. L., & Muller, K. E. (1989) · Technometrics · doi:10.2307/1270375 · Cited by 8,538 · see citing works
Longitudinal Data Analysis for Discrete and Continuous Outcomes
Zeger, S. L., & Liang, K. (1986) · Biometrics · doi:10.2307/2531248 · Cited by 7,730 · see citing works
Analysis of Incomplete Multivariate Data
Schafer, J. L. (1997) · doi:10.1201/9781439821862 · Cited by 6,012 · see citing works
Analysis of Incomplete Multivariate Data
Booth, D. E., & Schafer, J. L. (2000) · Technometrics · doi:10.2307/1271466 · Cited by 5,627 · see citing works
Multiple imputation after 18+ years
Rubin, D. B. (1996) · Journal of the American Statistical Association · doi:10.1080/01621459.1996.10476908 · Cited by 4,072 · see citing works
Multiple Imputation After 18+ Years
Rubin, D. B. (1996) · Journal of the American Statistical Association · doi:10.2307/2291635 · Cited by 2,686 · see citing works
Multiple Imputation for Nonresponse in Surveys.
Lynn, P. J., & Rubin, D. B. (1988) · Journal of the Royal Statistical Society Series D (The Statistician) · doi:10.2307/2348774 · Cited by 2,330 · see citing works
Statistical Analysis with Missing Data.
Chatfield, C., Little, R. J. A., & Rubin, D. B. (1988) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.2307/2982783 · Cited by 2,203 · see citing works
Modeling the Drop-Out Mechanism in Repeated-Measures Studies
Little, R. J. A. (1995) · Journal of the American Statistical Association · doi:10.1080/01621459.1995.10476615 · Cited by 1,522 · see citing works
Regression Analysis for Correlated Data
Liang, K., & Zeger, S. L. (1993) · Annual Review of Public Health · doi:10.1146/annurev.pu.14.050193.000355 · Cited by 765 · see citing works
An overview of methods for the analysis of longitudinal data
Zeger, S. L., & Liang, K. (1992) · Statistics in Medicine · doi:10.1002/sim.4780111406 · Cited by 684 · see citing works
The Amsterdam growth study : a longitudinal analysis of health, fitness, and lifestyle
Kemper, H. C. G. (1995) · Human Kinetics eBooks · Cited by 159 · see citing works
Testing for Random Dropouts in Repeated Measurement Data
Ridout, M. S., & Diggle, P. J. (1991) · Biometrics · doi:10.2307/2532413 · Cited by 145 · see citing works
Data quality in longitudinal research
Magnusson, D., & Bergman, L. R. (1990) · Cambridge University Press eBooks · Cited by 112 · see citing works
Twisk, J. W. R. (1997) · International Journal of Sports Medicine · doi:10.1055/s-2007-972718 · Cited by 72 · see citing works
Parametric likelihoods for multiple non-fatal competing risks and death
Shen, Y., & Thall, P. F. (1998) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19980515)17:9 · Cited by 17 · see citing works
EFFECT SIZE AND POWER FOR CLINICAL TRIALS THAT MEASURE YEARS OF HEALTHY LIFE
Diehr, P. H., Psaty, B. M., & Patrick, D. L. (1997) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19970615)16:11 · Cited by 11 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Multiple imputation of discrete and continuous data by fully conditional specification
van Buuren, S. (2007) · Statistical Methods in Medical Research · doi:10.1177/0962280206074463 · Cited by 2,827 · see citing works
A Review of Hot Deck Imputation for Survey Non‐response
Andridge, R., & Little, R. J. A. (2010) · International Statistical Review · doi:10.1111/j.1751-5823.2010.00103.x · Cited by 1,073 · see citing works
Loss to follow-up in cohort studies: how much is too much?
Kristman, V., Manno, M., & Côté, P. (2004) · European Journal of Epidemiology · doi:10.1023/b:ejep.0000036568.02655.f8 · Cited by 740 · see citing works
“Under this assumption, the probability of being lost to follow-up depends on the outcome to be measured and cannot be completely explained by the covariates [12].”
Influentialvan der Heijden, G. V. D., Donders, A., Stijnen, T., & Moons, K. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.01.015 · Cited by 664 · see citing works
Wood, A., White, I. R., & Thompson, S. G. (2004) · Clinical Trials · doi:10.1191/1740774504cn032oa · Cited by 546 · see citing works
“There is a large literature comparing methods to handle missing outcomes [7-10] and developing complex statistical procedures for handling MNAR situations and performing sensitivity analyses”
BackgroundEffective and viable mind-body stress reduction in the workplace: A randomized controlled trial.
Wolever, R. Q., Bobinet, K. J., McCabe, K., Mackenzie, E. R., Fekete, E., Kusnick, C. A., & Baime, M. (2012) · Journal of Occupational Health Psychology · doi:10.1037/a0027278 · Cited by 539 · see citing works
Effect of Online Hemodiafiltration on All-Cause Mortality and Cardiovascular Outcomes
Grooteman, M. P. C., van den Dorpel, M. A., Bots, M. L., Penne, E. L., van der Weerd, N. C., Mazairac, A. H. A., den Hoedt, C. H., van der Tweel, I., Levesque, R. L., Nubé, M. J., et al. (2012) · Journal of the American Society of Nephrology · doi:10.1681/asn.2011121140 · Cited by 527 · see citing works
Data cleaning: detecting, diagnosing, and editing data abnormalities
Van den Broeck, J., Argeseanu Cunningham, S., Eeckels, R., & Herbst, K. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020267 · Cited by 446 · see citing works
Exercise in Patients on Dialysis: A Multicenter, Randomized Clinical Trial
Manfredini, F., Mallamaci, F., D’Arrigo, G., Baggetta, R., Bolignano, D., Torino, C., Lamberti, N., Bertoli, S. V., Ciurlino, D., Rocca-Rey, L. A., et al. (2016) · Journal of the American Society of Nephrology · doi:10.1681/asn.2016030378 · Cited by 423 · see citing works
Missing data imputation: focusing on single imputation.
Zhang, Z. (2016) · PubMed Central · doi:10.3978/j.issn.2305-5839.2015.12.38 · Cited by 428 · see citing works
“There are also varieties of methods for longitudinal imputation (Table 1) (9-11).”
MethodKwok, O.-M., Underhill, A. T., Berry, J. W., Luo, W., Elliott, T. R., & Yoon, M. (2008) · Rehabilitation Psychology · doi:10.1037/a0012765 · Cited by 388 · see citing works
Attrition in longitudinal studies: who do you lose?
Young, A. F., Powers, J. R., & Bell, S. (2006) · Australian and New Zealand Journal of Public Health · doi:10.1111/j.1467-842x.2006.tb00849.x · Cited by 382 · see citing works
McCarthy, M. L., Zeger, S. L., Ding, R., Levin, S. R., Desmond, J. S., Lee, J., & Aronsky, D. (2009) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2009.03.006 · Cited by 367 · see citing works
Twisk, J. W. R. (2004) · European Journal of Epidemiology · doi:10.1023/b:ejep.0000036572.00663.f2 · Cited by 330 · see citing works
“A possible solution to avoid this is to use imputation techniques when missing observations occur [22, 23].”
Methodvan der Zwan, J. E., de Vente, W., Huizink, A. C., Bögels, S. M., & de Bruin, E. I. (2015) · Applied Psychophysiology and Biofeedback · doi:10.1007/s10484-015-9293-x · Cited by 338 · see citing works
“This technique adjusts for dependency of repeated measurements within one subject, and is capable of dealing with missing data (Twisk and de Vente 2002).”
BackgroundKazdin, A. E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108993647.010 · Cited by 279 · see citing works
Welsing, P. M. J., Landewé, R., van Riel, P. L. C. M., Boers, M., van Gestel, A. M., van der Linden, S., Swinkels, H. L., & van der Heijde, D. M. F. M. (2004) · Arthritis & Rheumatism · doi:10.1002/art.20350 · Cited by 271 · see citing works
Barzi, F. (2004) · American Journal of Epidemiology · doi:10.1093/aje/kwh175 · Cited by 263 · see citing works
Maddison, R., Mhurchú, C. N., Jiang, Y., Hoorn, S. V., Rodgers, A., Lawes, C. M. M., & Rush, E. C. (2007) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/1479-5868-4-62 · Cited by 260 · see citing works
Development of CBT for chemotherapy‐related cognitive change: results of a waitlist control trial
Ferguson, R. J., McDonald, B. C., Rocque, M. A., Furstenberg, C. T., Horrigan, S., Ahles, T. A., & Saykin, A. J. (2010) · Psycho-Oncology · doi:10.1002/pon.1878 · Cited by 242 · see citing works
“Amore complete discussion can be seen by Twisk [53].”
BackgroundThiazide Diuretics and the Risk for Hip Fracture
Schoofs, M. W. C. J., van der Klift, M., Albert Hofman, D., Laet, C. E. D. H. D., Herings, R. M. C., Stijnen, T., Pols, H. A. P., & Stricker, B. H. C. (2003) · Annals of Internal Medicine · doi:10.7326/0003-4819-139-6-200309160-00010 · Cited by 219 · see citing works
“The combined variance accounts for the uncertainty introduced by estimating the missing values (34).”
BackgroundHaukoos, J. S., & Newgard, C. D. (2007) · Academic Emergency Medicine · doi:10.1197/j.aem.2006.11.037 · Cited by 207 · see citing works
Bossen, D., Veenhof, C., Beek, K. E. V., Spreeuwenberg, P. M. M., Dekker, J., & de Bakker, D. H. (2013) · Journal of Medical Internet Research · doi:10.2196/jmir.2662 · Cited by 193 · see citing works
“Since GEE analyses are tolerant to data missing, no imputation techniques were used [44].”
Haukoos, J. S., & Newgard, C. D. (2007) · Academic Emergency Medicine · doi:10.1111/j.1553-2712.2007.tb01855.x · Cited by 176 · see citing works
“15 A much more common pattern of censoring appears relatively arbitrary and is referred to as a nonmonotone pattern.”
BackgroundNeurocognitive deficits in the (putative) prodrome and first episode of psychosis
Eastvold, A. D., Heaton, R. K., & Cadenhead, K. S. (2007) · Schizophrenia Research · doi:10.1016/j.schres.2007.03.013 · Cited by 174 · see citing works
Peters, S. A. E., Bots, M. L., den Ruijter, H. M., Palmer, M. K., Grobbee, D. E., Crouse, J. R., O’Leary, D. H., Evans, G. W., Raichlen, J. S., Moons, K. G. M., et al. (2012) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.11.012 · Cited by 165 · see citing works
Torp, N. C., Dahl, K., Skarphéðinsson, G., Thomsen, P. H., Valderhaug, R., Weidle, B., Melin, K., Hybel, K. A., Nissen, J. B., Lenhard, F., et al. (2014) · Behaviour Research and Therapy · doi:10.1016/j.brat.2014.11.005 · Cited by 156 · see citing works
Biering, K., Hjøllund, N. H., & Frydenberg, M. (2015) · Clinical Epidemiology · doi:10.2147/clep.s72247 · Cited by 152 · see citing works
“In a longitudinal simulation study, Twisk and de Vente compared different imputation approaches including MI.(36) They recommended MI or longitudinal single imputation that led to similar different point estimates after imputation; however, MI had more valid variability compared to longitudinal single imputation.”
InfluentialFibrates as drugs with senolytic and autophagic activity for osteoarthritis therapy
Nogueira-Recalde, U., Lorenzo-Gómez, I., Blanco, F. J., Loza, M. I., Grassi, D., Shirinsky, V., Shirinsky, I., Lotz, M., Robbins, P. D., Domínguez, E., et al. (2019) · EBioMedicine · doi:10.1016/j.ebiom.2019.06.049 · Cited by 155 · see citing works
“GEE allows for not using imputation methods for the missing data, because the participants with missing data are not excluded from the analysis [46].”
MethodShoulder pain in swimmers: A 12-month prospective cohort study of incidence and risk factors
Walker, H., Gabbe, B. J., Wajswelner, H., BLANCH, P. D., & Bennell, K. L. (2012) · Physical Therapy in Sport · doi:10.1016/j.ptsp.2012.01.001 · Cited by 153 · see citing works
Postponement and childlessness - Evidence from two British cohorts
Kneale, D., & Joshi, H. (2008) · Demographic Research · doi:10.4054/demres.2008.19.58 · Cited by 149 · see citing works
Ødegård, S., Landewé, R., van der Heijde, D. M. F. M., Kvien, T. K., Mowinckel, P., & Uhlig, T. (2005) · Arthritis & Rheumatism · doi:10.1002/art.21548 · Cited by 150 · see citing works
Sartor, C. D., Hasue, R. H., Cacciari, L., Butugan, M. K., Watari, R., Pássaro, A. D. C., Giacomozzi, C., & Sacco, I. D. C. N. (2014) · BMC Musculoskeletal Disorders · doi:10.1186/1471-2474-15-137 · Cited by 152 · see citing works
“The missing data was treated as missing completely at random (MCAR) [43], using the ‘mean of series’ imputation method [44] for the values with normal distribution (TPP, PP, PTI, mean COP velocity, kinetic and kinematic variables, MNSI questionnaire score and ABC score) and using the median for non-parametric variable…”
MethodLonjon, G., Porcher, R., Ergina, P., Fouet, M., & Boutron, I. (2016) · Annals of Surgery · doi:10.1097/sla.0000000000001797 · Cited by 149 · see citing works
Kaptein, K. I., de Jonge, P., van den Brink, R. H. S., & Korf, J. (2006) · Psychosomatic Medicine · doi:10.1097/01.psy.0000233237.79085.57 · Cited by 146 · see citing works
“Single imputation can be used to replace missing data when less than 10% of the data are missing, without biasing the results (34).”
MethodThe relationship between childbearing and transitions from marriage and cohabitation in Britain
Steele, F., Kallis, C., Goldstein, H., & Joshi, H. E. (2005) · Demography · doi:10.1353/dem.2005.0038 · Cited by 146 · see citing works
Kloek, C. J. J., Bossen, D., Spreeuwenberg, P. M. M., Dekker, J., de Bakker, D. H., & Veenhof, C. (2018) · Physical Therapy · doi:10.1093/ptj/pzy045 · Cited by 141 · see citing works
“28 According to the recommendations of Twisk et al 29,30 about handling missing data in longitudinal mixed-model analyses, no imputation techniques were used.”
Jönsson, B., Öhrn, K., Oscarson, N., & Lindberg, P. (2009) · Journal of Clinical Periodontology · doi:10.1111/j.1600-051x.2009.01453.x · Cited by 137 · see citing works
Predicting the longer term outcomes of total knee arthroplasty
Gandhi, R., Dhotar, H. S., Razak, F. A., Tso, P., Davey, J. R., & Mahomed, N. N. (2009) · The Knee · doi:10.1016/j.knee.2009.06.003 · Cited by 132 · see citing works
“Multiple imputation assumes normally distributed data and generally gives point estimates that would be closest to those that would be derived from a complete dataset [20].”
BackgroundMethodInfluentialGrégoire, S., Lachance, L., Bouffard, T., & Dionne, F. (2018) · Behavior Therapy · doi:10.1016/j.beth.2017.10.003 · Cited by 129 · see citing works
The National Data Bank for rheumatic diseases: a multi-registry rheumatic disease data bank.
Wolfe, F., & Michaud, K. (2011) · Rheumatology · doi:10.1093/rheumatology/keq155 · Cited by 134 · see citing works
Noordman, B. J., Verdam, M. G. E., Lagarde, S. M., Hulshof, M. C. C. M., van Hagen, P., Henegouwen, M. I. V. B., Wijnhoven, B. P. L., van Laarhoven, H. W. M., Nieuwenhuijzen, G. A. P., Hospers, G. A. P., et al. (2017) · Journal of Clinical Oncology · doi:10.1200/jco.2017.73.7718 · Cited by 127 · see citing works
“QLQ-C30 Functional scales Role 85 (22) 88 (24) NA 57 (29) 55 (32) 60 (30) 68 (29) 68 (30) 73 (26) 76 (28) 78 (24) 77 (27) Emotional 71 (20) 71 (21) NA 74 (21) 75 (24) 78 (21) 76 (22) 80 (23) 80 (18) 82 (19) 77 (20) 83 (20) Cognitive 89 (16) 93 (15) NA 84 (21) 82 (21) 82 (22) 85 (18) 85 (17) 84 (18) 86 (17) 84 (19) 87…”
Health problems were the strongest predictors of attrition during follow-up of the GAZEL cohort
Goldberg, M., Chastang, J. F., Zins, M., Niedhammer, I., & Leclerc, A. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.02.020 · Cited by 125 · see citing works
Boyne, J. J., Vrijhoef, H. J., Spreeuwenberg, M. D., Weerd, G. D., Kragten, J. A., Gorgels, A. P. M., & on behalf of the TEHAF investigators (2013) · European Journal of Cardiovascular Nursing · doi:10.1177/1474515113487464 · Cited by 125 · see citing works
Tosoian, J. J., Loeb, S., Feng, Z., Isharwal, S., Landis, P., Elliot, D. J., Veltri, R. W., Epstein, J. I., Partin, A. W., Carter, H. B., et al. (2012) · The Journal of Urology · doi:10.1016/j.juro.2012.06.009 · Cited by 123 · see citing works
“For longitudinal analyses, if a specimen was not available for biomarker measurements at the last biopsy, biomarker results were imputed using the method of last observation carried forward.(19) All models were adjusted for age, date of diagnosis and PSA density (continuous variables).”
MethodHaines, T. P., Russell, T. G., Brauer, S., Erwin, S., Lane, P., Urry, S. R., Jasiewicz, J. M., & Condie, P. (2009) · Clinical Rehabilitation · doi:10.1177/0269215509338998 · Cited by 119 · see citing works
Teesson, M., Marel, C., Darke, S. G., Ross, J. E., Slade, T. N., Burns, L., Lynskey, M. T., Memedovic, S., White, J., & Mills, K. L. (2015) · Addiction · doi:10.1111/add.12860 · Cited by 118 · see citing works
“As GEE uses all available data, missing data were not imputed (20).”
MethodChallenges and solutions for N-of-1 design studies in health psychology
Kwaśnicka, D., Inauen, J., Nieuwenboom, W., Nurmi, J., Schneider, A., Short, C. E., Dekkers, T., Williams, A. J., Bierbauer, W., Haukkala, A., et al. (2019) · Health Psychology Review · doi:10.1080/17437199.2018.1564627 · Cited by 117 · see citing works
Socio-economic determinants of suicide: an ecological analysis of 35 countries
Milner, A. J., McClure, R., & Leo, D. D. (2010) · Social Psychiatry and Psychiatric Epidemiology · doi:10.1007/s00127-010-0316-x · Cited by 118 · see citing works
“This technique imputes data based on the assumption of a linear development of the variable [52].”
Methods to Account for Attrition in Longitudinal Data: Do They Work? A Simulation Study
Kristman, V. L., Manno, M., & Côté, P. (2005) · European Journal of Epidemiology · doi:10.1007/s10654-005-7919-7 · Cited by 116 · see citing works
“Regression imputation In regression imputation, baseline variables are used as independent variables to predict the missing outcome for the subjects who were lost to follow-up [4, 5, 8, 11, 17].”
BackgroundMethodResultEvaluation of an individually tailored oral health educational programme on periodontal health
Jönsson, B., Öhrn, K., Lindberg, P., & Oscarson, N. (2010) · Journal of Clinical Periodontology · doi:10.1111/j.1600-051x.2010.01590.x · Cited by 113 · see citing works
kmlShape: An Efficient Method to Cluster Longitudinal Data (Time-Series) According to Their Shapes
Genolini, C., Écochard, R., Benghezal, M. O., Driss, T., Andrieu, S., & Subtil, F. (2016) · PLoS ONE · doi:10.1371/journal.pone.0150738 · Cited by 112 · see citing works
“Therefore, it can be used to cluster irregular trajectories, the use of imputation methods for longitudinal data [45 – 48] is not necessary.”
MethodBerry, D. L., Halpenny, B., Hong, F., Wolpin, S. E., Lober, W. B., Russell, K., Ellis, W. J., Govindarajulu, U. S., Bosco, J. L. F., Davison, B. J., et al. (2011) · Urologic Oncology Seminars and Original Investigations · doi:10.1016/j.urolonc.2011.10.004 · Cited by 111 · see citing works
Copy Mean: A New Method to Impute Intermittent Missing Values in Longitudinal Studies
Genolini, C., Écochard, R., & Jacqmin-Gadda, H. (2013) · Open Journal of Statistics · doi:10.4236/ojs.2013.34a004 · Cited by 83 · see citing works
“Twisk [8] and Engels [3] compared several imputation methods for longitudinal studies.”
Backgroundde Waal, Y. C. M., Raghoebar, G. M., Slater, J. J. R. H., Meijer, H. J. A., Winkel, E. G., & Winkelhoff, A. J. V. (2012) · Journal Of Clinical Periodontology · doi:10.1111/jcpe.12034 · Cited by 110 · see citing works
Åsenlöf, P., Denison, E., & Lindberg, P. (2005) · Journal of Pain · doi:10.1016/j.jpain.2005.03.008 · Cited by 107 · see citing works
Slane, J. D., Klump, K. L., McGue, M., & Iacono, W. G. (2014) · International Journal of Eating Disorders · doi:10.1002/eat.22329 · Cited by 104 · see citing works
Dawson, B. A., & Williams, S. (2008) · Journal of Youth and Adolescence · doi:10.1007/s10964-007-9233-z · Cited by 115 · see citing works
Mucosal Immunity and Illness Incidence in Elite Rugby Union Players across a Season
Cunniffe, B., GRIFFITHS, H., Proctor, W., Davies, B., Baker, J. S., & Jones, K. (2010) · Medicine & Science in Sports & Exercise · doi:10.1249/mss.0b013e3181ef9d6b · Cited by 99 · see citing works
“The linear interpolation imputation method was then used to adjust for these missing values (36).”
MethodLongitudinal analyses of family functioning in veterans and their partners across treatment.
Evans, L., Cowlishaw, S., Forbes, D., Parslow, R., & Lewis, V. J. (2010) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0020457 · Cited by 98 · see citing works
van Stralen, M. M., de Vries, H., Mudde, A. N., Bolman, C. A. W., & Lechner, L. (2011) · Health Psychology · doi:10.1037/a0023579 · Cited by 98 · see citing works
Paradis, V., Cossette, S., Frasure‐Smith, N., Heppell, S., & Guertin, M. (2010) · The Journal of Cardiovascular Nursing · doi:10.1097/jcn.0b013e3181c52497 · Cited by 98 · see citing works
“Missing follow-up data for the outcomes were imputed using the lastobservation-carried-forward (LOCF) technique.(50) P G”
Methodvan Zuilen, A. D., Bots, M. L., Dülger, A., van der Tweel, I., van Buren, M., ten Dam, M. A. G. J., Kaasjager, K. A. H., Ligtenberg, G., Sijpkens, Y. W. J., Sluiter, H. E., et al. (2012) · Kidney International · doi:10.1038/ki.2012.137 · Cited by 97 · see citing works
Racial Differences in Depression Trajectories among Older Women
Spence, N. J., Adkins, D. E., & Dupre, M. E. (2011) · Journal of Health and Social Behavior · doi:10.1177/0022146511410432 · Cited by 97 · see citing works
Frändin, K., Grönstedt, H. K., Helbostad, J. L., Bergland, A., Andresen, M., Puggaard, L., Harms‐Ringdahl, K., Granbo, R., & Hellström, K. (2016) · Gerontology · doi:10.1159/000443611 · Cited by 97 · see citing works
Deeg, D. J. H. (2002) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00472-3 · Cited by 97 · see citing works
Effects of short-term exercise training on signs and symptoms of generalized anxiety disorder
Herring, M. P., Jacob, M. L., Suveg, C., & O’Connor, P. J. (2011) · Mental Health and Physical Activity · doi:10.1016/j.mhpa.2011.07.002 · Cited by 93 · see citing works
Hsieh, C.-C., Lin, P.-S., Hsu, W., Wang, J., Huang, Y.-C., Lim, A.-Y., & Hsu, Y. (2018) · Dementia and Geriatric Cognitive Disorders · doi:10.1159/000494659 · Cited by 95 · see citing works
Leenstra, T., Coutinho, H. M., Acosta, L. P., LANGDON, G. C., Su, L., Olveda, R. M., McGarvey, S. T., Kurtis, J. D., & Friedman, J. F. (2006) · Infection and Immunity · doi:10.1128/iai.00757-06 · Cited by 95 · see citing works
“These results were considered to be false-negative results (falling under the limit of detection for stool examination), and egg counts were imputed using the “last value carried over” method (31).”
MethodMethodological Issues in the Effects of Attrition: Simple Solutions for Social Scientists
Ahern, K., & Brocque, R. M. L. (2004) · Field Methods · doi:10.1177/1525822x04271006 · Cited by 93 · see citing works
Hou, I., Lin, H., Shen, S.-H., Chang, K., Tai, H., Tsai, A.-J., & Dykes, P. C. (2020) · JMIR mhealth and uhealth · doi:10.2196/17084 · Cited by 91 · see citing works
“GEE analysis showed a statistically significant difference ( P =.03) in the interaction between groups and time, which meant the experimental group had a greater mean summary score for the QLQ-C30 (difference=5.05) than the control group after the intervention at the 3-month assessment (see Table 3). e Diff: differenc…”
MethodInfluentialEfficacy of Two Tailored Interventions Promoting Physical Activity in Older Adults
van Stralen, M. M., de Vries, H., Mudde, A. N., Bolman, C. A. W., & Lechner, L. (2009) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2009.07.009 · Cited by 91 · see citing works
Predictors identified for losses to follow-up among HIV-seropositive patients
Lanoy, É., Mary‐Krause, M., Tattevin, P., Dray‐Spira, R., Duvivier, C., FISCHER, P., Obadia, Y., Lert, F., & Costagliola, D. G. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2005.11.024 · Cited by 91 · see citing works
Peels, D. A., Bolman, C. A. W., Golsteijn, R. H. J., de Vries, H., Mudde, A. N., van Stralen, M. M., & Lechner, L. (2013) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/1479-5868-10-104 · Cited by 89 · see citing works
“Applying multilevel analyses to an incomplete dataset has been shown to give more accurate estimations than applying imputation methods [17].”
BackgroundMethodNew methods for estimating follow-up rates in cohort studies
Xue, X., Agalliu, I., Kim, M., Wang, T., Lin, J., Ghavamian, R., & Strickler, H. D. (2017) · BMC Medical Research Methodology · doi:10.1186/s12874-017-0436-z · Cited by 89 · see citing works
Junaid, K. P., Kiran, T., Gupta, M., Kishore, K., & Siwatch, S. (2025) · Population Health Metrics · doi:10.1186/s12963-025-00364-2 · Cited by 94 · see citing works
The impact of attrition on the representativeness of cohort studies of older people
Brilleman, S. L., Pachana, N. A., & Dobson, A. J. (2010) · BMC Medical Research Methodology · doi:10.1186/1471-2288-10-71 · Cited by 88 · see citing works
“Random effects models and generalised estimating equations may be used for longitudinal analysis with data missing from participants who have dropped out [12].”
MethodEvidence and practice in spine registries
van Hooff, M. L., Jacobs, W. C. H., Willems, P. C. P. H., Wouters, M. W. J. M., de Kleuver, M., Peul, W. C., Ostelo, R., & Fritzell, P. (2015) · Acta Orthopaedica · doi:10.3109/17453674.2015.1043174 · Cited by 84 · see citing works
Cohen, D. M., Patel, C. B., Ahobila‐Vajjula, P., Sundberg, L. M., Chacko, T., Liu, S., & Narayana, P. A. (2009) · NMR in Biomedicine · doi:10.1002/nbm.1343 · Cited by 96 · see citing works
“, GEE can handle cases with missing data without the need for imputation (30).”
A novel imputation methodology for time series based on pattern sequence forecasting
Bokde, N. D., Beck, M. W., Alvarez, F. M., & Kulat, K. (2018) · Pattern Recognition Letters · doi:10.1016/j.patrec.2018.09.020 · Cited by 81 · see citing works
Dueñas, M., Ramirez, C., Arana, R., & Failde, I. (2011) · BMC Cardiovascular Disorders · doi:10.1186/1471-2261-11-24 · Cited by 83 · see citing works
“If a particular subject is missing one or more out of T repeated measurements, the remaining available data from the other measurements for that particular subject are used in the analyses [26].”
BackgroundTh2 Cytokines Are Associated with Persistent Hepatic Fibrosis in HumanSchistosoma japonicumInfection
Coutinho, H. M., Acosta, L. P., Wu, H. W., McGarvey, S. T., Su, L., Langdon, G., Jiz, M., Jarilla, B. R., Olveda, R. M., Friedman, J. F., et al. (2006) · The Journal of Infectious Diseases · doi:10.1086/510313 · Cited by 83 · see citing works
“puted using the last-value-carried-forward method [20].”
Noordman, B. J., Verdam, M. G. E., Lagarde, S. M., Shapiro, J., Hulshof, M. C. C. M., Henegouwen, M. I. V. B., Wijnhoven, B. P. L., Nieuwenhuijzen, G. A. P., Bonenkamp, J. J., Cuesta, M. A., et al. (2017) · Annals of Oncology · doi:10.1093/annonc/mdx726 · Cited by 83 · see citing works
Research Design in Clinical Psychology
Kazdin, A. E. (2021) · doi:10.1017/9781108993647 · Cited by 240 · see citing works
Effect of Physical Inactivity on Cognitive Performance after 2.5 Years of Follow-Up
Aichberger, M. C., Busch, M. A., Reischies, F. M., Ströhle, A., Heinz, A., & Rapp, M. A. (2010) · GeroPsych · doi:10.1024/1662-9647/a000003 · Cited by 82 · see citing works
Kids in the city study: research design and methodology
Oliver, M., Witten, K., KEARNS, R. A., Mavoa, S., Badland, H. M., Carroll, P., Drumheller, C., Tava’e, N., Asiasiga, L., Jelley, S., et al. (2011) · BMC Public Health · doi:10.1186/1471-2458-11-587 · Cited by 81 · see citing works
“Moreover, GEE models handle unbalanced and data missing at random [57].”
BackgroundStull, C. L., Messam, L. L. M., Collar, C. A., Peterson, N. G., Castillo, A. R., Reed, B., Andersen, K. L., & VerBoort, W. R. (2008) · Journal of Dairy Science · doi:10.3168/jds.2008-1215 · Cited by 81 · see citing works
“Missing test day values were imputed longitudinally by linear interpolation (Twisk and de Vente, 2002).”
MethodLongitudinal trajectory of early functional recovery in patients with first episode psychosis
Hall, M., Holton, K. M., Öngür, D., Montrose, D. M., & Keshavan, M. S. (2019) · Schizophrenia Research · doi:10.1016/j.schres.2019.02.003 · Cited by 80 · see citing works
Kempen, G. I. J. M., & VANSONDEREN, E. (2002) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00474-7 · Cited by 80 · see citing works
Haines, T. P., Bell, R., & Varghese, P. N. (2010) · Journal of the American Geriatrics Society · doi:10.1111/j.1532-5415.2010.02735.x · Cited by 79 · see citing works
“GEEs are similar to repeated-measures analysis of variance (ANOVA) but can be adapted for a range of data distributions for the dependent variable.(24) A negative binomial family with log link function and exchangeable working correlation structure for the GEE analyses was employed.”
MethodAnxiety sensitivity, its stability and longitudinal association with severity of anxiety symptoms
Hovenkamp-Hermelink, J. H. M., van der Veen, D. C., Oude Voshaar, R. C., Batelaan, N. M., Penninx, B. W. J. H., Jeronimus, B. F., Schoevers, R. A., & Riese, H. (2019) · Scientific Reports · doi:10.1038/s41598-019-39931-7 · Cited by 73 · see citing works
CopyMean: A new method to predict monotone missing values in longitudinal studies
Genolini, C., Lacombe, A., Écochard, R., & Subtil, F. (2016) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2016.04.010 · Cited by 52 · see citing works
Lechat, T., & Torrès, O. (2017) · International Journal of Entrepreneurship and Small Business · doi:10.1504/ijesb.2017.087838 · Cited by 76 · see citing works
Hinterlong, J. E. (2008) · Journal of Aging & Social Policy · doi:10.1080/08959420801977491 · Cited by 72 · see citing works
HMG-CoA Reductase Inhibitors and the Risk of Vertebral Fracture
Schoofs, M. W. C. J., Sturkenboom, M., van der Klift, M., Albert Hofman, D., Pols, H. A. P., & Stricker, B. H. C. (2004) · Journal of Bone and Mineral Research · doi:10.1359/jbmr.040607 · Cited by 72 · see citing works
“The combined variance accounts for the uncertainty introduced by estimating the missing values.((34)) In the original data set, there were no missing values for age, gender, and length of follow-up, and 170 (4.”
BackgroundSimple imputation methods were inadequate for missing not at random (MNAR) quality of life data
Fielding, S., Fayers, P. M., McDonald, A. M., McPherson, G. C., & Campbell, M. (2008) · Health and Quality of Life Outcomes · doi:10.1186/1477-7525-6-57 · Cited by 71 · see citing works
van Genugten, C. R., Schuurmans, J., Lamers, F., Riese, H., Penninx, B. W. J. H., Schoevers, R. A., Riper, H. M., & Smit, J. H. (2020) · Journal of Clinical Medicine · doi:10.3390/jcm9020322 · Cited by 71 · see citing works
“A more detailed description about GEE models is described in detail elsewhere [28,29].”
MethodFrost, D. M., Meyer, I. H., Lin, A., Wilson, B. D. M., Lightfoot, M., Russell, S. T., & Hammack, P. L. (2022) · Archives of Sexual Behavior · doi:10.1007/s10508-022-02288-6 · Cited by 71 · see citing works
“GEEs do not require dropping the entirety of a participant’s data if they did not participate in all waves (case-wise deletion), and GEEs do not require imputation of missing data for missing waves (Twisk & de Vente, 2002).”
BackgroundSainani, K. L. (2015) · PM&R · doi:10.1016/j.pmrj.2015.07.011 · Cited by 68 · see citing works
“Simulations show that multiple imputation along with maximum likelihood methods (described later) are optimal for most common missing data scenarios [1-4].”
MethodPeels, D. A., van Stralen, M. M., Bolman, C. A. W., Golsteijn, R. H. J., de Vries, H., Mudde, A. N., & Lechner, L. (2014) · Health Education Research · doi:10.1093/her/cyu039 · Cited by 67 · see citing works
“to an incomplete dataset have been shown to give more accurate estimations than applying imputation methods [51].”
Method
- Authors
- Twisk, J., & de Vente, W.
- Affiliations
- Vrije Universiteit Amsterdam (NL)
- Corresponding author
- Twisk, Jos
- ORCID
- Twisk, Jos
- Venue
- Journal of Clinical Epidemiology
- Publisher
- Elsevier BV
- Published
- 2002-04
- Volume
- 55
- Issue
- 4
- Pages
- 329-337
- ISSN
- 0895-4356 (ISSN-L)
- OpenAlex
- W2032594557
- Cited by
- 527
- References
- 18
- Citations by year
- 2026: 18 · 2025: 15 · 2024: 9 · 2023: 20 · 2022: 24 · 2021: 43 · 2020: 18 · 2019: 36 · 2018: 28 · 2017: 27 · 2016: 26 · 2015: 23 · 2014: 29 · 2013: 28 · 2012: 27
- FWCI
- 14.23
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Clinical Epidemiology (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Keywords
- attrition, missing data, longitudinal study, imputation methods, multiple imputation, generalized estimating equations, listwise deletion, hot deck imputation, cross-sectional regression, standard errors
- MeSH terms
- Analysis of Variance, Data Interpretation, Statistical, Epidemiologic Methods, Humans, Longitudinal Studies, Mathematical Computing, Research Design
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
