Imputation of systematically missing predictors in an individual participant data meta-analysis: a generalized approach using MICE
Open accessAbstract
Individual participant data meta-analyses (IPD-MA) are increasingly used for developing and validating multivariable (diagnostic or prognostic) risk prediction models. Unfortunately, some predictors or even outcomes may not have been measured in each study and are thus systematically missing in some individual studies of the IPD-MA. As a consequence, it is no longer possible to evaluate between-study heterogeneity and to estimate study-specific predictor effects, or to include all individual studies, which severely hampers the development and validation of prediction models. Here, we describe a novel approach for imputing systematically missing data and adopt a generalized linear mixed model to allow for between-study heterogeneity. This approach can be viewed as an extension of Resche-Rigon's method (Stat Med 2013), relaxing their assumptions regarding variance components and allowing imputation of linear and nonlinear predictors. We illustrate our approach using a case study with IPD-MA of 13 studies to develop and validate a diagnostic prediction model for the presence of deep venous thrombosis. We compare the results after applying four methods for dealing with systematically missing predictors in one or more individual studies: complete case analysis where studies with systematically missing predictors are removed, traditional multiple imputation ignoring heterogeneity across studies, stratified multiple imputation accounting for heterogeneity in predictor prevalence, and multilevel multiple imputation (MLMI) fully accounting for between-study heterogeneity. We conclude that MLMI may substantially improve the estimation of between-study heterogeneity parameters and allow for imputation of systematically missing predictors in IPD-MA aimed at the development and validation of prediction models.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Missing predictor values were imputed using multivariate imputation via chained equations exclusively using observed values of the predictors, without using the MRI score response.
“we imputed the missing predictor values using the multivariate imputation via chained equations (11) algorithm implemented in R exclusively using observed values of the predictors (i.e., without using the MRI score response that one targets to predict)”
From Lab values in neonates with hypoxic ischemic encephalopathy over time during and after therapeutic hypothermia (2026) · cites this paper as [11]
Multilevel imputation has been shown to lead to better outcomes than complete case analysis and traditional multiple imputation.
“Multilevel imputation has been shown to lead to better outcomes than complete case analysis and traditional multiple imputation.”
From What is the prevalence of fear of cancer recurrence in cancer survivors and patients? A systematic review and individual participant data meta‐analysis (2022) · cites this paper as [37]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Multiple Imputation for Nonresponse in Surveys
Rubin, D. B. (1987) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316696 · Cited by 20,701 · see citing works
Statistical Analysis With Missing Data
Lahiff, M., Little, R. J. A., & Rubin, D. B. (1989) · Journal of the American Statistical Association · doi:10.2307/2289883 · Cited by 17,384 · see citing works
<b>mice</b>: Multivariate Imputation by Chained Equations in<i>R</i>
Buuren, S. V., & Groothuis-Oudshoorn, K. (2011) · Journal of Statistical Software · doi:10.18637/jss.v045.i03 · Cited by 13,870 · see citing works
Self-citationMultiple imputation using chained equations: Issues and guidance for practice
White, I. R., Royston, P., & Wood, A. M. (2011) · Statistics in Medicine · doi:10.1002/sim.4067 · Cited by 9,792 · see citing works
Generalized linear mixed models: a practical guide for ecology and evolution
Bolker, B. M., Brooks, M. E., Clark, C. J., Geange, S. W., Poulsen, J. R., Stevens, M. H. H., & White, J.-S. S. (2009) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2008.10.008 · Cited by 8,910 · see citing works
Statistical Analysis with Missing Data
Little, R. J. A., & Rubin, D. B. (2002) · Wiley Series in Probability and Statistics · doi:10.1002/9781119013563 · Cited by 8,473 · see citing works
Linear Mixed-Effects Models using 'Eigen' and S4
Bates, D. M., Maechler, M., Bolker, B. M., & Walker, S. C. (2015) · Cited by 5,924 · see citing works
Schafer, J. L. (1999) · Statistical Methods in Medical Research · doi:10.1177/096228029900800102 · Cited by 3,949 · see citing works
Multiple imputation of discrete and continuous data by fully conditional specification
van Buuren, S. (2007) · Statistical Methods in Medical Research · doi:10.1177/0962280206074463 · Cited by 2,827 · see citing works
How Many Imputations are Really Needed? Some Practical Clarifications of Multiple Imputation Theory
Graham, J. W., Olchowski, A. E., & Gilreath, T. D. (2007) · Prevention Science · doi:10.1007/s11121-007-0070-9 · Cited by 2,795 · cited 1× · see citing works
“Although five imputations should suffice in many applications, it has previously been recommended to allow for 20–100 imputations [40].”
Prognosis Research Strategy (PROGRESS) 3: Prognostic Model Research
Steyerberg, E. W., Moons, K. G. M., van der Windt, D. A., Hayden, J. A., Perel, P., Schroter, S., Riley, R. D., Hemingway, H., Altman, D. G., & for the PROGRESS Group (2013) · PLoS Medicine · doi:10.1371/journal.pmed.1001381 · Cited by 1,508 · see citing works
Self-citationSteyerberg, E. W., Mushkudiani, N. A., Perel, P., Butcher, I., Lü, J., McHugh, G. S., Murray, G. D., Marmarou, A., Roberts, I., Habbema, J. D. F., et al. (2008) · PLoS Medicine · doi:10.1371/journal.pmed.0050165 · Cited by 1,437 · see citing works
Fully conditional specification in multivariate imputation
Van Buuren, S., Brand, J. P. L., Groothuis-Oudshoorn, C. G. M., & Rubin, D. B. (2006) · Journal of Statistical Computation and Simulation · doi:10.1080/10629360600810434 · Cited by 1,217 · see citing works
Approximations to the Log-Likelihood Function in the Nonlinear Mixed-Effects Model
Pinheiro, J. C., & Bates, D. M. (1995) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.1995.10474663 · Cited by 1,058 · see citing works
Moons, K. G. M., Kengne, A. P., Woodward, M., Royston, P., Vergouwe, Y., Altman, D. G., & Grobbee, D. E. (2012) · Heart · doi:10.1136/heartjnl-2011-301246 · Cited by 889 · see citing works
Self-citationPrognosis Research Strategy (PROGRESS) 2: Prognostic Factor Research
Riley, R. D., Hayden, J. A., Steyerberg, E. W., Moons, K. G. M., Abrams, K., Kyzas, P. A., Malats, N., Briggs, A., Schroter, S., Altman, D. G., et al. (2013) · PLoS Medicine · doi:10.1371/journal.pmed.1001380 · Cited by 823 · see citing works
Self-citationMultiple-Imputation Inferences with Uncongenial Sources of Input
Meng, X. (1994) · Statistical Science · doi:10.1214/ss/1177010269 · Cited by 761 · see citing works
Handbook of Advanced Multilevel Analysis
Hox, J. J., & Roberts, J. (2011) · Routledge eBooks · doi:10.4324/9780203848852 · Cited by 751 · see citing works
Debray, T. P. A., Vergouwe, Y., Koffijberg, H., Nieboer, D., Steyerberg, E. W., & Moons, K. G. M. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2014.06.018 · Cited by 597 · see citing works
Self-citationRecent developments in meta‐analysis
Sutton, A. J., & Higgins, J. P. T. (2007) · Statistics in Medicine · doi:10.1002/sim.2934 · Cited by 583 · see citing works
Approximations to the Log-Likelihood Function in the Nonlinear Mixed-Effects Model
Pinheiro, J. C., Bates, D. M., & Pinheiro, J. C. (1995) · Journal of Computational and Graphical Statistics · doi:10.2307/1390625 · Cited by 444 · see citing works
Heinze, G. (2006) · Statistics in Medicine · doi:10.1002/sim.2687 · Cited by 380 · see citing works
Problems due to Small Samples and Sparse Data in Conditional Logistic Regression Analysis
Greenland, S., Schwartzbaum, J., & Finkle, W. D. (2000) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a010240 · Cited by 367 · see citing works
Geersing, G., Zuithoff, N. P. A., Kearon, C., Anderson, D. R., ten Cate‐Hoek, A. J., Elf, J. L., Bates, S. M., Hoes, A. W., Kraaijenhagen, R. A., Oudega, R., et al. (2014) · BMJ · doi:10.1136/bmj.g1340 · Cited by 227 · see citing works
Self-citationDebray, T. P. A., Moons, K. G. M., Ahmed, I., Koffijberg, H., & Riley, R. D. (2013) · Statistics in Medicine · doi:10.1002/sim.5732 · Cited by 215 · see citing works
Self-citationIndividual Participant Data Meta-Analysis for a Binary Outcome: One-Stage or Two-Stage?
Debray, T. P. A., Moons, K. G. M., Abo-Zaid, G., Koffijberg, H., & Riley, R. D. (2013) · PLoS ONE · doi:10.1371/journal.pone.0060650 · Cited by 201 · see citing works
Self-citationAustin, P. C. (2010) · The International Journal of Biostatistics · doi:10.2202/1557-4679.1195 · Cited by 193 · see citing works
Ruling out deep venous thrombosis in primary care
Moons, K. G. M., Hoes, A. W., & Oudega, R. (2005) · Thrombosis and Haemostasis · doi:10.1160/th04-12-0829 · Cited by 144 · see citing works
Self-citationRoyston, P. J., Parmar, M., & Sylvester, R. J. (2004) · Statistics in Medicine · doi:10.1002/sim.1691 · Cited by 129 · see citing works
Resche‐Rigon, M., White, I. R., Bartlett, J. W., Peters, S. A. E., Thompson, S. G., & on behalf of the PROG‐IMT Study Group (2013) · Statistics in Medicine · doi:10.1002/sim.5894 · Cited by 106 · see citing works
Combining multiple imputation and meta‐analysis with individual participant data
Burgess, S., White, I. R., Resche‐Rigon, M., & Wood, A. (2013) · Statistics in Medicine · doi:10.1002/sim.5844 · Cited by 105 · see citing works
Developing and validating risk prediction models in an individual participant data meta-analysis
Ahmed, I., Debray, T. P. A., Moons, K. G. M., & Riley, R. D. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-3 · Cited by 105 · see citing works
Self-citationInfinitely Imbalanced Logistic Regression
Owen, A. B. (2007) · doi:10.5555/1248659.1248686 · Cited by 94 · see citing works
REML estimation for binary data in GLMMs
Noh, M., & Lee, Y. (2007) · Journal of Multivariate Analysis · doi:10.1016/j.jmva.2006.11.009 · Cited by 91 · see citing works
Prognostic models based on literature and individual patient data in logistic regression analysis
Steyerberg, E. W., Eijkemans, M. J. C., van Houwelingen, H. C., Lee, K. L., & Habbema, J. D. F. (2000) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(20000130)19:2 · Cited by 90 · see citing works
A clinical prediction model to assess the risk of operative delivery
Schuit, E., Kwee, A., Westerhuis, M., van Dessel, H., Graziosi, G., van Lith, J., Nijhuis, J., Oei, S., Oosterbaan, H., Schuitemaker, N., et al. (2012) · BJOG: an International Journal of Obstetrics and Gynaecology · doi:10.1111/j.1471-0528.2012.03334.x · Cited by 57 · see citing works
Self-citationThe Fibrinogen Studies Collaboration (2009) · Statistics in Medicine · doi:10.1002/sim.3540 · Cited by 56 · see citing works
Phillips, R. S., Sutton, A. J., Riley, R. D., Chisholm, J., Picton, S. V., & Stewart, L. A. (2012) · Systematic Reviews · doi:10.1186/2046-4053-1-8 · Cited by 33 · see citing works
REML Estimation of Variance Parameters in Nonlinear Mixed Effects Models Using the SAEM Algorithm
Meza, C., Jaffrézic, F., & Foulley, J. (2007) · Biometrical Journal · doi:10.1002/bimj.200610348 · Cited by 23 · see citing works
Incorporating published univariable associations in diagnostic and prognostic modeling
Debray, T. P. A., Koffijberg, H., Lu, D., Vergouwe, Y., Steyerberg, E. W., & Moons, K. G. M. (2012) · BMC Medical Research Methodology · doi:10.1186/1471-2288-12-121 · Cited by 16 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Moons, K. G. M., Wolff, R. F., Riley, R. D., Whiting, P. F., Westwood, M., Collins, G. S., Reitsma, J. B., Kleijnen, J., & Mallett, S. (2019) · Annals of Internal Medicine · doi:10.7326/m18-1377 · Cited by 1,669 · see citing works
Self-citationRiley, R. D., Ensor, J., Snell, K. I. E., Debray, T. P. A., Altman, D. G., Moons, K. G. M., & Collins, G. S. (2016) · BMJ · doi:10.1136/bmj.i3140 · Cited by 585 · see citing works
Self-citationBurke, D. L., Ensor, J., & Riley, R. D. (2017) · Statistics in Medicine · doi:10.1002/sim.7141 · Cited by 538 · see citing works
Handling missing data in clinical research
Heymans, M. W., & Twisk, J. W. R. (2022) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2022.08.016 · Cited by 399 · see citing works
The Consortium on Thyroid and Pregnancy—Study Group on Preterm Birth, Korevaar, T. I. M., Derakhshan, A., Taylor, P. N., Meima, M., Chen, L., Bliddal, S., Carty, D. M., Meems, M., Vaidya, B., et al. (2019) · JAMA · doi:10.1001/jama.2019.10931 · Cited by 378 · see citing works
Get real in individual participant data (IPD) meta‐analysis: a review of the methodology
Debray, T. P. A., Moons, K. G. M., van Valkenhoef, G., Efthimiou, O., Hummel, N., Groenwold, R. H. H., Reitsma, J. B., & on behalf of the GetReal methods review group (2015) · Research Synthesis Methods · doi:10.1002/jrsm.1160 · Cited by 348 · see citing works
Self-citationLuigjes‐Huizer, Y. L., Tauber, N. M., Humphris, G., Kasparian, N. A., Lam, W. W. T., Lebel, S., Simard, S., Smith, A. B., Zachariae, R., Afiyanti, Y., et al. (2022) · Psycho-Oncology · doi:10.1002/pon.5921 · Cited by 332 · see citing works
“>10 years 194 (44) 51 (11) 109 (24) 91 (21) 33 (37) 14 (15) 28 (30) 17 (18)”
Rozental, A., Magnusson, K., Boettcher, J., Andersson, G., & Carlbring, P. (2017) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000158 · Cited by 137 · see citing works
Methods to Address Confounding and Other Biases in Meta-Analyses: Review and Recommendations
Mathur, M. B., & VanderWeele, T. J. (2022) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-051920-114020 · Cited by 85 · see citing works
Xu, Y., Derakhshan, A., Hysaj, O., Wildisen, L., Ittermann, T., Pingitore, A., Abolhassani, N., Medici, M., Kiemeney, L. A. L. M., Riksen, N. P., et al. (2023) · The Lancet Diabetes & Endocrinology · doi:10.1016/s2213-8587(23)00227-9 · Cited by 61 · see citing works
Samoli, E., Rodopoulou, S., Hvidtfeldt, U. A., Wolf, K., Stafoggia, M., Brunekreef, B., Strak, M., Chen, J., Andersen, Z. J., Atkinson, R., et al. (2021) · Environment International · doi:10.1016/j.envint.2020.106371 · Cited by 33 · see citing works
Luigjes‐Huizer, Y. L., Helsper, C. W., de Wit, N. J., & van der Lee, M. L. (2023) · Psycho-Oncology · doi:10.1002/pon.6231 · Cited by 18 · see citing works
Elias, M., Bliznyuk, N., Yasova Barbeau, D., Sukumar, S., Carlos Roig, J., Rajderkar, D., Sura, L., Wynn, J. L., & Weiss, M. D. (2026) · Frontiers in pediatrics · doi:10.3389/fped.2026.1743749
- Authors
- Jolani, S., Debray, T. P. A., Koffijberg, H., van Buuren, S., & Moons, K. G. M.
- Affiliations
- Utrecht University (NL)
- University Medical Center Utrecht (NL)
- The Netherlands Organisation for Applied Scientific Research TNO Leiden The Netherlands
- Corresponding author
- Jolani, Shahab
- Venue
- Statistics in Medicine
- Publisher
- Wiley
- Published
- 2015-02-09
- Volume
- 34
- Issue
- 11
- Pages
- 1841-1863
- ISSN
- 0277-6715 (ISSN-L)
- 1097-0258
- OpenAlex
- W1957946432
- Cited by
- 192
- References
- 40
- Citations by year
- 2026: 10 · 2025: 21 · 2024: 16 · 2023: 19 · 2022: 15 · 2021: 22 · 2020: 26 · 2019: 31 · 2018: 11 · 2017: 8 · 2016: 8 · 2015: 4 · 2012: 1
- FWCI
- 9.49
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Statistics in Medicine (publishedVersion)
- PubMed (publishedVersion)
- Research Publications (Maastricht University) (publishedVersion, other-oa)
- Utrecht University Repository (Utrecht University) (submittedVersion, other-oa)
- TNO Repository (submittedVersion)
- University of Twente Research Information (submittedVersion)
- Repository hosted by TU Delft Library (TU Delft) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Research Publications (Maastricht University) (submittedVersion)
- Utrecht University Repository (Utrecht University) (submittedVersion)
- Funders
- Netherlands Organization for Scientific Research (91611126, 91208004, 91810615)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- individual patient data meta-analysis, multiple imputation, mice, study heterogeneity, generalized linear mixed models, prediction model validation, deep vein thrombosis, diagnostic prediction model, complete case analysis
- MeSH terms
- Algorithms, Computer Simulation, Humans, Predictive Value of Tests, Probability, Risk Factors, Meta-Analysis as Topic, Linear Models, Risk Assessment, Venous Thrombosis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
