Random effects meta-analysis of event outcome in the framework of the generalized linear mixed model with applications in sparse data
Abstract
We consider random effects meta-analysis where the outcome variable is the occurrence of some event of interest. The data structures handled are where one has one or more groups in each study, and in each group either the number of subjects with and without the event, or the number of events and the total duration of follow-up is available. Traditionally, the meta-analysis follows the summary measures approach based on the estimates of the outcome measure(s) and the corresponding standard error(s). This approach assumes an approximate normal within-study likelihood and treats the standard errors as known. This approach has several potential disadvantages, such as not accounting for the standard errors being estimated, not accounting for correlation between the estimate and the standard error, the use of an (arbitrary) continuity correction in case of zero events, and the normal approximation being bad in studies with few events. We show that these problems can be overcome in most cases occurring in practice by replacing the approximate normal within-study likelihood by the appropriate exact likelihood. This leads to a generalized linear mixed model that can be fitted in standard statistical software. For instance, in the case of odds ratio meta-analysis, one can use the non-central hypergeometric distribution likelihood leading to mixed-effects conditional logistic regression. For incidence rate ratio meta-analysis, it leads to random effects logistic regression with an offset variable. We also present bivariate and multivariate extensions. We present a number of examples, especially with rare events, among which an example of network meta-analysis.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Treating the total number of observed events as count data in a one-stage random-effects model is an alternative approach.
“Alternatively, we may treat the total number of observed events as count data in a one-stage random-effects model43”
From A framework for meta-analysis of prediction model studies with binary and time-to-event outcomes (2019) · cites this paper as [43]
A one-stage random-effects model with a binomial likelihood avoids continuity corrections by explicitly accounting for sampling error in observed event rates.
“One approach to avoid continuity corrections is to explicitly account for sampling error in the observed event rates by modelling the binomial likelihood directly in a one-stage random-effects model”
From A framework for meta-analysis of prediction model studies with binary and time-to-event outcomes (2019) · cites this paper as [43]
Cites
Meta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552 · see citing works
Approximate Inference in Generalized Linear Mixed Models
Breslow, N. E., & Clayton, D. G. (1993) · Journal of the American Statistical Association · doi:10.1080/01621459.1993.10594284 · Cited by 4,595 · see citing works
Random-effects model for meta-analysis of clinical trials: An update
DerSimonian, R., & Kacker, R. (2007) · Contemporary Clinical Trials · doi:10.1016/j.cct.2006.04.004 · Cited by 2,374 · see citing works
Combination of direct and indirect evidence in mixed treatment comparisons
Lu, G., & Ades, A. E. (2004) · Statistics in Medicine · doi:10.1002/sim.1875 · Cited by 2,149 · see citing works
Covariance Analysis of Censored Survival Data
Breslow, N. E. (1974) · Biometrics · doi:10.2307/2529620 · Cited by 1,778 · see citing works
What to add to nothing? Use and avoidance of continuity corrections in meta‐analysis of sparse data
J. Sweeting, M., J. Sutton, A., & C. Lambert, P. (2004) · Statistics in Medicine · doi:10.1002/sim.1761 · Cited by 1,734 · see citing works
Advanced methods in meta‐analysis: multivariate approach and meta‐regression
van Houwelingen, H. C., Arends, L. R., & Stijnen, T. (2002) · Statistics in Medicine · doi:10.1002/sim.1040 · Cited by 1,574 · see citing works
Self-citationMuch ado about nothing: a comparison of the performance of meta-analytical methods with rare events
Bradburn, M. J., Deeks, J. J., Berlin, J. A., & Russell Localio, A. (2007) · Statistics in Medicine · doi:10.1002/sim.2528 · Cited by 938 · see citing works
Chu, H., & Cole, S. R. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.06.011 · Cited by 708 · see citing works
Bayesian approaches to random-effects meta-analysis: a comparative study
Smith, T. C., Spiegelhalter, D. J., & Thomas, A. (1995) · Statistics in Medicine · doi:10.1002/sim.4780142408 · Cited by 568 · see citing works
A LIKELIHOOD APPROACH TO META-ANALYSIS WITH RANDOM EFFECTS
Hardy, R., & Thompson, S. G. (1996) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19960330)15:6 · Cited by 517 · see citing works
Lambert, P. C., Sutton, A. J., Burton, P. R., Abrams, K. R., & Jones, D. R. (2005) · Statistics in Medicine · doi:10.1002/sim.2112 · Cited by 511 · see citing works
An Assessment of Estimation Procedures for Multilevel Models with Binary Responses
Rodrı́guez, G., & Goldman, N. (1995) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.2307/2983404 · Cited by 480 · see citing works
Friedrich, J. O., Adhikari, N. K. J., & Beyene, J. (2007) · BMC Medical Research Methodology · doi:10.1186/1471-2288-7-5 · Cited by 460 · see citing works
metandi: Meta-analysis of diagnostic accuracy using hierarchical logistic regression
Harbord, R. M., & Whiting, P. (2009) · The Stata Journal: Promoting communications on statistics and Stata · doi:10.1177/1536867x0900900203 · Cited by 453 · cited 1× · see citing works
“There is some discussion in the literature on whether and how double-zero trials should be included in a meta-analysis, see for instance references [7] and [32].”
BackgroundApproximations for Standard Errors of Estimators of Fixed and Random Effects in Mixed Linear Models
Kackar, R. N., & Harville, D. A. (1984) · Journal of the American Statistical Association · doi:10.1080/01621459.1984.10477102 · Cited by 445 · see citing works
Statistical Models in Epidemiology
Leger, A. S. S. (1994) · Journal of Epidemiology & Community Health · doi:10.1136/jech.48.6.607 · Cited by 428 · see citing works
The binomial distribution of meta-analysis was preferred to model within-study variability
Hamza, T. H., van Houwelingen, H. C., & Stijnen, T. (2007) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.03.016 · Cited by 388 · see citing works
Self-citationA bivariate approach to meta-analysis
Van Houwelingen, H. C., Zwinderman, K. H., & Stijnen, T. (1993) · Statistics in Medicine · doi:10.1002/sim.4780122405 · Cited by 327 · see citing works
Self-citationBivariate Random Effects Meta-Analysis of ROC Curves
Arends, L. R., Hamza, T. H., van Houwelingen, H. C., Heijenbrok‐Kal, M. H., Hunink, M. G. M., & Stijnen, T. (2008) · Medical Decision Making · doi:10.1177/0272989x08319957 · Cited by 325 · see citing works
Self-citationDevelopment, feasibility and performance of a health risk appraisal questionnaire for older persons
Stuck, A. E., Kharicha, K., Dapp, U., Anders, J., von Renteln-Kruse, W., Meier-Baumgartner, H. P., Harari, D., Swift, C. G., Ivanova, K., Egger, M., et al. (2007) · BMC Medical Research Methodology · doi:10.1186/1471-2288-7-1 · Cited by 324 · see citing works
Pinheiro, J. C., & Chao, E. C. (2006) · Journal of Computational and Graphical Statistics · doi:10.1198/106186006x96962 · Cited by 289 · see citing works
Bivariate random-effects meta-analysis and the estimation of between-study correlation
Riley, R. D., Abrams, K. R., Sutton, A. J., Lambert, P. C., & Thompson, J. R. (2007) · BMC Medical Research Methodology · doi:10.1186/1471-2288-7-3 · Cited by 243 · see citing works
Investigating underlying risk as a source of heterogeneity in meta-analysis
Thompson, S. G., Smith, T. C., & Sharp, S. J. (1997) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19971215)16:23 · Cited by 227 · see citing works
Alzheimer Disease: Operating Characteristics of PET— A Meta-Analysis
Patwardhan, M. B., McCrory, D. C., Matchar, D. B., Samsa, G. P., & Rutschmann, O. (2004) · Radiology · doi:10.1148/radiol.2311021620 · Cited by 137 · see citing works
An assessment of the use of the continuity correction for sparse data in meta-analysis
Sankey, S. S., Weissfeld, L. A., Fine, M. J., & Kapoor, W. N. (1996) · Communications in Statistics - Simulation and Computation · doi:10.1080/03610919608813357 · Cited by 123 · see citing works
Some general points in estimating heterogeneity variance with the DerSimonian-Laird estimator
Böhning, D. (2002) · Biostatistics · doi:10.1093/biostatistics/3.4.445 · Cited by 122 · see citing works
Yafune, A., & Ishiguro, M. (1999) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19990315)18:5 · Cited by 109 · see citing works
Shuster, J. J., Jones, L. S., & Salmon, D. A. (2007) · Statistics in Medicine · doi:10.1002/sim.3060 · Cited by 105 · see citing works
Trying to be precise about vagueness.
Senn, S. (2007) · Statistics in Medicine · doi:10.1002/sim.2639 · Cited by 105 · see citing works
Combining multiple outcome measures in a meta‐analysis: an application
Arends, L. R., Vokó, Z., & Stijnen, T. (2003) · Statistics in Medicine · doi:10.1002/sim.1370 · Cited by 74 · see citing works
Self-citationRestricted maximum likelihood estimation of a common mean and the Mandel–Paule algorithm
Rukhin, A. L., Biggerstaff, B. J., & Vangel, M. G. (2000) · Journal of Statistical Planning and Inference · doi:10.1016/s0378-3758(99)00098-1 · Cited by 66 · see citing works
Anti-infective-treated central venous catheters: a systematic review of randomized controlled trials
Niël-Weise, B. S., Stijnen, T., & van den Broek, P. J. (2007) · Intensive Care Medicine · doi:10.1007/s00134-007-0897-3 · Cited by 41 · see citing works
Self-citationMeta‐analysis of binary data: which within study variance estimate to use?
Chang, B., WATERNAUX, C. M., & Lipsitz, S. R. (2001) · Statistics in Medicine · doi:10.1002/sim.823 · Cited by 32 · see citing works
Niël-Weise, B. S., Stijnen, T., & van den Broek, P. J. (2008) · Journal of Hospital Infection · doi:10.1016/j.jhin.2008.02.020 · Cited by 28 · see citing works
Self-citationHauschke (1999) · doi:10.1002/(sici)1097-0258(19990115)18:1
Meta-analysis: formulating, evaluating, combining, and reporting
Normand, S.-L. T. (1999) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19990215)18:3
Hahn, S., Williamson, P. R., Hutton, J. L., Garner, P., & Flynn, E. V. (2000) · Statistics in Medicine · doi:10.1002/1097-0258(20001230)19:24
Flagged first, then by citations
Cited by
Sorted by citations
Global Prevalence of Diabetic Retinopathy and Projection of Burden through 2045
Teo, Z. L., Tham, Y.-C., Yu, M., Chee, M. L., Rim, T. H., Cheung, N., Bikbov, M. M., Wang, Y. X., Tang, Y., Lu, Y., et al. (2021) · Ophthalmology · doi:10.1016/j.ophtha.2021.04.027 · Cited by 2,420 · see citing works
Methods to estimate the between‐study variance and its uncertainty in meta‐analysis
Veroniki, A. A., Jackson, D., Viechtbauer, W., Bender, R., Bowden, J., Knapp, G., Kuss, O., Higgins, J. P., Langan, D., & Salanti, G. (2016) · Research Synthesis Methods · doi:10.1002/jrsm.1164 · Cited by 1,685 · see citing works
A revised airway epithelial hierarchy includes CFTR-expressing ionocytes
Montoro, D. T., Haber, A. L., Biton, M., Vinarsky, V., Lin, B., Birket, S. E., Yuan, F., Chen, S., Leung, H. M., Villoria, J., et al. (2018) · Nature · doi:10.1038/s41586-018-0393-7 · Cited by 1,206 · see citing works
Wang, H., Li, N., Chivese, T., Werfalli, M., Sun, H., Yuen, L., Hoegfeldt, C. A., Elise Powe, C., Immanuel, J., Karuranga, S., et al. (2022) · Diabetes Research and Clinical Practice · doi:10.1016/j.diabres.2021.109050 · Cited by 1,260 · see citing works
White, I. R., Barrett, J. K., Jackson, D., & Higgins, J. P. T. (2012) · Research Synthesis Methods · doi:10.1002/jrsm.1045 · Cited by 1,114 · see citing works
A comparison of heterogeneity variance estimators in simulated random‐effects meta‐analyses
Langan, D., Higgins, J. P. T., Jackson, D., Bowden, J., Veroniki, A. A., Kontopantelis, E., Viechtbauer, W., & Simmonds, M. (2019) · Research Synthesis Methods · doi:10.1002/jrsm.1316 · Cited by 1,074 · see citing works
Cénat, J. M., Blais-Rochette, C., Kokou-Kpolou, C. K., Noorishad, P.-G., Mukunzi, J. N., McIntee, S.-E., Dalexis, R. D., Goulet, M.-A., & Labelle, P. R. (2021) · Psychiatry Research · doi:10.1016/j.psychres.2020.113599 · Cited by 952 · see citing works
The prevalence of loneliness across 113 countries: systematic review and meta-analysis
Surkalim, D. L., Luo, M., Eres, R., Gebel, K., van Buskirk, J., Bauman, A., & Ding, D. (2022) · BMJ · doi:10.1136/bmj-2021-067068 · Cited by 821 · see citing works
Schwarzer, G., Chemaitelly, H., Abu‐Raddad, L. J., & Rücker, G. (2019) · Research Synthesis Methods · doi:10.1002/jrsm.1348 · Cited by 625 · see citing works
Power analysis for random-effects meta-analysis
Jackson, D., & Turner, R. (2017) · Research Synthesis Methods · doi:10.1002/jrsm.1240 · Cited by 619 · see citing works
PRISMA harms checklist: improving harms reporting in systematic reviews
Zorzela, L., Loke, Y. K., Ioannidis, J. P., Golder, S., Santaguida, P., Altman, D. G., Moher, D., Vohra, S., & PRISMA harms group (2016) · BMJ · doi:10.1136/bmj.i157 · Cited by 557 · see citing works
Navarese, E. P., Robinson, J. G., Kowalewski, M., Kolodziejczak, M., Andreotti, F., Bliden, K., Tantry, U., Kubica, J., Raggi, P., & Gurbel, P. A. (2018) · JAMA · doi:10.1001/jama.2018.2525 · Cited by 549 · see citing works
Burke, D. L., Ensor, J., & Riley, R. D. (2017) · Statistics in Medicine · doi:10.1002/sim.7141 · Cited by 538 · see citing works
<b>netmeta</b> : An <i>R</i> Package for Network Meta-Analysis Using Frequentist Methods
Balduzzi, S., Rücker, G., Nikolakopoulou, A., Papakonstantinou, T., Salanti, G., Efthimiou, O., & Schwarzer, G. (2023) · Journal of Statistical Software · doi:10.18637/jss.v106.i02 · Cited by 532 · see citing works
Carter, G., Milner, A., McGill, K., Pirkis, J., Kapur, N., & Spittal, M. J. (2017) · British Journal of Psychiatry · doi:10.1192/bjp.bp.116.182717 · Cited by 361 · see citing works
One-stage dose–response meta-analysis for aggregated data
Crippa, A., Discacciati, A., Bottai, M., Spiegelman, D., & Orsini, N. (2019) · Statistical Methods in Medical Research · doi:10.1177/0962280218773122 · Cited by 365 · see citing works
Meta-analysis of proportions using generalized linear mixed models.
Lin, L., & Chu, H. (2020) · Epidemiology · doi:10.1097/ede.0000000000001232 · Cited by 345 · see citing works
Get real in individual participant data (IPD) meta‐analysis: a review of the methodology
Debray, T. P. A., Moons, K. G. M., van Valkenhoef, G., Efthimiou, O., Hummel, N., Groenwold, R. H. H., Reitsma, J. B., & on behalf of the GetReal methods review group (2015) · Research Synthesis Methods · doi:10.1002/jrsm.1160 · Cited by 348 · see citing works
A framework for meta-analysis of prediction model studies with binary and time-to-event outcomes
Debray, T. P., Damen, J. A., Riley, R. D., Snell, K., Reitsma, J. B., Hooft, L., Collins, G. S., & Moons, K. G. (2019) · Statistical Methods in Medical Research · doi:10.1177/0962280218785504 · Cited by 278 · see citing works
Treatment-resistant depression and suicidality
Bergfeld, I. O., Mantione, M., Figee, M., Schuurman, P. R., Lok, A., & Denys, D. (2018) · Journal of Affective Disorders · doi:10.1016/j.jad.2018.04.016 · Cited by 256 · see citing works
Methods for evidence synthesis in the case of very few studies
Bender, R., Friede, T., Koch, A., Kuss, O., Schlattmann, P., Schwarzer, G., & Skipka, G. (2018) · Research Synthesis Methods · doi:10.1002/jrsm.1297 · Cited by 250 · see citing works
Practical guide to the meta-analysis of rare events
Efthimiou, O. (2018) · Evidence Based Mental Health · doi:10.1136/eb-2018-102911 · Cited by 231 · see citing works
Kuss, O. (2015) · Statistics in Medicine · doi:10.1002/sim.6383 · Cited by 206 · see citing works
Zhou, Y., Hou, Y., Shen, J., Mehra, R., Kallianpur, A., Culver, D. A., Gack, M. U., Farha, S., Zein, J., Comhair, S., et al. (2020) · PLoS Biology · doi:10.1371/journal.pbio.3000970 · Cited by 180 · see citing works
AGA technical review on gastric intestinal metaplasia-natural history and clinical outcomes
Gawron, A. J., Shah, S. C., Altayar, O., Davitkov, P., Morgan, D., Turner, K., & Mustafa, R. A. (2020) · Gastroenterology · doi:10.1053/j.gastro.2019.12.001 · Cited by 175 · see citing works
Carter, K., & Worthington, S. (2015) · Journal of Dental Research · doi:10.1177/0022034515581644 · Cited by 141 · see citing works
AGA technical review on gastric intestinal metaplasia-epidemiology and risk factors
Altayar, O., Davitkov, P., Shah, S. C., Gawron, A. J., Morgan, D. R., Turner, K., & Mustafa, R. A. (2020) · Gastroenterology · doi:10.1053/j.gastro.2019.12.002 · Cited by 135 · see citing works
Woodford, R., Spittal, M. J., Milner, A., McGill, K., Kapur, N., Pirkis, J., Mitchell, A., & Carter, G. (2019) · Suicide and Life-Threatening Behavior · doi:10.1111/sltb.12395 · Cited by 92 · see citing works
Schwarzer, G., & Rücker, G. (2022) · Methods in Molecular Biology · doi:10.1007/978-1-0716-1566-9_10 · Cited by 90 · see citing works
Ju, K., Lin, L., Chu, H., Cheng, L.-L., & Xu, C. (2020) · BMC Medical Research Methodology · doi:10.1186/s12874-020-01035-6 · Cited by 61 · see citing works
Park, H. Y., Suh, C. H., Woo, S., Kim, P. H., & Kim, K. W. (2022) · Korean Journal of Radiology · doi:10.3348/kjr.2021.0808 · Cited by 61 · see citing works
Dossa, F., Medeiros, B., Keng, C., Acuna, S. A., & Baxter, N. N. (2020) · Gastrointestinal Endoscopy · doi:10.1016/j.gie.2019.12.047 · Cited by 56 · see citing works
Hoffmann, I., Kohl, M., von Stengel, S., Jakob, F., Kerschan-Schindl, K., Lange, U., Peters, S., Schoene, D., Sieber, C., Thomasius, F., et al. (2023) · Osteoporosis International · doi:10.1007/s00198-022-06592-8 · Cited by 49 · see citing works
Osinga, J. A. J., Derakhshan, A., Feldt-Rasmussen, U., Huang, K., Vrijkotte, T. G. M., Männistö, T., Bassols, J., López-Bermejo, A., Aminorroaya, A., Vafeiadi, M., et al. (2024) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/clinem/dgad564 · Cited by 27 · see citing works
Widyaputri, F., Rogers, S., & Lim, L. (2022) · JAMA Ophthalmology · doi:10.1001/jamaophthalmol.2022.3722 · Cited by 10 · see citing works
Diagnostic accuracy of teledermatology for skin diseases: a systematic review and meta-analysis
Martyin, K., Meznerics, F. A., Bokor, L. A., Szabó, B., Hegyi, P., Kiss, N., & Bánvölgyi, A. (2026) · Frontiers in Medicine · doi:10.3389/fmed.2026.1739592 · Cited by 4 · see citing works
- Authors
- Stijnen, T., Hamza, T. H., & Özdemir, P.
- Affiliations
- Leiden University Medical Center (NL)
- New York State Department of Health (US)
- Corresponding author
- Stijnen, Theo
- Venue
- Statistics in Medicine
- Publisher
- Wiley
- Published
- 2010-09-08
- Volume
- 29
- Issue
- 29
- Pages
- 3046-3067
- ISSN
- 0277-6715 (ISSN-L)
- OpenAlex
- W2148361291
- Cited by
- 757
- References
- 38
- Citations by year
- 2026: 113 · 2025: 69 · 2024: 73 · 2023: 55 · 2022: 71 · 2021: 104 · 2020: 80 · 2019: 41 · 2018: 40 · 2017: 34 · 2016: 15 · 2015: 20 · 2014: 16 · 2013: 12 · 2012: 10
- FWCI
- 2.30
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Statistics in Medicine (publishedVersion)
- PubMed (publishedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Leiden Repository (Leiden University) (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- random-effects meta-analysis, event outcomes, generalized linear mixed models, data sparsity, rare events, random effects logistic regression, incidence rate ratio, odds ratio, bivariate meta-analysis, multivariate meta-analysis, network meta-analysis, exact likelihood
- MeSH terms
- Algorithms, Bayes Theorem, Cerebral Infarction, Computer Simulation, Fibrosis, Hemorrhage, Humans, Meta-Analysis as Topic, Models, Statistical, Incidence, Multivariate Analysis, Binomial Distribution, Normal Distribution, Poisson Distribution, Likelihood Functions, Linear Models, Odds Ratio, Endarterectomy, Carotid, Treatment Outcome, Disease-Free Survival, Controlled Clinical Trials as Topic, Diagnostic Techniques and Procedures
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
