Selecting the best scale for measuring treatment effect in a network meta‐analysis: a case study in childhood nocturnal enuresis
The DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.
Abstract
Dichotomous outcomes in pairwise meta‐analysis are typically summarised using the odds ratio (OR), relative risk (RR) or risk difference (RD). The hazard ratio (HR) may also be used where events occur over time. Choice of scale is often determined by ease of interpretation or mathematical properties of a measure, as there is frequently insufficient power to compare goodness‐of‐fit across different scales. Network meta‐analysis (NMA) combines evidence across a network of treatment comparisons. NMA allows the combination of a greater numbers of trials, so there is greater potential to use goodness‐of‐fit statistics to determine an appropriate scale on which the effects of treatments are additive. In this paper, we explore choice of scale in an NMA of childhood nocturnal enuresis for the outcome ‘failure to achieve 14 days consecutive dry nights’. We compare OR, RR of both the harmful (RR H) and the beneficial (RR B) outcomes, RD and HR. Using a Bayesian framework, the posterior mean residual deviance and deviance information criterion are used to evaluate model fit and selection between the different summary effect measures. We use I 2 to examine within‐comparison heterogeneity for the pairwise analyses. The results suggest that the HR and RR B provide the best fit. We conclude that choice of scale should be based on physiological and epidemiological understanding of the disease process, together with an empirical assessment of model adequacy. HR should be given greater consideration where there is an underlying time‐to‐event process. We demonstrate how results can be transformed to an alternative scale to aid interpretability. Copyright © 2012 John Wiley & Sons, Ltd.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Checking consistency in mixed treatment comparison meta-analysis
Dias, S., Welton, N. J., Caldwell, D. M., & Ades, A. E. (2010) · Statistics in Medicine · doi:10.1002/sim.3767 · Cited by 2,521 · cited 1×
“For each analysis, we also examined leverage plots to help identify any specific data points (trial arms) that were fitting poorly in each model (Dias et al., 2010).”
MethodSelf-citationSimultaneous comparison of multiple treatments: Combining direct and indirect evidence.
Caldwell, D. M., Ades, A. E., & Higgins, J. P. T. (2005) · BMJ · doi:10.1136/bmj.331.7521.897 · Cited by 1,600 · cited 2×
“Network meta-analysis (Higgins and Whitehead, 1996; Lu and Ades, 2004; Caldwell et al., 2005; Salanti et al., 2008), also termed mixed treatment comparisons, refers to ensembles of trial evidence in which direct and indirect evidence on relative treatment effects are pooled.”
BackgroundSelf-citationEvaluation of networks of randomized trials
Salanti, G., Higgins, J. P., Ades, A., & Ioannidis, J. P. (2008) · Statistical Methods in Medical Research · doi:10.1177/0962280207080643 · Cited by 1,078
Self-citationCommentary: Heterogeneity in meta-analysis should be expected and appropriately quantified
Higgins, J. P. T. (2008) · International Journal of Epidemiology · doi:10.1093/ije/dyn204 · Cited by 995 · cited 1×
“We used the I2 statistic (Higgins et al., 2003) to assess statistical heterogeneity as it enables a standardised approach across the five effect measures we evaluate here (Patsopoulos et al., 2008; Higgins, 2008).”
MethodDeeks, J. J. (2002) · Statistics in Medicine · doi:10.1002/sim.1188 · Cited by 660 · cited 6×
“Coory and Jordan (2010) performed a pooled analysis of five such studies, all of which compared RD and OR (Deeks, 2002; Furukawa et al., 2002; Berlin et al., 1989; Engels et al., 2000; DerSimonian and Laird, 1986), but only two also examined the RR (Deeks, 2002; Furukawa et al., 2002).”
BackgroundMethodInfluentialFurukawa, T. A., Guyatt, G. H., & Griffith, L. E. (2002) · International Journal of Epidemiology · doi:10.1093/ije/31.1.72 · Cited by 318
Warn, D. E., Thompson, S. G., & Spiegelhalter, D. J. (2002) · Statistics in Medicine · doi:10.1002/sim.1189 · Cited by 277 · cited 1×
“We also used the approach suggested by Warn et al. (2002) to constrain the log RR and RD estimates within their permissible bounds.”
MethodThe direct use of likelihood for significance testing
Dempster, A. P. (1997) · Statistics and Computing · doi:10.1023/a:1018598421607 · Cited by 220 · cited 1×
“If the model fits well, we would expect the posterior mean of the summed deviance contributions to be close to the number of data points, in this case 79 (Dempster, 1997).”
Method
Cited by
Hutton, B., Salanti, G., Caldwell, D. M., Chaimani, A., Schmid, C. H., Cameron, C., Ioannidis, J. P. A., Straus, S., Thorlund, K., Jansen, J. P., et al. (2015) · Annals of Internal Medicine · doi:10.7326/m14-2385 · Cited by 8,269
Self-citationDias, S., Sutton, A. J., Ades, A. E., & Welton, N. J. (2013) · Medical Decision Making · doi:10.1177/0272989x12458724 · Cited by 1,207
Self-citationDias, S., Welton, N. J., Sutton, A. J., Caldwell, D. M., Lu, G., & Ades, A. E. (2013) · Medical Decision Making · doi:10.1177/0272989x12455847 · Cited by 728
Self-citationEvaluation of inconsistency in networks of interventions
Veroniki, A. A., Vasiliadis, H. S., Higgins, J. P., & Salanti, G. (2013) · International Journal of Epidemiology · doi:10.1093/ije/dys222 · Cited by 576
GetReal in network meta-analysis: a review of the methodology
Efthimiou, O., Debray, T. P. A., van Valkenhoef, G., Trelle, S., Panayidou, K., Moons, K. G. M., Reitsma, J. B., Shang, A., Salanti, G., & on behalf of GetReal Methods Review Group (2016) · Research Synthesis Methods · doi:10.1002/jrsm.1195 · Cited by 401
“Moreover, HR should always be considered as a suitable choice of scale for the case there is an underlying time-to-event process and the proportional hazards assumption is deemed plausible (Caldwell et al., 2012; Soares et al., 2013) (see Section 6.”
BackgroundMethodInfluentialMethods for Population-Adjusted Indirect Comparisons in Health Technology Appraisal
Phillippo, D. M., Ades, A. E., Dias, S., Palmer, S., Abrams, K. R., & Welton, N. J. (2018) · Medical Decision Making · doi:10.1177/0272989x17725740 · Cited by 396
Self-citationAssessing key assumptions of network meta‐analysis: a review of methods
Donegan, S., Williamson, P., D'Alessandro, U., & Tudur Smith, C. (2013) · Research Synthesis Methods · doi:10.1002/jrsm.1085 · Cited by 267
Nurchis, M. C., Altamura, G., Riccardi, M. T., Radio, F. C., Chillemi, G., Bertini, E. S., Garlasco, J., Tartaglia, M., Dallapiccola, B., & Damiani, G. (2023) · Archives of Public Health · doi:10.1186/s13690-023-01112-4 · Cited by 65
Metadata
- Authors
- Caldwell, D. M., Welton, N. J., Dias, S., & Ades, A.
- Venue
- Research Synthesis Methods
- Publisher
- Wiley
- Published
- 2012-06
- Volume
- 3
- Issue
- 2
- Pages
- 126-141
- ISSN
- 1759-2879 (ISSN-L)
- 1759-2887
- OpenAlex
- W1974837708
- Cited by
- 27
- References
- 8
- FWCI
- 4.95
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.