The DOI matches its Crossref record; 65 of the 65 references it cites were found in a registry or an index.
Bivariate Random Effects Meta-Analysis of ROC Curves
Abstract
Meta-analysis of receiver operating characteristic (ROC)-curve data is often done with fixed-effects models, which suffer many shortcomings. Some random-effects models have been proposed to execute a meta-analysis of ROC-curve data, but these models are not often used in practice. Straightforward modeling techniques for multivariate random-effects meta-analysis of ROC-curve data are needed. The 1st aim of this article is to present a practical method that addresses the drawbacks of the fixedeffects summary ROC (SROC) method of Littenberg and Moses. Sensitivities and specificities are analyzed simultaneously using a bivariate random-effects model. The 2nd aim is to show that other SROC curves can also be derived from the bivariate model through different characterizations of the estimated bivariate normal distribution. Thereby the authors show that the bivariate random-effects approach not only extends the SROC approach but also provides a unifying framework for other approaches. The authors bring the statistical meta-analysis of ROC-curve data back into a framework of relatively standard multivariate meta-analysis with random effects. The analyses were carried out using the software package SAS (Proc NLMIXED).
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Measuring inconsistency in meta-analyses
Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003) · BMJ · doi:10.1136/bmj.327.7414.557 · Cited by 63,139
Meta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552
Applied Multivariate Statistical Analysis.
Johnson, A. R., & Wichern, D. W. (1988) · Biometrics · doi:10.2307/2531616 · Cited by 11,353
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2021) · doi:10.1002/9781119558378 · Cited by 9,799
Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine
Zweig, M. H., & Campbell, G. (1993) · Clinical Chemistry · doi:10.1093/clinchem/39.4.561 · Cited by 6,569
Primary, Secondary, and Meta-Analysis of Research
GLASS, G. V. (1976) · Educational Researcher · doi:10.3102/0013189x005010003 · Cited by 4,890
Reitsma, J. B., Glas, A. S., Rutjes, A. W. S., Scholten, R. J. P. M., Bossuyt, P. M., & Zwinderman, A. H. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2005.02.022 · Cited by 3,369
(2000) · Queen Mary Research Online (Queen Mary University of London) · Cited by 1,795
Measurement Error in Nonlinear Models
Carroll, R. J., Ruppert, D., & Stefanski, L. A. (1995) · Springer eBooks · doi:10.1007/978-1-4899-4477-1 · Cited by 1,749
Advanced methods in meta‐analysis: multivariate approach and meta‐regression
van Houwelingen, H. C., Arends, L. R., & Stijnen, T. (2002) · Statistics in Medicine · doi:10.1002/sim.1040 · Cited by 1,580
Self-citationApplied Multivariate Statistical Analysis
Ziegel, E. R., Johnson, R. A., & Wichern, D. W. (1999) · Technometrics · doi:10.2307/1271011 · Cited by 1,547
A hierarchical regression approach to meta‐analysis of diagnostic test accuracy evaluations
Rutter, C. M., & Gatsonis, C. A. (2001) · Statistics in Medicine · doi:10.1002/sim.942 · Cited by 1,505
(1998) · Spoken Language System and Corpus Design · doi:10.1515/9783110809817.1 · Cited by 1,488
Moses, L. E., Shapiro, D., & Littenberg, B. (1993) · Statistics in Medicine · doi:10.1002/sim.4780121403 · Cited by 1,405
Systematic reviews of evaluations of diagnostic and screening tests
Deeks, J. J. (2001) · BMJ · doi:10.1136/bmj.323.7305.157 · Cited by 1,340
Systematic Review: Why sources of heterogeneity in meta-analysis should be investigated
Thompson, S. G. (1994) · BMJ · doi:10.1136/bmj.309.6965.1351 · Cited by 1,017
WinBUGS user manual version 1.4
Spiegelhalter, D. J. (2003) · Medical Entomology and Zoology · Cited by 983
Chu, H., & Cole, S. R. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.06.011 · Cited by 708
Littell, R. C., Stroup, W. W., & Freund, R. J. (2002) · Cited by 687
Properties of the summary receiver operating characteristic (SROC) curve for diagnostic test data
Walter, S. D. (2002) · Statistics in Medicine · doi:10.1002/sim.1099 · Cited by 672
ROC Curves for Continuous Data
Krzanowski, W. J., & Hand, D. J. (2009) · Chapman & Hall/CRC eBooks · doi:10.1201/9781439800225 · Cited by 666
Meta-analysis of screening and diagnostic tests.
Hasselblad, V., & Hedges, L. V. (1995) · Psychological Bulletin · doi:10.1037/0033-2909.117.1.167 · Cited by 579
Meta-analytic methods for diagnostic test accuracy
Irwig, L., Macaskill, P., Glasziou, P., & Fahey, M. (1995) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(94)00099-c · Cited by 467
Estimating Diagnostic Accuracy from Multiple Conflicting Reports: A New Meta-analytic Method
Littenberg, B., & Moses, L. E. (1993) · Medical Decision Making · doi:10.1177/0272989x9301300408 · Cited by 434
Nederkoorn, P. J., van der Graaf, Y., & Hunink, M. G. M. (2003) · Stroke · doi:10.1161/01.str.0000068367.08991.a2 · Cited by 382
Self-citationMR Imaging of the Menisci and Cruciate Ligaments: A Systematic Review
Oei, E. H. G., Nikken, J. J., Verstijnen, A. C. M., Ginai, A. Z., & Myriam Hunink, M. G. (2003) · Radiology · doi:10.1148/radiol.2263011892 · Cited by 371
A bivariate approach to meta-analysis
Van Houwelingen, H. C., Zwinderman, K. H., & Stijnen, T. (1993) · Statistics in Medicine · doi:10.1002/sim.4780122405 · Cited by 327
Self-citationRadiological Evaluation of Lymph Node Metastases in Patients With Cervical Cancer: A Meta-analysis
Scheidler, J. (1997) · JAMA · doi:10.1001/jama.1997.03550130070040 · Cited by 320
Meta-analysis of multiple outcomes by regression with random effects
Berkey, C. S., Hoaglin, D. C., Antczak-Bouckoms, A., Mosteller, F., & Colditz, G. A. (1998) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19981130)17:22<2537::aid-sim953>3.0.co;2-c · Cited by 280
A multivariate mixed linear model for meta-analysis.
Kalaian, H. A., & Raudenbush, S. W. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.3.227 · Cited by 229
Macaskill, P. (2004) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2003.12.019 · Cited by 210
Visser, K., & Myriam Hunink, M. G. (2000) · Radiology · doi:10.1148/radiology.216.1.r00jl0367 · Cited by 187
A Handbook of Statistical Analyses using SAS
Der, G., & Everitt, B. S. (2008) · Chapman & Hall/CRC eBooks · doi:10.1201/9781584887850 · Cited by 166
(1985) · Annals of Internal Medicine · doi:10.7326/0003-4819-103-1-147 · Cited by 160
Ariga, R., Bloom, K., Reddy, V., Kluskens, L. F., Francescatti, D., Dowlat, K., Siziopikou, P., & Gattuso, P. (2002) · The American Journal of Surgery · doi:10.1016/s0002-9610(02)01014-0 · Cited by 141
Giard, R. W. M., & Hermans, J. (1992) · Cancer · Cited by 135
Fine-Needle Aspiration of the Breast
Grant, C. S., Goellner, J. R., WELCH, J. S., & Martin, J. K. (1986) · Mayo Clinic Proceedings · doi:10.1016/s0025-6196(12)61956-9 · Cited by 127
Meta-Analysis of Diagnostic Tests With Imperfect Reference Standards
Walter, S. D., Irwig, L., & Glasziou, P. (1999) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(99)00086-4 · Cited by 120
Meta‐analysis of Diagnostic Test Accuracy Assessment Studies with Varying Number of Thresholds
Dukić, V., & Gatsonis, C. (2003) · Biometrics · doi:10.1111/j.0006-341x.2003.00108.x · Cited by 118
Kaufman, Z., Shpitz, B., Shapiro, M., Rona, R., Lew, S., & Dinbar, A. (1994) · Journal of Surgical Oncology · doi:10.1002/jso.2930560413 · Cited by 117
AN ALTERNATIVE TO THE USE OF TWO-SIDED TESTS IN CLINICAL TRIALS
Dunnett, C. W., & Gent, M. (1996) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19960830)15:16<1713::aid-sim331>3.0.co;2-d · Cited by 110
Kester, A. D. M., & Buntinx, F. (2000) · Medical Decision Making · doi:10.1177/0272989x0002000407 · Cited by 101
Chaiwun, B., Settakorn, J., Ya‐In, C., Wisedmongkol, W., Rangdaeng, S., & Thorner, P. S. (2002) · Diagnostic Cytopathology · doi:10.1002/dc.10067 · Cited by 96
Senn, S. J., & Brand, R. (1994) · Statistics in Medicine · doi:10.1002/sim.4780130310 · Cited by 91
Silverman, J. F., Lannin, D. R., O'Brien, K. F., & Ht, N. (1987) · PubMed · Cited by 90
Kardaun, J. W. P. F., & Kardaun, O. J. W. F. (1990) · Methods of Information in Medicine · doi:10.1055/s-0038-1634759 · Cited by 81
Baseline risk as predictor of treatment benefit: three clinical meta-re-analyses
Arends, L. R., Hoes, A. W., Lubsen, J. C., Grobbee, D. E., & Stijnen, T. (2000) · Statistics in Medicine · doi:10.1002/1097-0258(20001230)19:24<3497::aid-sim830>3.0.co;2-h · Cited by 80
Self-citationCombining multiple outcome measures in a meta‐analysis: an application
Arends, L. R., Vokó, Z., & Stijnen, T. (2003) · Statistics in Medicine · doi:10.1002/sim.1370 · Cited by 74
Self-citationCombining and Comparing Area Estimates across Studies or Strata
McClish, D. K. (1992) · Medical Decision Making · doi:10.1177/0272989x9201200405 · Cited by 71
Field, A. P. (2003) · Sussex Research Online (University of Sussex) · Cited by 69
Bayesian Biostatistics and Diagnostic Medicine
Broemeling, L. D. (2007) · Chapman & Hall/CRC eBooks · doi:10.1201/9781584887683 · Cited by 67
Bivariate Random-effects Meta-analysis of Sensitivity and Specificity with SAS PROC GLIMMIX
Menke, J. (2009) · Methods of Information in Medicine · doi:10.3414/me09-01-0001 · Cited by 51
Fine Needle Aspiration of the Breast
Wang, H. H., & Ducatman, B. S. (1998) · Acta Cytologica · doi:10.1159/000331604 · Cited by 50
Accuracy of aspiration cytology in detecting breast cancer.
Norton, L. W., Davis, J. R., Wiens, J., Trego, D. C., & Dunnington, G. L. (1984) · PubMed · Cited by 45
Fine needle aspiration of breast masses. An analysis of 276 aspirates.
Ej, W., Cm, S., Cm, F., Da, F., & Ki, B. (1989) · PubMed · Cited by 41
Fine Needle Aspiration of Breast Masses
Domínguez, F., Riera, J. R. M., Tojo, S., & Junco, P. (1997) · Acta Cytologica · doi:10.1159/000332522 · Cited by 37
Bayesian Approaches to Meta-analysi of ROC Curves
Hellmich, M., Abrams, K. R., & Sutton, A. J. (1999) · Medical Decision Making · doi:10.1177/0272989x9901900304 · Cited by 34
Meta‐analysis of binary data: which within study variance estimate to use?
Chang, B., WATERNAUX, C. M., & Lipsitz, S. R. (2001) · Statistics in Medicine · doi:10.1002/sim.823 · Cited by 34
Akçil, M., Karaağaoğlu, E., & Demirhan, B. (2008) · Diagnostic Cytopathology · doi:10.1002/dc.20809 · Cited by 23
Blettner, M., & Schlattmann, P. (2005) · Springer eBooks · doi:10.1007/978-3-540-26577-1_21 · Cited by 21
Ha, G., Hsiu, J.-G., Jt, M., Jaffe, A. H., & D'Amato, N. A. (1990) · PubMed · Cited by 16
Toft, N., & Nielsen, S. S. (2009) · Preventive Veterinary Medicine · doi:10.1016/j.prevetmed.2009.08.019 · Cited by 11
Assessment of diagnostic imaging technologies for cardiovascular disease
Heijenbrok‐Kal, M. H. (2004) · EUR Research Repository (Erasmus University Rotterdam) · Cited by 3
Self-citationMeta-analysis: formulating, evaluating, combining, and reporting
Normand, S.-L. T. (1999) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19990215)18:3<321::aid-sim28>3.0.co;2-p
Detecting and describing heterogeneity in meta-analysis
Hardy, R. J., & Thompson, S. G. (1998) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19980430)17:8<841::aid-sim781>3.0.co;2-d
Cited by
Systematic Reviews of Diagnostic Test Accuracy
Leeflang, M. M. G., Deeks, J. J., Gatsonis, C., & Bossuyt, P. M. M. (2008) · Annals of Internal Medicine · doi:10.7326/0003-4819-149-12-200812160-00008 · Cited by 1,124
Multivariate meta-analysis for non-linear and other multi-parameter associations
Gasparrini, A., Armstrong, B., & Kenward, M. G. (2012) · Statistics in Medicine · doi:10.1002/sim.5471 · Cited by 772
“The methodology illustrated here is not limited to model multi-parameterized non-linear exposure– response dependencies: investigators have also applied multivariate meta-analysis to synthesize survival curves [30, 65], longitudinal profiles [66], ROC curves [67] and heat wave effects [68].”
Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy
(2023) · doi:10.1002/9781119756194 · Cited by 757
Stijnen, T., Hamza, T. H., & Özdemir, P. (2010) · Statistics in Medicine · doi:10.1002/sim.4040 · Cited by 757
Self-citationNiekel, M. C., Bipat, S., & Stoker, J. (2010) · Radiology · doi:10.1148/radiol.10100729 · Cited by 584
Metadata
- Authors
- Arends, L. R., Hamza, T. H., van Houwelingen, J. C., Heijenbrok-Kal, M. H., Hunink, M. G. M., & Stijnen, T.
- Venue
- Medical Decision Making
- Publisher
- SAGE Publications
- Published
- 2008-06-30
- Volume
- 28
- Issue
- 5
- Pages
- 621-638
- ISSN
- 0272-989X (ISSN-L)
- 1552-681X
- OpenAlex
- W2096014593
- Cited by
- 325
- References
- 65
- FWCI
- 4.91
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.