Comparing the Areas Under Two Correlated ROC Curves: Parametric and Non‐Parametric Approaches
Abstract
In order to compare the discriminatory effectiveness of two diagnostic markers the equality of the areas under the respective Receiver Operating Characteristic Curves is commonly tested. A non‐parametric test based on the Mann‐Whitney statistic is generally used. Weiand et al. (1989) present a parametric test based on normal distributional assumptions. We extend this test using the Box‐Cox power family of transformations to non‐normal situations. These three test procedures are compared in terms of significance level and power by means of a large simulation study. Overall we find that transforming to normality is to be preferred. An example of two pancreatic cancer serum biomarkers is used to illustrate the methodology. (© 2006 WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim)
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Areas under the ROC-curve were compared using the DeLong-test as described in Molodianovitch et al. (2006).
“Areas under the ROC-curve were compared using the DeLong-test as described in42.”
From Medical imaging deep learning with differential privacy (2021) · cites this paper as [42]
Cites
DeLong, E. R., DeLong, D. M., & Clarke-Pearson, D. L. (1988) · Biometrics · doi:10.2307/2531595 · Cited by 22,649 · see citing works
Estimation of the Youden Index and its Associated Cutoff Point
Fluss, R., Faraggi, D., & Reiser, B. (2005) · Biometrical Journal · doi:10.1002/bimj.200410135 · Cited by 2,604 · see citing works
Self-citationThe Statistical Evaluation of Medical Tests for Classification and Prediction
PEPE, M. S. (2003) · doi:10.1093/oso/9780198509844.001.0001 · Cited by 2,206 · see citing works
Greiner, M., Pfeiffer, D., & Smith, R. D. (2000) · Preventive Veterinary Medicine · doi:10.1016/s0167-5877(00)00115-x · Cited by 1,923 · see citing works
The Statistical Evaluation of Medical Tests for Classification and Prediction
PEPE, M. S. (2005) · Technometrics · doi:10.1198/tech.2005.s278 · Cited by 1,907 · see citing works
A Multivariate Exponential Distribution
Marshall, A. W., & Olkin, I. (1967) · Journal of the American Statistical Association · doi:10.1080/01621459.1967.10482885 · Cited by 1,609 · see citing works
Continuous Multivariate Distributions
Kotz, S., Balakrishnan, N., & Johnson, N. L. (2000) · Wiley series in probability and statistics · doi:10.1002/0471722065 · Cited by 1,411 · see citing works
Statistical Methods in Diagnostic Medicine
Zhou, X., Obuchowski, N. A., & McClish, D. K. (2002) · Wiley series in probability and statistics · doi:10.1002/9780470317082 · Cited by 1,181 · see citing works
Some Practical Issues of Experimental Design and Data Analysis in Radiological ROC Studies
Metz, C. E. (1989) · Investigative Radiology · doi:10.1097/00004424-198903000-00012 · Cited by 861 · see citing works
The use of receiver operating characteristic curves in biomedical informatics
Lasko, T. A., Bhagwat, J. G., Zou, K. H., & Lucila Ohno-machado, A. (2005) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2005.02.008 · Cited by 821 · see citing works
Estimation of the area under the ROC curve
Faraggi, D., & Reiser, B. (2002) · Statistics in Medicine · doi:10.1002/sim.1228 · Cited by 440 · see citing works
Self-citationThe interpretation of diagnostic tests
Shapiro, D. E. (1999) · Statistical Methods in Medical Research · doi:10.1177/096228029900800203 · Cited by 375 · see citing works
A Family of Nonparametric Statistics for Comparing Diagnostic Markers with Paired or Unpaired Data
Wieand, S., Gail, M. H., James, B. R., & James, K. L. (1989) · Biometrika · doi:10.1093/biomet/76.3.585 · Cited by 354 · see citing works
Continuous Multivariate Distributions, Volume 1: Models and Applications
Strawderman, R. L. (2001) · Journal of the American Statistical Association · doi:10.1198/jasa.2001.s405 · Cited by 337 · see citing works
Hsiao, J. (1989) · Archives of General Psychiatry · doi:10.1001/archpsyc.1989.01810070090014 · Cited by 270 · see citing works
Effects of dependent errors in the assessment of diagnostic test performance
Torrance-Rynard, V., & Walter, S. D. (1997) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19971015)16:19 · Cited by 157 · see citing works
Receiver operator characteristic (ROC) curves and non‐normal data: An empirical study
Goddard, M. J., & Hinberg, I. (1990) · Statistics in Medicine · doi:10.1002/sim.4780090315 · Cited by 123 · see citing works
Two transformation models for estimating an ROC curve derived from continuous data
Zou, K. H., & Hall, W. J. (2000) · Journal of Applied Statistics · doi:10.1080/02664760050076443 · Cited by 112 · see citing works
Statistical Tests Based on Transformed Data
Doksum, K. A., & Wong, C.-W. (1983) · Journal of the American Statistical Association · doi:10.1080/01621459.1983.10477986 · Cited by 72 · see citing works
Confidence Intervals for the Generalized ROC Criterion
Reiser, B., & Faraggi, D. (1997) · Biometrics · doi:10.2307/2533964 · Cited by 67 · see citing works
Self-citationAdjusting the generalized ROC curve for covariates
Schisterman, E. F., Faraggi, D., & Reiser, B. (2004) · Statistics in Medicine · doi:10.1002/sim.1908 · Cited by 66 · see citing works
Self-citationAoki, K., Misumi, J., Kimura, T., et al. (1997) · Journal of Epidemiology · doi:10.2188/jea.7.143 · Cited by 51 · see citing works
ROC analysis for markers with mass at zero
Schisterman, E. F., Reiser, B., & Faraggi, D. (2005) · Statistics in Medicine · doi:10.1002/sim.2301 · Cited by 34 · see citing works
Self-citationThe INFO-ROC Technique: A Method for Comparing and Optimizing Inspection Systems
Somoza, E. C., Mossman, D., & McFeeters, L. (1990) · Review of Progress in Quantitative Nondestructive Evaluation · doi:10.1007/978-1-4684-5772-8_75 · Cited by 20 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Medical imaging deep learning with differential privacy
Ziller, A., Usynin, D., Braren, R., Makowski, M., Rueckert, D., & Kaissis, G. (2021) · Scientific Reports · doi:10.1038/s41598-021-93030-0 · Cited by 172 · see citing works
- Authors
- Molodianovitch, K., Faraggi, D., & Reiser, B.
- Affiliations
- University of Haifa (IL)
- Corresponding author
- Reiser, Benjamin
- Venue
- Biometrical Journal
- Publisher
- Wiley
- Published
- 2006-10
- Volume
- 48
- Issue
- 5
- Pages
- 745-757
- ISSN
- 0323-3847 (ISSN-L)
- 1521-4036
- OpenAlex
- W2035781406
- Cited by
- 131
- References
- 24
- Citations by year
- 2026: 13 · 2025: 11 · 2024: 9 · 2023: 10 · 2022: 12 · 2021: 12 · 2020: 8 · 2019: 5 · 2018: 5 · 2017: 4 · 2016: 3 · 2015: 4 · 2014: 5 · 2013: 4 · 2012: 3
- FWCI
- 4.09
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Biometrical Journal (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
- Physical Sciences › Mathematics › Statistics and Probability › Probability and Statistical Research
- Keywords
- ROC curve comparison, area under the ROC curve, diagnostic biomarkers, Mann-Whitney test, Box-Cox transformation, parametric tests, nonparametric tests, normality assumption, statistical power, significance level, pancreatic cancer biomarkers, serum biomarkers
- MeSH terms
- Computer Simulation, Data Interpretation, Statistical, Humans, Pancreatic Neoplasms, ROC Curve, Biomarkers, CA-125 Antigen, CA-19-9 Antigen, Statistics, Nonparametric, Area Under Curve
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
