Cited by 22,649Imbalanced Data Classification TechniquesStatistical Methods in EpidemiologyStatistical Methods in Clinical Trials

The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.

Comparing the Areas under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric Approach

DeLong, Elizabeth R., DeLong, David M., Clarke-Pearson, Daniel L.·1988·Biometrics

DOI 10.2307/2531595

Open this paper in CoChat — free account

Abstract

Methods of evaluating and comparing the performance of diagnostic tests are of increasing importance as new tests are developed and marketed. When a test is based on an observed variable that lies on a continuous or graded scale, an assessment of the overall value of the test can be made through the use of a receiver operating characteristic (ROC) curve. The curve is constructed by varying the cutpoint used to determine which values of the observed variable will be considered abnormal and then plotting the resulting sensitivities against the corresponding false positive rates. When two or more empirical curves are constructed based on tests performed on the same individuals, statistical analysis on differences between curves must take into account the correlated nature of the data. This paper presents a nonparametric approach to the analysis of areas under correlated ROC curves, by using the theory on generalized U-statistics to generate an estimated covariance matrix.

Integrity signals

Registry

Crossref · 10.2307/2531595

Cites

12 of 12 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

The meaning and use of the area under a receiver operating characteristic (ROC) curve.
Hanley, J. A., & McNeil, B. J. (1982) · Radiology · doi:10.1148/radiology.143.1.7063747 · Cited by 21,856
Basic principles of ROC analysis
Metz, C. E. (1978) · Seminars in Nuclear Medicine · doi:10.1016/s0001-2998(78)80014-2 · Cited by 6,122
A Class of Statistics with Asymptotically Normal Distribution
Hoeffding, W. (1992) · Springer series in statistics · doi:10.1007/978-1-4612-0919-5_20 · Cited by 1,898
The area above the ordinal dominance graph and the area below the receiver operating characteristic graph
Bamber, D. (1975) · Journal of Mathematical Psychology · doi:10.1016/0022-2496(75)90001-2 · Cited by 1,452
Untitled work
(1982) · Cited by 1,288
Maximum-likelihood estimation of parameters of signal-detection theory and determination of confidence intervals—Rating-method data
Dorfman, D. D., & Alf, E. F. (1969) · Journal of Mathematical Psychology · doi:10.1016/0022-2496(69)90019-4 · Cited by 878
A New Approach for Testing the Significance of Differences Between ROC Curves Measured from Correlated Data
Metz, C. E., Wang, P.-L., & Kronman, H. B. (1984) · Information Processing in Medical Imaging · doi:10.1007/978-94-009-6045-9_25 · Cited by 359
Jackknifing $U$-Statistics
Arvesen, J. N. (1969) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177697287 · Cited by 286
On Some Convergence Properties of UStatistics
Sen, P. K. (1960) · Calcutta Statistical Association Bulletin · doi:10.1177/0008068319600101 · Cited by 182
Surgical management of bowel obstruction in advanced ovarian carcinoma.
Krebs, H. B., & Goplerud, D. R. (1983) · PubMed · Cited by 132

Flagged first, then by citations

Show all 12 references

Sorted by citations

Maximum entropy modeling of species geographic distributions
Phillips, S. J., Anderson, R. P., & Schapire, R. E. (2006) · Ecological Modelling · doi:10.1016/j.ecolmodel.2005.03.026 · Cited by 15,396
FastTree 2 – Approximately Maximum-Likelihood Trees for Large Alignments
Price, M. N., Dehal, P. S., & Arkin, A. P. (2010) · PLoS ONE · doi:10.1371/journal.pone.0009490 · Cited by 14,894
pROC: an open-source package for R and S+ to analyze and compare ROC curves
Robin, X., Turck, N., Hainard, A., Tiberti, N., Lisacek, F., Sanchez, J.-C., & Müller, M. (2011) · BMC Bioinformatics · doi:10.1186/1471-2105-12-77 · Cited by 13,697
Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data
Quan, H., Sundararajan, V., Halfon, P., Fong, A., Burnand, B., Luthi, J.-C., Saunders, L. D., Beck, C. A., Feasby, T. E., & Ghali, W. A. (2005) · Medical Care · doi:10.1097/01.mlr.0000182534.19832.83 · Cited by 10,669
Acute Respiratory Distress Syndrome
Ards Definition Task Force (2012) · JAMA · doi:10.1001/jama.2012.5669 · Cited by 9,066
Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine
ZWEIG, M. H., & Campbell, G. (1993) · Clinical Chemistry · doi:10.1093/clinchem/39.4.561 · Cited by 6,595
Evaluating the added predictive ability of a new marker: From area under the ROC curve to reclassification and beyond
Pencina, M. J., D' Agostino, R. B., D' Agostino, R. B., & Vasan, R. S. (2008) · Statistics in Medicine · doi:10.1002/sim.2929 · Cited by 6,493
Using Standardized Serum Creatinine Values in the Modification of Diet in Renal Disease Study Equation for Estimating Glomerular Filtration Rate
Levey, A. S., Coresh, J., Greene, T., Stevens, L. A., Zhang, Y., Hendriksen, S. M., Kusek, J. W., & Lente, F. V. (2006) · Annals of Internal Medicine · doi:10.7326/0003-4819-145-4-200608150-00004 · Cited by 5,726
FastTree: Computing Large Minimum Evolution Trees with Profiles instead of a Distance Matrix
Price, M. N., Dehal, P. S., & Arkin, A. P. (2009) · Molecular Biology and Evolution · doi:10.1093/molbev/msp077 · Cited by 5,602
Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): Explanation and Elaboration
Moons, K. G. M., Altman, D. G., Reitsma, J. B., Ioannidis, J. P. A., Macaskill, P., Steyerberg, E. W., Vickers, A. J., Ransohoff, D. F., & Collins, G. S. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0698 · Cited by 5,147
Anxiety disorders in primary care: prevalence, impairment, comorbidity, and detection
Kroenke, K., Spitzer, R. L., Williams, J. B. W., Monahan, P. O., & Löwe, B. (2007) · Annals of Internal Medicine · doi:10.7326/0003-4819-146-5-200703060-00004 · Cited by 4,741
Receiver Operating Characteristic Curve in Diagnostic Test Assessment
Mandrekar, J. N. (2010) · Journal of Thoracic Oncology · doi:10.1097/jto.0b013e3181ec173d · Cited by 4,506
An Ultra-Brief Screening Scale for Anxiety and Depression: The PHQ–4
Kroenke, K., Spitzer, R. L., Williams, J. B. W., & Löwe, B. (2009) · Psychosomatics · doi:10.1016/s0033-3182(09)70864-3 · Cited by 4,261
International evaluation of an AI system for breast cancer screening
McKinney, S. M., Sieniek, M., Godbole, V., Godwin, J., Antropova, N., Ashrafian, H., Back, T., Chesus, M., Corrado, G. S., Darzi, A., et al. (2020) · Nature · doi:10.1038/s41586-019-1799-6 · Cited by 3,287
REVEL: An Ensemble Method for Predicting the Pathogenicity of Rare Missense Variants
Ioannidis, N. M., Rothstein, J. H., Pejaver, V., Middha, S., McDonnell, S. K., Baheti, S., Musolf, A., Li, Q., Holzinger, E., Karyadi, D., et al. (2016) · The American Journal of Human Genetics · doi:10.1016/j.ajhg.2016.08.016 · Cited by 2,908
Obesity and the risk of myocardial infarction in 27,000 participants from 52 countries: a case–control study
Yusuf, S., Hawken, S., Ôunpuu, S., Bautista, L., Franzosi, M. G., Commerford, P., Lang, C. C., Rumboldt, Z., Onen, C. L., Lisheng, L., et al. (2005) · The Lancet · doi:10.1016/s0140-6736(05)67663-5 · Cited by 2,903
Glypican-1 identifies cancer exosomes and detects early pancreatic cancer
Melo, S. A., Luecke, L. B., Kahlert, C., Fernandez, A. F., Gammon, S. T., Kaye, J., LeBleu, V. S., Mittendorf, E. A., Weitz, J., Rahbari, N., et al. (2015) · Nature · doi:10.1038/nature14581 · Cited by 2,880
Time‐Dependent ROC Curves for Censored Survival Data and a Diagnostic Marker
Heagerty, P. J., Lumley, T., & PEPE, M. S. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00337.x · Cited by 2,870
Clinical-grade computational pathology using weakly supervised deep learning on whole slide images
Campanella, G., Hanna, M. G., Geneslaw, L., Miraflor, A., Werneck Krauss Silva, V., Busam, K. J., Brogi, E., Reuter, V. E., Klimstra, D. S., & Fuchs, T. J. (2019) · Nature Medicine · doi:10.1038/s41591-019-0508-1 · Cited by 2,799
Frailty as a predictor of surgical outcomes in older patients
Makary, M. A., Segev, D. L., Pronovost, P. J., Syin, D., Bandeen-Roche, K., Patel, P., Takenaga, R., Devgan, L., Holzmueller, C. G., Tian, J., et al. (2010) · Journal of the American College of Surgeons · doi:10.1016/j.jamcollsurg.2010.01.028 · Cited by 2,184
Logistic regression and Artificial Neural Network Classification models: A methodology review
Dreiseitl, S., & Ohno-Machado, L. (2002) · Journal of Biomedical Informatics · doi:10.1016/s1532-0464(03)00034-0 · Cited by 2,177
Age-associated changes in skeletal muscles and their effect on mobility: an operational diagnosis of sarcopenia
Lauretani, F., Russo, C. R., Bandinelli, S., Bartali, B., Cavazzini, C., Di Iorio, A., Corsi, A. M., Rantanen, T., Guralnik, J. M., & Ferrucci, L. (2003) · Journal of Applied Physiology · doi:10.1152/japplphysiol.00246.2003 · Cited by 1,946
Diagnostic Performance of Digital versus Film Mammography for Breast-Cancer Screening
Pisano, E. D., Gatsonis, C. A., Hendrick, E. R., Yaffe, M. J., Baum, J. K., Acharyya, S., Conant, E. F., Fajardo, L. L., Bassett, L. W., D'Orsi, C. J., et al. (2005) · New England Journal of Medicine · doi:10.1056/nejmoa052911 · Cited by 1,937
Using Artificial Intelligence to Detect COVID-19 and Community-acquired Pneumonia Based on Pulmonary CT: Evaluation of the Diagnostic Accuracy
Li, L., Qin, L., Xu, Z., Yin, Y., Wang, X., Kong, B., Bai, J., Lu, Y., Fang, Z., Song, Q., et al. (2020) · Radiology · doi:10.1148/radiol.2020200905 · Cited by 1,927
Arterial Stiffness and Risk of Coronary Heart Disease and Stroke: The Rotterdam Study
Mattace-Raso, F. U. S., van der Cammen, T. J. M., Hofman, A., van Popele, N. M., Bos, M. L., Schalekamp, M. A. D. H., Asmar, R., Reneman, R. S., Hoeks, A. P. G., Breteler, M. M. B., et al. (2006) · Circulation · doi:10.1161/circulationaha.105.555235 · Cited by 1,914
Simplified criteria for the diagnosis of autoimmune hepatitis
Hennes, E. M., Zeniya, M., Czaja, A. J., Parés, A., Dalekos, G. N., Krawitt, E. L., Bittencourt, P. L., Porta, G., Boberg, K. M., Hofer, H., et al. (2008) · Hepatology · doi:10.1002/hep.22322 · Cited by 1,901
Estimating and comparing time‐dependent areas under receiver operating characteristic curves for censored event times with competing risks
Blanche, P., Dartigues, J., & Jacqmin‐Gadda, H. (2013) · Statistics in Medicine · doi:10.1002/sim.5958 · Cited by 1,917
Deep Learning at Chest Radiography: Automated Classification of Pulmonary Tuberculosis by Using Convolutional Neural Networks
Lakhani, P., & Sundaram, B. (2017) · Radiology · doi:10.1148/radiol.2017162326 · Cited by 1,775
Evaluating presence–absence models in ecology: the need to account for prevalence
Manel, S., Williams, H. C., & Ormerod, S. J. (2001) · Journal of Applied Ecology · doi:10.1046/j.1365-2664.2001.00647.x · Cited by 1,759
Clinical impact of COVID-19 on patients with cancer (CCC19): a cohort study
Kuderer, N. M., Choueiri, T. K., Shah, D. P., Shyr, Y., Rubinstein, S. M., Rivera, D. R., Shete, S., Hsu, C.-Y., Desai, A., de Lima Lopes, G., et al. (2020) · The Lancet · doi:10.1016/s0140-6736(20)31187-9 · Cited by 1,726
Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: A cross-sectional study
Zech, J. R., Badgeley, M. A., Liu, M., Costa, A. B., Titano, J. J., & Oermann, E. K. (2018) · PLOS Medicine · doi:10.1371/journal.pmed.1002683 · Cited by 1,719
Rethinking receiver operating characteristic analysis applications in ecological niche modeling
Peterson, A. T., Papeş, M., & Soberón, J. (2008) · Ecological Modelling · doi:10.1016/j.ecolmodel.2007.11.008 · Cited by 1,526
High performance plasma amyloid-β biomarkers for Alzheimer's disease
Nakamura, A., Kaneko, N., Villemagne, V. L., Kato, T., Doecke, J., Doré, V., Fowler, C., Li, Q.-X., Martins, R., Rowe, C., et al. (2018) · Nature · doi:10.1038/nature25456 · Cited by 1,681
Early Diagnosis of Myocardial Infarction with Sensitive Cardiac Troponin Assays
Reichlin, T., Hochholzer, W., Bassetti, S., Steuer, S., Stelzig, C., Hartwiger, S., Biedert, S., Schaub, N., Buerge, C., Potocki, M., et al. (2009) · New England Journal of Medicine · doi:10.1056/nejmoa0900428 · Cited by 1,678
Sample size, power and effect size revisited: simplified and practical approaches in pre-clinical, clinical and laboratory studies
Serdar, C. C., Cihan, M., Yücel, D., & Serdar, M. A. (2020) · Biochemia Medica · doi:10.11613/bm.2021.010502 · Cited by 1,868
EXPERT SYSTEMS WITH APPLICATIONS
Communication, S., Been-chian Chien, A., Jung Yi Lin, A., Tzung-pei Hong, B., & Com/locate/eswa, W. E. (2004) · Expert Systems with Applications · doi:10.1016/j.eswa.2004.10.010 · Cited by 1,670
Low Diagnostic Yield of Elective Coronary Angiography
Patel, M. R., Peterson, E. D., Dai, D. W. H., Brennan, J. M., Redberg, R. F., Anderson, H. V., Brindis, R. G., & Douglas, P. S. (2010) · New England Journal of Medicine · doi:10.1056/nejmoa0907272 · Cited by 1,685
AUDIT‐C as a Brief Screen for Alcohol Misuse in Primary Care
Bradley, K. A., Rubinsky, A. D., Volk, R. J., Williams, E. C., Frank, D., & Kivlahan, D. R. (2007) · Alcoholism Clinical and Experimental Research · doi:10.1111/j.1530-0277.2007.00403.x · Cited by 1,634
Discriminative accuracy of plasma phospho-tau217 for Alzheimer disease vs other neurodegenerative disorders
Palmqvist, S., Janelidze, S., Quiroz, Y. T., Zetterberg, H., Lopera, F., Stomrud, E., Su, Y., Chen, Y., Serrano, G. E., Leuzy, A., et al. (2020) · JAMA · doi:10.1001/jama.2020.12134 · Cited by 1,603
Receiver operating characteristic curve: overview and practical use for clinicians
Nahm, F. S. (2022) · Korean Journal of Anesthesiology · doi:10.4097/kja.21209 · Cited by 1,680
Combined Screening With Ultrasound and Mammography vs Mammography Alone in Women at Elevated Risk of Breast Cancer
Berg, W. A. (2008) · JAMA · doi:10.1001/jama.299.18.2151 · Cited by 1,527
Diagnostic Performance of Noninvasive Fractional Flow Reserve Derived From Coronary Computed Tomography Angiography in Suspected Coronary Artery Disease
Nørgaard, B. L., Leipsic, J. A., Gaur, S., SENEVIRATNE, S. K., Ko, B. S. H., Ito, H., Jensen, J. M., Mauri, L., Bruyne, B. D., Bezerra, H. G., et al. (2014) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2013.11.043 · Cited by 1,511
Receiver-operating characteristic analysis for evaluating diagnostic tests and predictive models
Zou, K. H., O’Malley, A. J., & Mauri, L. (2007) · Circulation · doi:10.1161/circulationaha.105.594929 · Cited by 1,473
Severe hypoglycemia and risks of vascular events and death
Zoungas, S., Patel, A., Chalmers, J., de Galan, B. E., Li, Q., Billot, L., Woodward, M., Ninomiya, T., Neal, B., MacMahon, S., et al. (2010) · New England Journal of Medicine · doi:10.1056/nejmoa1003795 · Cited by 1,471
Validation of Screening Questions for Limited Health Literacy in a Large VA Outpatient Population
Chew, L. D., Griffin, J. M., Partin, M. R., Noorbaloochi, S., Grill, J. P., Snyder, A., Bradley, K. A., Nugent, S. M., Simon, A. B., & van Ryn, M. (2008) · Journal of General Internal Medicine · doi:10.1007/s11606-008-0520-5 · Cited by 1,432
Evaluation metrics and statistical tests for machine learning
Rainio, O., Teuho, J., & Klén, R. (2024) · Scientific Reports · doi:10.1038/s41598-024-56706-x · Cited by 1,338
Pulmonary Hypertension in Heart Failure With Preserved Ejection Fraction
Lam, C. S. P., Roger, V. L., Rodeheffer, R. J., Borlaug, B. A., Enders, F., & Redfield, M. M. (2009) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2008.11.051 · Cited by 1,345
Meta-analysis: Natural history of non-alcoholic fatty liver disease (NAFLD) and diagnostic accuracy of non-invasive tests for liver disease severity
Musso, G., Gambino, R., Cassader, M., & Pagano, G. (2010) · Annals of Medicine · doi:10.3109/07853890.2010.518623 · Cited by 1,304
Limitations of the Odds Ratio in Gauging the Performance of a Diagnostic, Prognostic, or Screening Marker
Pepe, M. S. (2004) · American Journal of Epidemiology · doi:10.1093/aje/kwh101 · Cited by 1,276
The Diagnosis of Bronchopulmonary Dysplasia in Very Preterm Infants. An Evidence-based Approach
Jensen, E. A., Dysart, K., Gantz, M. G., McDonald, S. A., Bamat, N. A., Keszler, M., Kirpalani, H. M., Laughon, M. M., Poindexter, B. B., Duncan, A. F., et al. (2019) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.201812-2348oc · Cited by 1,274
A computational atlas of the hippocampal formation using ex vivo, ultra-high resolution MRI: Application to adaptive segmentation of in vivo MRI
Iglesias, J. E., Augustinack, J. C., Nguyen, K., Player, C. M., Player, A., Wright, M., Roy, N., Frosch, M. P., McKee, A. C., Wald, L. L., et al. (2015) · NeuroImage · doi:10.1016/j.neuroimage.2015.04.042 · Cited by 1,267
Extending the Limits of Quantitative Proteome Profiling with Data-Independent Acquisition and Application to Acetaminophen-Treated Three-Dimensional Liver Microtissues
Bruderer, R., Bernhardt, O. M., Gandhi, T., Miladinović, S. M., Cheng, L.-Y., Messner, S., Ehrenberger, T., Zanotelli, V., Butscheid, Y., Escher, C., et al. (2015) · Molecular & Cellular Proteomics · doi:10.1074/mcp.m114.044305 · Cited by 1,238
Biomarkers of Cardiovascular Disease: Molecular Basis and Practical Considerations
Vasan, R. S. (2006) · Circulation · doi:10.1161/circulationaha.104.482570 · Cited by 1,212
Detection of Breast Cancer With Addition of Annual Screening Ultrasound or a Single Screening MRI to Mammography in Women With Elevated Breast Cancer Risk
Zhang, Z. (2012) · JAMA · doi:10.1001/jama.2012.388 · Cited by 1,181
All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously
Fisher, A., Rudin, C., & Dominici, F. (2018) · arXiv · doi:10.48550/arxiv.1801.01489 · Cited by 1,165
Left atrial volume as a morphophysiologic expression of left ventricular diastolic dysfunction and relation to cardiovascular risk burden
Tsang, T. S. M., Barnes, M. E., Gersh, B. J., Bailey, K. R., & Seward, J. B. (2002) · The American Journal of Cardiology · doi:10.1016/s0002-9149(02)02864-3 · Cited by 1,159
Diagnostic Accuracy of Fractional Flow Reserve From Anatomic CT Angiography
Min, J. K., Leipsic, J. A., Pencina, M. J., Berman, D. S., Koo, B., van Mieghem, C. A. G., Ērglis, A., Lin, F. Y.-H., Dunning, A. M., Apruzzese, P. K., et al. (2012) · JAMA · doi:10.1001/2012.jama.11274 · Cited by 1,157
Comparison of novel risk markers for improvement in cardiovascular risk assessment in intermediate-risk individuals
Yeboah, J., McClelland, R. L., Polonsky, T. S., Burke, G. L., Sibley, C. T., O’Leary, D., Carr, J. J., Goff, D. C., Greenland, P., & Herrington, D. M. (2012) · JAMA · doi:10.1001/jama.2012.9624 · Cited by 1,122
Financial Distress Prediction in an International Context: A Review and Empirical Analysis of Altman's Z‐ Score Model
Altman, E. I., Iwanicz‐Drozdowska, M., Laitinen, E. K., & Suvas, A. (2016) · Journal of International Financial Management and Accounting · doi:10.1111/jifm.12053 · Cited by 1,111
Predicting aberrant behaviors in opioid-treated patients: preliminary validation of the Opioid Risk Tool
Webster, L. R., & Webster, R. M. (2005) · Pain Medicine · doi:10.1111/j.1526-4637.2005.00072.x · Cited by 1,096
Artificial intelligence–enabled rapid diagnosis of patients with COVID-19
Mei, X., Lee, H.-C., Diao, K.-Y., Huang, M., Lin, B., Liu, C., Xie, Z., Ma, Y., Robson, P. M., Chung, M., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-0931-3 · Cited by 1,084
Comparative validity of three screening questionnaires for DSM-IV depressive disorders and physicians? diagnoses
LOWE, B. (2004) · Journal of Affective Disorders · doi:10.1016/s0165-0327(02)00237-9 · Cited by 1,060
Diagnosing schizophrenia in the initial prodromal phase
Klosterkötter, J., Hellmich, M., Steinmeyer, E. M., & Schultze-Lutter, F. (2001) · Archives of General Psychiatry · doi:10.1001/archpsyc.58.2.158 · Cited by 1,049
Sample size estimation in diagnostic test studies of biomedical informatics
Hajian‐Tilaki, K. (2014) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2014.02.013 · Cited by 1,035
Fully Automated Echocardiogram Interpretation in Clinical Practice: Feasibility and Diagnostic Accuracy
Zhang, J., Gajjala, S., Agrawal, P., Tison, G. H., Hallock, L. A., Beussink-Nelson, L., Lassen, M. H., Fan, E., Aras, M. A., Jordan, C., et al. (2018) · Circulation · doi:10.1161/circulationaha.118.034338 · Cited by 1,006
“Left atrial volume, mL 4800 53 (40–71) 9 (16) 16 (29) 33 (66)”
Screening with magnetic resonance imaging and mammography of a UK population at high familial risk of breast cancer: a prospective multicentre cohort study (MARIBS)
Leach, M. O., Boggis, C. R. M., Dixon, A. K., Easton, D. F., Eeles, R. A., Evans, D. G., Gilbert, F. J., Griebsch, I., Hoff, R., Kessar, P., et al. (2005) · The Lancet · doi:10.1016/s0140-6736(05)66481-1 · Cited by 1,005
Identification of serum metabolites associated with risk of type 2 diabetes using a targeted metabolomic approach
Floegel, A., Stefan, N., Yu, Z., Mühlenbruch, K., Drogan, D., Joost, H.-G., Fritsche, A., Häring, H.-U., Hrabě de Angelis, M., Peters, A., et al. (2013) · Diabetes · doi:10.2337/db12-0495 · Cited by 988
Transferability and model evaluation in ecological niche modeling: a comparison of GARP and Maxent
Peterson, A. T., Papeş, M., & Eaton, M. D. (2007) · Ecography · doi:10.1111/j.0906-7590.2007.05102.x · Cited by 966
Opportunistic screening for osteoporosis using abdominal computed tomography scans obtained for other indications
Pickhardt, P. J., Pooler, B. D., Lauder, T., del Rio, A. M., Bruce, R. J., & Binkley, N. (2013) · Annals of Internal Medicine · doi:10.7326/0003-4819-158-8-201304160-00003 · Cited by 954
A two-gene expression ratio predicts clinical outcome in breast cancer patients treated with tamoxifen
Ma, X.-J., Wang, Z., Ryan, P. D., Isakoff, S. J., Barmettler, A., Fuller, A. P., Muir, B., Mohapatra, G., Salunga, R. C., Tuggle, J. T., et al. (2004) · Cancer Cell · doi:10.1016/j.ccr.2004.05.015 · Cited by 959
The early prediction of mortality in acute pancreatitis: a large population-based study
Wu, B. U., Johannes, R. S., Sun, X., Tabak, Y. P., Conwell, D. L., & Banks, P. A. (2008) · Gut · doi:10.1136/gut.2008.152702 · Cited by 920
On the Rise of FinTechs: Credit Scoring Using Digital Footprints
Berg, T., Burg, V., Gombović, A., & Puri, M. (2020) · The Review of Financial Studies · doi:10.1093/rfs/hhz099 · Cited by 919
Receiver Operating Characteristic (ROC) Curve: Practical Review for Radiologists
Park, S. H., Goo, J. M., & Jo, C.-H. (2004) · Korean Journal of Radiology · doi:10.3348/kjr.2004.5.1.11 · Cited by 909
Benchmarking state-of-the-art classification algorithms for credit scoring
Baesens, B., Van Gestel, T., Viaene, S., Stepanova, M., Suykens, J., & Vanthienen, J. (2003) · Journal of the Operational Research Society · doi:10.1057/palgrave.jors.2601545 · Cited by 904
Genic Intolerance to Functional Variation and the Interpretation of Personal Genomes
Petrovski, S., Wang, Q., Heinzen, E. L., Allen, A. S., & Goldstein, D. B. (2013) · PLoS Genetics · doi:10.1371/journal.pgen.1003709 · Cited by 901
Clinical Risk Score for Persistent Postconcussion Symptoms Among Children With Acute Concussion in the ED
Zemek, R., Barrowman, N., Freedman, S. B., Gravel, J., Gagnon, I., McGahern, C., Aglipay, M., Sangha, G., Boutis, K., Beer, D., et al. (2016) · JAMA · doi:10.1001/jama.2016.1203 · Cited by 900
Brachial Flow-Mediated Dilation Predicts Incident Cardiovascular Events in Older Adults
Yeboah, J., Crouse, J. R., Hsu, F., Burke, G. L., & Herrington, D. M. (2007) · Circulation · doi:10.1161/circulationaha.106.678276 · Cited by 902
Carotid Intimal-Medial Thickness Is Related to Cardiovascular Risk Factors Measured From Childhood Through Middle Age
Davis, P. H., Dawson, J. D., Riley, W. A., & Lauer, R. M. (2001) · Circulation · doi:10.1161/hc4601.099486 · Cited by 881
REFUGE Challenge: A unified framework for evaluating automated methods for glaucoma assessment from fundus photographs
Orlando, J. I., Fu, H., Barbosa‐Breda, J., Keer, K. V., Bathula, D. R., Diaz‐Pinto, A., Fang, R., Heng, P., Kim, J., Lee, J., et al. (2019) · Medical Image Analysis · doi:10.1016/j.media.2019.101570 · Cited by 910
B-Type Natriuretic Peptide and Clinical Judgment in Emergency Diagnosis of Heart Failure
McCullough, P. A., Nowak, R. M., McCord, J., Hollander, J. E., Herrmann, H. C., Steg, P. G., Duc, P., Westheim, A. S., Omland, T., Knudsen, C. W., et al. (2002) · Circulation · doi:10.1161/01.cir.0000025242.79963.4c · Cited by 875
Receiver operating characteristic (ROC) curve for medical researchers
Kumar, R., & Indrayan, A. (2011) · Indian Pediatrics · doi:10.1007/s13312-011-0055-4 · Cited by 872
Lipids, lipoproteins, and apolipoproteins as risk markers of myocardial infarction in 52 countries (the INTERHEART study): a case–control study
McQueen, M. J., Hawken, S., Wang, X., Ounpuu, S., Sniderman, A., Probstfield, J., Steyn, K., Sanderson, J. E., Hasani, M., Volkova, E., et al. (2008) · The Lancet · doi:10.1016/s0140-6736(08)61076-4 · Cited by 864
DISOPRED3: precise disordered region predictions with annotated protein-binding activity
Jones, D. T., & Cozzetto, D. (2014) · Bioinformatics · doi:10.1093/bioinformatics/btu744 · Cited by 877
Areas beneath the relative operating characteristics (ROC) and relative operating levels (ROL) curves: Statistical significance and interpretation
Mason, S. J., & Graham, N. E. (2002) · Quarterly Journal of the Royal Meteorological Society · doi:10.1256/003590002320603584 · Cited by 860
Receiver Operating Characteristic Curves and Their Use in Radiology
Obuchowski, N. A. (2003) · Radiology · doi:10.1148/radiol.2291010898 · Cited by 843
The use of receiver operating characteristic curves in biomedical informatics
Lasko, T. A., Bhagwat, J. G., Zou, K. H., & Lucila Ohno-machado, A. (2005) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2005.02.008 · Cited by 821
Distinguishing features of long COVID identified through immune profiling
Klein, J., Wood, J., Jaycox, J. R., Dhodapkar, R. M., Lu, P., Gehlhausen, J. R., Tabachnikova, A., Greene, K., TABACOF, L. R., Malik, A. A., et al. (2023) · Nature · doi:10.1038/s41586-023-06651-y · Cited by 852
Cell-free DNA Analysis for Noninvasive Examination of Trisomy
Norton, M. E., Jacobsson, B., Swamy, G. K., Laurent, L. C., Ranzini, A. C., Brar, H., Tomlinson, M. W., Pereira, L., Spitz, J. L., Hollemon, D., et al. (2015) · New England Journal of Medicine · doi:10.1056/nejmoa1407349 · Cited by 817
Magnetic Resonance Imaging More Accurately Classifies Steatosis and Fibrosis in Patients With Nonalcoholic Fatty Liver Disease Than Transient Elastography
Imajo, K., Kessoku, T., Honda, Y., Tomeno, W., Ogawa, Y., Mawatari, H., Fujita, K., Yoneda, M., Taguri, M., Hyogo, H., et al. (2015) · Gastroenterology · doi:10.1053/j.gastro.2015.11.048 · Cited by 815
An Index Combining Respiratory Rate and Oxygenation to Predict Outcome of Nasal High-Flow Therapy
Roca, O., Caralt, B., Messika, J., Samper, M., Sztrymf, B., Hernández, G., García-de-Acilu, M., Frat, J., Masclans, J. R., & Ricard, J.-D. (2018) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.201803-0589oc · Cited by 834
The Metabolic Syndrome as Predictor of Type 2 Diabetes
Lorenzo, C., Okoloise, M., Williams, K., Stern, M. P., & Haffner, S. M. (2003) · Diabetes Care · doi:10.2337/diacare.26.11.3153 · Cited by 800
Coronary Risk Stratification, Discrimination, and Reclassification Improvement Based on Quantification of Subclinical Coronary Atherosclerosis
Erbel, R. M., Möhlenkamp, S., Moebus, S., Schmermund, A., Lehmann, N., Stang, A., Dragano, N., Grönemeyer, D. H. W., Seibel, R. M. M., Kälsch, H., et al. (2010) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2010.06.030 · Cited by 783
Parental History of Myopia, Sports and Outdoor Activities, and Future Myopia
Jones, L. A., Sinnott, L. T., Mutti, D. O., Mitchell, G. L., Moeschberger, M. L., & Zadnik, K. (2007) · Investigative Ophthalmology & Visual Science · doi:10.1167/iovs.06-1118 · Cited by 769
An experimental comparison of classification algorithms for imbalanced credit scoring data sets
Brown, I., & Mues, C. (2011) · Expert Systems with Applications · doi:10.1016/j.eswa.2011.09.033 · Cited by 813
Postoperative Biomarkers Predict Acute Kidney Injury and Poor Outcomes after Adult Cardiac Surgery
Parikh, C. R., Coca, S. G., Thiessen‐Philbrook, H., Shlipak, M. G., Koyner, J. L., Wang, Z., Edelstein, C. L., Devarajan, P., Patel, U. D., Zappitelli, M., et al. (2011) · Journal of the American Society of Nephrology · doi:10.1681/asn.2010121302 · Cited by 772
Genotype score in addition to common risk factors for prediction of type 2 diabetes
Meigs, J. B., Shrader, P., Sullivan, L. M., McAteer, J. B., Fox, C. S., Dupuis, J., Manning, A. K., Florez, J. C., Wilson, P. W. F., D'Agostino, R. B., et al. (2008) · New England Journal of Medicine · doi:10.1056/nejmoa0804742 · Cited by 759
A Deep Learning Mammography-based Model for Improved Breast Cancer Risk Prediction
Yala, A., Lehman, C., Schuster, T., Portnoi, T., & Barzilay, R. (2019) · Radiology · doi:10.1148/radiol.2019182716 · Cited by 758
Does the Metabolic Syndrome Improve Identification of Individuals at Risk of Type 2 Diabetes and/or Cardiovascular Disease?
Stern, M. P., Williams, K., González-Villalpando, C., Hunt, K. J., & Haffner, S. M. (2004) · Diabetes Care · doi:10.2337/diacare.27.11.2676 · Cited by 735
Deep-learning-assisted diagnosis for knee magnetic resonance imaging: Development and retrospective validation of MRNet
Bien, N., Rajpurkar, P., Ball, R. L., Irvin, J., Park, A., Jones, E., Bereket, M., Patel, B. N., Yeom, K. W., Shpanskaya, K., et al. (2018) · PLOS Medicine · doi:10.1371/journal.pmed.1002699 · Cited by 734
Cardiovascular Mortality Risk in Chronic Kidney Disease
Shlipak, M. G., Fried, L. F., Cushman, M., Manolio, T. A., Peterson, D., Stehman‐Breen, C. O., Bleyer, A. J., Newman, A. B., Siscovick, D. S., & Psaty, B. M. (2005) · JAMA · doi:10.1001/jama.293.14.1737 · Cited by 728

Authors: DeLong, E. R., DeLong, D. M., & Clarke-Pearson, D. L.
Affiliations: IQVIA (United States) (US)
IQVIA (United Kingdom) (GB)
ORCID: DeLong, Elizabeth R.
Venue: Biometrics
Publisher: JSTOR
Published: 1988-09
Volume: 44
Issue: 3
Pages: 837
ISSN: 0006-341X (ISSN-L)
DOI: 10.2307/2531595
OpenAlex: W2328176404
Cited by: 22,649
References: 12
Citations by year: 2026: 1897 · 2025: 1546 · 2024: 1644 · 2023: 1628 · 2022: 1861 · 2021: 1946 · 2020: 1651 · 2019: 1364 · 2018: 1193 · 2017: 1076 · 2016: 1001 · 2015: 924 · 2014: 775 · 2013: 745 · 2012: 621
FWCI: 4.52
Type: journal article
Language: en
Open access: closed
Locations: Biometrics (publishedVersion)
PubMed (publishedVersion)
Funders: Health Services Research and Development
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
Keywords: receiver operating characteristic, area under the ROC curve, diagnostic test comparison, nonparametric covariance estimation, sensitivity and false positive rate
MeSH terms: Algorithms, Analysis of Variance, Female, Humans, Intestinal Obstruction, Ovarian Neoplasms, Predictive Value of Tests, ROC Curve, Models, Statistical

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Comparing the Areas under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric Approach

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks