Cited by 22,649Imbalanced Data Classification TechniquesStatistical Methods in EpidemiologyStatistical Methods in Clinical Trials

The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.

Comparing the Areas under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric Approach

DeLong, Elizabeth R., DeLong, David M., Clarke-Pearson, Daniel L.·1988·Biometrics

DOI 10.2307/2531595

Open this paper in CoChat — free account

Abstract

Methods of evaluating and comparing the performance of diagnostic tests are of increasing importance as new tests are developed and marketed. When a test is based on an observed variable that lies on a continuous or graded scale, an assessment of the overall value of the test can be made through the use of a receiver operating characteristic (ROC) curve. The curve is constructed by varying the cutpoint used to determine which values of the observed variable will be considered abnormal and then plotting the resulting sensitivities against the corresponding false positive rates. When two or more empirical curves are constructed based on tests performed on the same individuals, statistical analysis on differences between curves must take into account the correlated nature of the data. This paper presents a nonparametric approach to the analysis of areas under correlated ROC curves, by using the theory on generalized U-statistics to generate an estimated covariance matrix.

Integrity signals

Registry

Crossref · 10.2307/2531595

Cites

12 of 12 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

The meaning and use of the area under a receiver operating characteristic (ROC) curve.
Hanley, J. A., & McNeil, B. J. (1982) · Radiology · doi:10.1148/radiology.143.1.7063747 · Cited by 21,856
Basic principles of ROC analysis
Metz, C. E. (1978) · Seminars in Nuclear Medicine · doi:10.1016/s0001-2998(78)80014-2 · Cited by 6,122
A Class of Statistics with Asymptotically Normal Distribution
Hoeffding, W. (1992) · Springer series in statistics · doi:10.1007/978-1-4612-0919-5_20 · Cited by 1,898
The area above the ordinal dominance graph and the area below the receiver operating characteristic graph
Bamber, D. (1975) · Journal of Mathematical Psychology · doi:10.1016/0022-2496(75)90001-2 · Cited by 1,452
Untitled work
(1982) · Cited by 1,288
Maximum-likelihood estimation of parameters of signal-detection theory and determination of confidence intervals—Rating-method data
Dorfman, D. D., & Alf, E. F. (1969) · Journal of Mathematical Psychology · doi:10.1016/0022-2496(69)90019-4 · Cited by 878
A New Approach for Testing the Significance of Differences Between ROC Curves Measured from Correlated Data
Metz, C. E., Wang, P.-L., & Kronman, H. B. (1984) · Information Processing in Medical Imaging · doi:10.1007/978-94-009-6045-9_25 · Cited by 359
Jackknifing $U$-Statistics
Arvesen, J. N. (1969) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177697287 · Cited by 286
On Some Convergence Properties of UStatistics
Sen, P. K. (1960) · Calcutta Statistical Association Bulletin · doi:10.1177/0008068319600101 · Cited by 182
Surgical management of bowel obstruction in advanced ovarian carcinoma.
Krebs, H. B., & Goplerud, D. R. (1983) · PubMed · Cited by 132

Flagged first, then by citations

Show all 12 references

Sorted by citations

Items 501–600 of 1,197

Circulating Biomarkers of Distinct Pathophysiological Pathways in Heart Failure with Preserved vs. Reduced Left Ventricular Ejection Fraction
Wijk, S. S., van Empel, V. P. M., Davarzani, N., Maeder, M. T., Handschin, R., Pfisterer, M. E., Rocca, H. B., & for the TIME-CHF investigators (2015) · European Journal of Heart Failure · doi:10.1002/ejhf.414 · Cited by 258
Serum Indicators for the Diagnosis of Pneumocystis Pneumonia
Tasaka, S., Hasegawa, N., Kobayashi, S., Yamada, W., Nishimura, T., Takeuchi, T., & Ishizaka, A. (2007) · CHEST Journal · doi:10.1378/chest.06-1467 · Cited by 258
Prostate-Specific Antigen (PSA) Isoform p2PSA in Combination with Total PSA and Free PSA Improves Diagnostic Accuracy in Prostate Cancer Detection
Jansen, F. H., van Schaik, R. H. N., Kurstjens, J., Horninger, W., Klocker, H., Bektić, J., Wildhagen, M. F., Roobol, M. J., Bangma, C. H., & Bartsch, G. H. (2010) · European Urology · doi:10.1016/j.eururo.2010.02.003 · Cited by 258
Relation of central adiposity and body mass index to the development of diabetes in the Diabetes Prevention Program
Bray, G. A., Jablonski, K. A., Fujimoto, W. Y., Barrett-Connor, E. L., Haffner, S. M., Hanson, R. L., Hill, J. O., Hubbard, V. S., Kriska, A. M., Stamm, E. R., et al. (2008) · American Journal of Clinical Nutrition · doi:10.1093/ajcn/87.5.1212 · Cited by 258
New models and online calculator for predicting non-sentinel lymph node status in sentinel lymph node positive breast cancer patients
Kohrt, H. E. K., Olshen, R. A., Bermas, H. R., Goodson, W. H., Wood, D., Henry, A. S., Rouse, R. V., Bailey, L., Philben, V. J., Dirbas, F. M., et al. (2008) · BMC Cancer · doi:10.1186/1471-2407-8-66 · Cited by 257
Clock-drawing: is it the ideal cognitive screening test?
Shulman, K. I. (2000) · International Journal of Geriatric Psychiatry · doi:10.1002/1099-1166(200006)15:6 · Cited by 1,354
Psychometric Properties of the Mini-Balance Evaluation Systems Test (Mini-BESTest) in Community-Dwelling Individuals With Chronic Stroke
Tsang, C. S. L., Liao, L.-R., Chung, R. C. K., & Pang, M. Y. C. (2013) · Physical Therapy · doi:10.2522/ptj.20120454 · Cited by 257
Assessment of Liver Viscoelasticity by Using Shear Waves Induced by Ultrasound Radiation Force
Chen, S., Sánchez, W., Callstrom, M. R., Gorman, B. P., Lewis, J. T., Sanderson, S. O., Greenleaf, J. F., Xie, H., Shi, Y., Pashley, M. D., et al. (2012) · Radiology · doi:10.1148/radiol.12120837 · Cited by 257
Incident Heart Failure Prediction in the Elderly
Butler, J., Kalogeropoulos, A. P., Georgiopoulou, V. V., BeLue, R., Rodondi, N., García, M., Bauer, D. C., Satterfield, S., Smith, A. L., Vaccarino, V., et al. (2008) · Circulation Heart Failure · doi:10.1161/circheartfailure.108.768457 · Cited by 257
qSOFA, SIRS and NEWS for predicting inhospital mortality and ICU admission in emergency admissions treated as sepsis
Goulden, R., Hoyle, M.-C., Monis, J., Railton, D., Riley, V., Martin, P. S. P., Martina, R., & Nsutebu, E. F. (2018) · Emergency Medicine Journal · doi:10.1136/emermed-2017-207120 · Cited by 257
PremPS: Predicting the impact of missense mutations on protein stability
Chen, Y., Lu, H., Zhang, N., Zhu, Z., Wang, S., & Li, M. (2020) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1008543 · Cited by 263
Aortic Regurgitation Quantification Using Cardiovascular Magnetic Resonance
Myerson, S. G., D’Arcy, J. L., Mohiaddin, R. H., Greenwood, J. P., Karamitsos, T. D., Francis, J. M., BANNING, A. P., Christiansen, J., & Neubauer, S. (2012) · Circulation · doi:10.1161/circulationaha.111.083600 · Cited by 256
The Napkin-Ring Sign Indicates Advanced Atherosclerotic Lesions in Coronary CT Angiography
Maurovich‐Horvat, P., Schlett, C. L., Alkadhi, H., Nakano, M., Otsuka, F., Stolzmann, P., Scheffel, H., Ferencik, M., Kriegel, M. F., Seifarth, H., et al. (2012) · JACC. Cardiovascular imaging · doi:10.1016/j.jcmg.2012.03.019 · Cited by 255
A Mouse to Human Search for Plasma Proteome Changes Associated with Pancreatic Tumor Development
Faça, V. M., Song, K. S., Wang, H., Zhang, Q., Krasnoselsky, A. L., Newcomb, L. F., Plentz, R. R., Gurumurthy, S., Redston, M. S., Pitteri, S. J., et al. (2008) · PLoS Medicine · doi:10.1371/journal.pmed.0050123 · Cited by 255
Predicting cognitive decline with deep learning of brain metabolism and amyloid imaging
Choi, H., & Jin, K. H. (2018) · Behavioural Brain Research · doi:10.1016/j.bbr.2018.02.017 · Cited by 268
Using Deep Learning and Transfer Learning to Accurately Diagnose Early-Onset Glaucoma From Macular Optical Coherence Tomography Images
Asaoka, R., Murata, H., Hirasawa, K., Fujino, Y., Matsuura, M., Miki, A., Kanamoto, T., Ikeda, Y., Mori, K., Iwase, A., et al. (2018) · American Journal of Ophthalmology · doi:10.1016/j.ajo.2018.10.007 · Cited by 255
Discernibility and rough sets in medicine : tools and applications.
Øhrn, A. (2000) · BIBSYS Brage (BIBSYS (Norway)) · Cited by 255
1001985.pdf
Data, M. C. (2020) · OAPEN (The OAPEN Foundation) · doi:10.1007/978-3-319-43742-2 · Cited by 273
The HIT Expert Probability (HEP) Score: a novel pre‐test probability model for heparin‐induced thrombocytopenia based on broad expert opinion
Cuker, A. C., Arepally, G. M., Crowther, M. A., Rice, L., DATKO, F., Hook, K. M., Propert, K. J., Kuter, D. J., Ortel, T. L., Konkle, B. A., et al. (2010) · Journal of Thrombosis and Haemostasis · doi:10.1111/j.1538-7836.2010.04059.x · Cited by 254
Comparative Impact of Multiple Biomarkers and N-Terminal Pro-Brain Natriuretic Peptide in the Context of Conventional Risk Factors for the Prediction of Recurrent Cardiovascular Events in the Heart Outcomes Prevention Evaluation (HOPE) Study
Blankenberg, S., McQueen, M., Smieja, M., Pogue, J., Balion, C. M., Lonn, E., Rupprecht, H. J., Bickel, C., Tiret, L., Cambien, F., et al. (2006) · Circulation · doi:10.1161/circulationaha.105.590927 · Cited by 254
Extraction and analysis of signatures from the Gene Expression Omnibus by the crowd
Wang, Z., Monteiro, C. D., Jagodnik, K. M., Fernandez, N., Gundersen, G. W., Rouillard, A. D., Jenkins, S. L., Feldmann, A. S., Hu, K. S., McDermott, M. G., et al. (2016) · Nature Communications · doi:10.1038/ncomms12846 · Cited by 261
Automated abnormality classification of chest radiographs using deep convolutional neural networks
Tang, Y., Peng, Y., Peng, Y., Yan, K., Bagheri, M., Redd, B. A., Brandon, C. J., Lu, Z., Han, M., Xiao, J. Z., et al. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0273-z · Cited by 262
Fully automated hybrid approach to predict the IDH mutation status of gliomas via deep learning and radiomics
Choi, Y. S., Bae, S., Chang, J. H., Kang, S., Kim, S. H., Kim, J., Rim, T. H., Choi, S. H., Jain, R., & Lee, S. (2020) · Neuro-Oncology · doi:10.1093/neuonc/noaa177 · Cited by 255
Use of electron beam tomography data to develop models for prediction of hard coronary events
Raggi, P., Cooil, B., & Callister, T. Q. (2001) · American Heart Journal · doi:10.1067/mhj.2001.113220 · Cited by 254
Effects of contrast-enhancement, reconstruction slice thickness and convolution kernel on the diagnostic performance of radiomics signature in solitary pulmonary nodule
He, L., Huang, Y., Ma, Z., Liang, C., Liang, C., & Liu, Z. (2016) · Scientific Reports · doi:10.1038/srep34921 · Cited by 253
Deep learning-enabled breast cancer hormonal receptor status determination from base-level H&E stains
Naik, N., Madani, A., Esteva, A., Keskar, N. S., Press, M. F., Ruderman, D., Agus, D. B., & Socher, R. (2020) · Nature Communications · doi:10.1038/s41467-020-19334-3 · Cited by 262
Prospective assessment of breast cancer risk from multimodal multiview ultrasound images via clinically applicable deep learning
Qian, X., Pei, J., Zheng, H., Xie, X., Lin, Y., Zhang, H., Han, C., Gao, X., Zhang, H., Zheng, W., et al. (2021) · Nature Biomedical Engineering · doi:10.1038/s41551-021-00711-2 · Cited by 257
Semiquantitative and visual analysis of FDG-PET images in pulmonary abnormalities.
Lowe, V. J., Hoffman, J. M., DeLong, D. M., Patz, E. F., & Coleman, R. E. (1994) · PubMed · Cited by 253
Self-citation
Electrocardiogram screening for aortic valve stenosis using artificial intelligence
Cohen-Shelly, M., Attia, Z. I., Friedman, P. A., Ito, S., Essayagh, B. A., Ko, W.-Y., Murphree, D. H., Michelena, H. I., Enriquez-Sarano, M., Carter, R. E., et al. (2021) · European Heart Journal · doi:10.1093/eurheartj/ehab153 · Cited by 252
Comparison of blood tests for liver fibrosis specific or not to NAFLD
Calès, P., Lainé, F., Boursier, J., Deugnier, Y., Moal, V., Oberti, F., Hunault, G., Rousselet, M. C., Hubert, I., Laafi, J., et al. (2008) · Journal of Hepatology · doi:10.1016/j.jhep.2008.07.035 · Cited by 255
A Highly Predictive Risk Model for Pacemaker Implantation After TAVR
Maeno, Y., Abramowitz, Y., Kawamori, H., Kazuno, Y., Kubo, S., Takahashi, N., Mangat, G., Okuyama, K., Kashif, M., Chakravarty, T., et al. (2017) · JACC. Cardiovascular imaging · doi:10.1016/j.jcmg.2016.11.020 · Cited by 252
Diabetic retinopathy detection using red lesion localization and convolutional neural networks
Zago, G. T., Andreão, R. V., Dorizzi, B., & Salles, E. O. T. (2019) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2019.103537 · Cited by 252
Mammographic density adds accuracy to both the Tyrer-Cuzick and Gail breast cancer risk models in a prospective UK screening cohort
Brentnall, A. R., Harkness, E. F., Astley, S. M., Donnelly, L. S., Stavrinos, P., Sampson, S., Fox, L., Sergeant, J. C., Harvie, M. N., Wilson, M., et al. (2015) · Breast Cancer Research · doi:10.1186/s13058-015-0653-5 · Cited by 252
High-density genotyping of immune-related loci identifies new SLE risk variants in individuals with Asian ancestry
Sun, C., Molineros, J. E., Looger, L. L., Zhou, S., Kim, K.-W., Okada, Y., Ma, J., Qi, Y., Kim-Howard, X., Motghare, P. N., et al. (2016) · Nature Genetics · doi:10.1038/ng.3496 · Cited by 256
An updated PREDICT breast cancer prognostication and treatment benefit prediction model with independent validation
dos Reis, F. J. C., Wishart, G. C., Dicks, E., Greenberg, D. C., Rashbass, J., Schmidt, M. K., van den Broek, A. J., Ellis, I. O., Green, A. R., Rakha, E. A., et al. (2017) · Breast Cancer Research · doi:10.1186/s13058-017-0852-3 · Cited by 251
A 10‐item smell identification scale related to risk for Alzheimer's disease
Tabert, M. H., Liu, X., Doty, R. L., Serby, M., Zamora, D., Pelton, G. H., Marder, K. S., Albers, M. W., Stern, Y., & Devanand, D. P. (2005) · Annals of Neurology · doi:10.1002/ana.20533 · Cited by 250
Macular and Peripapillary Retinal Nerve Fiber Layer Measurements by Spectral Domain Optical Coherence Tomography in Normal-Tension Glaucoma
Seong, M., Sung, K. R., Choi, E. H., Kang, S. Y., Cho, J. W., Um, T. W., Kim, Y. J., Park, S. B., Hong, H., & Kook, M. S. (2010) · Investigative Ophthalmology & Visual Science · doi:10.1167/iovs.09-4258 · Cited by 250
Prognostic Utility of Plasma Neutrophil Gelatinase-Associated Lipocalin in Patients with Acute Heart Failure: The NGAL EvaLuation Along with B-type NaTriuretic Peptide in Acutely Decompensated Heart Failure (GALLANT) Trial
Maisel, A. S., Mueller, C. E., Fitzgerald, R. L., Brikhan, R., Hiestand, B. C., Iqbal, N., Clopton, P., & van Veldhuisen, D. J. (2011) · European Journal of Heart Failure · doi:10.1093/eurjhf/hfr087 · Cited by 254
Fasting Versus Postload Plasma Glucose Concentration and the Risk for Future Type 2 Diabetes
Abdul‐Ghani, M. A., Lyssenko, V., Tuomilehto, J., DeFronzo, R. A., & Groop, L. (2008) · Diabetes Care · doi:10.2337/dc08-1264 · Cited by 253
Prediction of 6-Month Survival of Nursing Home Residents With Advanced Dementia Using ADEPT vs Hospice Eligibility Guidelines
Mitchell, S. L., Miller, S. C., Teno, J. M., Kiely, D. K., Davis, R. B., & Shaffer, M. L. (2010) · JAMA · doi:10.1001/jama.2010.1572 · Cited by 249
Prognostic markers and long-term outcome of placental-site trophoblastic tumours: a retrospective observational study
Schmid, P., Nagai, Y., Agarwal, R., Hancock, B. W., Savage, P. M., Sebire, N. J., Lindsay, I., Wells, M. L., Fisher, R. A., Short, D., et al. (2009) · The Lancet · doi:10.1016/s0140-6736(09)60618-8 · Cited by 249
CT and MR imaging in the local staging of primary malignant musculoskeletal neoplasms: Report of the Radiology Diagnostic Oncology Group.
Panicek, D. M., Gatsonis, C. G., Rosenthal, D. I., Seeger, L. L., Huvos, A. G., Moore, S., Caudry, D. J., Palmer, W. E., & McNeil, B. J. (1997) · Radiology · doi:10.1148/radiology.202.1.8988217 · Cited by 249
A multifocal electroretinogram model predicting the development of diabetic retinopathy
Bearse, M. A., Adams, A. J., Han, Y., Schneck, M. E., Ng, J., Bronson-Castain, K., & Barez, S. (2006) · Progress in Retinal and Eye Research · doi:10.1016/j.preteyeres.2006.07.001 · Cited by 250
Added Value of SPECT/CT Fusion in Assessing Suspected Bone Metastasis: Comparison with Scintigraphy Alone and Nonfused Scintigraphy and CT
Utsunomiya, D., Shiraishi, S., Imuta, M., Tomiguchi, S., Kawanaka, K., Morishita, S., Awai, K., & Yamashita, Y. (2006) · Radiology · doi:10.1148/radiol.2373041358 · Cited by 248
An integrated model for the prediction of preeclampsia using maternal factors and uterine artery Doppler velocimetry in unselected low-risk women
Yu, C.-S., Smith, G. C. S., Papageorghiou, A. T., Cacho, A. M., & Nicolaides, K. H. (2005) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2004.12.014 · Cited by 248
Serum Calcification Propensity Predicts All-Cause Mortality in Predialysis CKD
Smith, E. R., Ford, M. L., Tomlinson, L. A., Bodenham, E., McMahon, L. P., Farese, S., Rajkumar, C., Holt, S. G., & Pasch, A. (2013) · Journal of the American Society of Nephrology · doi:10.1681/asn.2013060635 · Cited by 247
Ordered multiple‐class ROC analysis with continuous measurements
Nakas, C. T., & Yiannoutsos, C. T. (2004) · Statistics in Medicine · doi:10.1002/sim.1917 · Cited by 247
The impact of rare variation on gene expression across tissues
Li, X., Kim, Y., Tsang, E. K., Davis, J. R., Damani, F. N., Chiang, C., Hess, G. T., Zappala, Z., Strober, B. J., Scott, A. J., et al. (2017) · Nature · doi:10.1038/nature24267 · Cited by 253
Coronary CT Angiography–derived Fractional Flow Reserve: Machine Learning Algorithm versus Computational Fluid Dynamics Modeling
Tesche, C., Cecco, C. N. D., Baumann, S., Renker, M., McLaurin, T. W., Duguay, T. M., Bayer, R. R., Steinberg, D. H., Grant, K. L. R., Canstein, C., et al. (2018) · Radiology · doi:10.1148/radiol.2018171291 · Cited by 247
MRI Radiomics Model Predicts Pathologic Complete Response of Rectal Cancer Following Chemoradiotherapy
Shin, J., Seo, N., Baek, S., Son, N., Lim, J. S., Kim, N. K., Koom, W. S., & Kim, S. (2022) · Radiology · doi:10.1148/radiol.211986 · Cited by 247
Predicting Drug-Target Interactions for New Drug Compounds Using a Weighted Nearest Neighbor Profile
van Laarhoven, T., & Marchiori, E. (2013) · PLoS ONE · doi:10.1371/journal.pone.0066952 · Cited by 249
SuSPect: Enhanced Prediction of Single Amino Acid Variant (SAV) Phenotype Using Network Features
Yates, C. M., Filippis, I., Kelley, L. A., & Sternberg, M. J. E. (2014) · Journal of Molecular Biology · doi:10.1016/j.jmb.2014.04.026 · Cited by 254
Mental Disorders Following War in the Balkans
Priebe, S., Bogić, M., Ajduković, D., Frančišković, T., Galeazzi, G. M., Kučukalić, A., Lečić‐Toševski, D., Nexhmedin Morina, B., Popovski, M., Wang, D., et al. (2010) · Archives of General Psychiatry · doi:10.1001/archgenpsychiatry.2010.37 · Cited by 249
Development of a deep residual learning algorithm to screen for glaucoma from fundus photography
Shibata, N., Tanito, M., Mitsuhashi, K., Fujino, Y., Matsuura, M., Murata, H., & Asaoka, R. (2018) · Scientific Reports · doi:10.1038/s41598-018-33013-w · Cited by 251
Early Changes in Blood Urea Nitrogen Predict Mortality in Acute Pancreatitis
Wu, B. U., Johannes, R. S., Sun, X., Conwell, D. L., & Banks, P. A. (2009) · Gastroenterology · doi:10.1053/j.gastro.2009.03.056 · Cited by 245
Comparison of the GRACE, HEART and TIMI score to predict major adverse cardiac events in chest pain patients at the emergency department
Poldervaart, J. M., Langedijk, M., Backus, B. E., Dekker, I. M. C., Six, A. J., Doevendans, P. A., Hoes, A. W., & Reitsma, J. B. (2016) · International Journal of Cardiology · doi:10.1016/j.ijcard.2016.10.080 · Cited by 244
Development of a Claims-based Frailty Indicator Anchored to a Well-established Frailty Phenotype
Segal, J. B., Chang, H., Du, Y., Walston, J. D., Carlson, M. C., & Varadhan, R. (2017) · Medical Care · doi:10.1097/mlr.0000000000000729 · Cited by 250
Receiver Operating Characteristic (ROC) Area Under the Curve (AUC): A Diagnostic Measure for Evaluating the Accuracy of Predictors of Education Outcomes
Bowers, A. J., & Zhou, X. (2019) · Journal of Education for Students Placed at Risk (JESPAR) · doi:10.1080/10824669.2018.1523734 · Cited by 260
Prediction of Total Knee Replacement and Diagnosis of Osteoarthritis by Using Deep Learning on Knee Radiographs: Data from the Osteoarthritis Initiative
Leung, K., Zhang, B., Tan, J., Shen, Y., Geras, K. J., Babb, J. S., Cho, K., Chang, G. A., & Deniz, C. M. (2020) · Radiology · doi:10.1148/radiol.2020192091 · Cited by 251
Absolute and Relative Kinetic Changes of High-Sensitivity Cardiac Troponin T in Acute Coronary Syndrome and in Patients with Increased Troponin in the Absence of Acute Coronary Syndrome
Müeller, M., Biener, M., Vafaie, M., Doerr, S., Keller, T., Blankenberg, S., Katus, H. A., & Giannitsis, E. (2011) · Clinical Chemistry · doi:10.1373/clinchem.2011.171827 · Cited by 243
Is Carotid Intima-Media Thickness Useful in Cardiovascular Disease Risk Assessment?
del Sol, A. I., Moons, K. G. M., Hollander, M., Albert Hofman, D., Koudstaal, P. J., Grobbee, D. E., Breteler, M. M. B., Witteman, J. C. M., & Bots, M. L. (2001) · Stroke · doi:10.1161/01.str.32.7.1532 · Cited by 243
The Patient-Specific Functional Scale: Validity, Reliability, and Responsiveness in Patients With Upper Extremity Musculoskeletal Problems
Hefford, C., Abbott, J. H., Arnold, R., & Baxter, D. G. (2012) · Journal of Orthopaedic and Sports Physical Therapy · doi:10.2519/jospt.2012.3953 · Cited by 243
Improving Accuracy and Efficiency with Concurrent Use of Artificial Intelligence for Digital Breast Tomosynthesis
Conant, E. F., Toledano, A. Y., Periaswamy, S., Fotin, S. V., Go, J., Boatsman, J. E., & Hoffmeister, J. W. (2019) · Radiology Artificial Intelligence · doi:10.1148/ryai.2019180096 · Cited by 243
Evaluating Machine Learning Models and Their Diagnostic Value
Varoquaux, G., & Colliot, O. (2023) · Neuromethods · doi:10.1007/978-1-0716-3195-9_20 · Cited by 242
Estimating Activity and Sedentary Behavior from an Accelerometer on the Hip or Wrist
Rosenberger, M. E., Haskell, W. L., Albinali, F., Mota, S., Nawyn, J., & Intille, S. S. (2013) · Medicine & Science in Sports & Exercise · doi:10.1249/mss.0b013e31827f0d9c · Cited by 258
Mammographic Breast Density and the Gail Model for Breast Cancer Risk Prediction in a Screening Population
Tice, J. A., Cummings, S. R., Ziv, E., & Kerlikowske, K. M. (2005) · Breast Cancer Research and Treatment · doi:10.1007/s10549-005-5152-4 · Cited by 242
High‐order resting‐state functional connectivity network for MCI classification
Chen, X., Zhang, H., Gao, Y., Chong-yaw Wee, A., Li, G., Shen, D. G., & Initiative, T. A. D. N. (2016) · Human Brain Mapping · doi:10.1002/hbm.23240 · Cited by 244
Development of a clinical decision support system for severity risk prediction and triage of COVID-19 patients at hospital admission: an international multicentre study
Wu, G., Yang, P., Woodruff, H. C., Rao, X., Guiot, J., Frix, A.-N., Moutschen, M. P., Louis, R. E., Li, J., Li, J., et al. (2020) · European Respiratory Journal · doi:10.1183/13993003.01104-2020 · Cited by 243
Severity assessment of COVID-19 using CT image features and laboratory indices
Tang, Z., Zhao, W., Xie, X., Zhong, Z., Shi, F., Ma, T., Liu, J., & Shen, D. G. (2020) · Physics in Medicine and Biology · doi:10.1088/1361-6560/abbf9e · Cited by 242
Association of Extracellular Vesicle Biomarkers With Alzheimer Disease in the Baltimore Longitudinal Study of Aging
Kapogiannis, D., Mustapic, M., Shardell, M. D., Berkowitz, S. T., Diehl, T. C., Spangler, R. D., Tran, J., Lazaropoulos, M. P., Chawla, S., Gulyani, S., et al. (2019) · JAMA Neurology · doi:10.1001/jamaneurol.2019.2462 · Cited by 241
Prostate specific antigen concentration at age 60 and death or metastasis from prostate cancer: case-control study
Vickers, A. J., Cronin, A. E., Björk, T., Manjer, J., Nilsson, P. M., Dahlin, A., Bjartell, A. S., Scardino, P. T., Ulmert, D., & Lilja, H. G. (2010) · BMJ · doi:10.1136/bmj.c4521 · Cited by 241
Multicentre validation of the bedside paediatric early warning system score: a severity of illness score to detect evolving critical illness in hospitalised children
Parshuram, C. S., Duncan, H. P., Joffe, A. R., FARRELL, C. A., Lacroix, J., Middaugh, K., Hutchison, J. S., Wensley, D., Blanchard, N., Beyene, J., et al. (2011) · Critical Care · doi:10.1186/cc10337 · Cited by 241
National Early Warning Score 2 (NEWS2) on admission predicts severe disease and in-hospital mortality from Covid-19 – a prospective cohort study
Myrstad, M., Ihle‐Hansen, H., Tveita, A. A., Andersen, E. L., Nygård, S., Tveit, A., & Berge, T. (2020) · Scandinavian Journal of Trauma Resuscitation and Emergency Medicine · doi:10.1186/s13049-020-00764-3 · Cited by 244
Robust machine learning segmentation for large-scale analysis of heterogeneous clinical brain MRI datasets
Billot, B., Magdamo, C., Cheng, Y., Arnold, S. E., Das, S., & Iglesias, J. E. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2216399120 · Cited by 261
Optimizing the use of the AUDIT for alcohol screening in college students.
DeMartini, K. S., & Carey, K. B. (2012) · Psychological Assessment · doi:10.1037/a0028519 · Cited by 240
Performance success or failure is influenced by weeks lost to injury and illness in elite Australian track and field athletes: A 5-year prospective study
Raysmith, B. P., & Drew, M. K. (2016) · Journal of science and medicine in sport · doi:10.1016/j.jsams.2015.12.515 · Cited by 240
Signal Detectability
Centor, R. M. (1991) · Medical Decision Making · doi:10.1177/0272989x9101100205 · Cited by 240
Testing for improvement in prediction model performance
PEPE, M. S., Kerr, K. F., Longton, G., & Wang, Z. (2013) · Statistics in Medicine · doi:10.1002/sim.5727 · Cited by 239
Prospective Comparison of Spleen and Liver Stiffness by Using Shear-Wave and Transient Elastography for Detection of Portal Hypertension in Cirrhosis
Elkrief, L., Rautou, P., Ronot, M., Lambert, S. A., Burgio, M. D., Francoz, C., Plessier, A., Durand, F., Valla, D. C., Lebrec, D., et al. (2014) · Radiology · doi:10.1148/radiol.14141210 · Cited by 238
Reducing Unnecessary Biopsy During Prostate Cancer Screening Using a Four-Kallikrein Panel: An Independent Replication
Vickers, A. J., Cronin, A. M., Roobol, M. J., Savage, C. J., Peltola, M. T., Pettersson, K. S. I., Scardino, P. T., Schröder, F. H., & Lilja, H. G. (2010) · Journal of Clinical Oncology · doi:10.1200/jco.2009.24.1968 · Cited by 237
Clinical value of liver ultrasound for the diagnosis of nonalcoholic fatty liver disease in overweight and obese patients
Bril, F., Ortiz‐Lopez, C., Lomonaco, R., Orsak, B., Freckleton, M. W., Chintapalli, K. N., Hardies, J., Lai, S., Solano, F., Tio, F. O., et al. (2015) · Liver International · doi:10.1111/liv.12840 · Cited by 237
Standardization of ROC Curve Areas for Diagnostic Evaluation of Liver Fibrosis Markers Based on Prevalences of Fibrosis Stages
Poynard, T., Halfon, P., Castéra, L., Munteanu, M., Imbert‐Bismut, F., Ratziu, V., Benhamou, Y., Bourlière, M., de Lédinghen, V., & FibroPaca Group (2007) · Clinical Chemistry · doi:10.1373/clinchem.2007.085795 · Cited by 239
Comparative analysis of urinary biomarkers for early detection of acute kidney injury following cardiopulmonary bypass
Liangos, O., Tighiouart, H., Perianayagam, M. C., Kolyada, A. Y., Han, W. K., Wald, R., Bonventre, J. V., & Jaber, B. L. (2009) · Biomarkers · doi:10.1080/13547500903067744 · Cited by 236
SCREENING FOR DEPRESSION IN ADOLESCENTS: VALIDITY OF THE PATIENT HEALTH QUESTIONNAIRE IN PEDIATRIC CARE: Research Article: Validity of the PHQ in Pediatric Care
Allgaier, A.-K., Pietsch, K., Frühe, B., Sigl-Glöckner, J., & Schulte-Körne, G. (2012) · Depression and Anxiety · doi:10.1002/da.21971 · Cited by 235
Prognostic Significance of Blood Lactate and Lactate Clearance in Trauma Patients
Régnier, M.-A., Raux, M., Manach, Y. L., Asencio, Y. Y., Gaillard, J., Devilliers, C., Langeron, O., & Riou, B. (2012) · Anesthesiology · doi:10.1097/aln.0b013e318273349d · Cited by 235
The Changes in Pulse Pressure Variation or Stroke Volume Variation After a “Tidal Volume Challenge” Reliably Predict Fluid Responsiveness During Low Tidal Volume Ventilation*
Myatra, S. N., Prabu, N. R., Divatia, J. V., Monnet, X., Kulkarni, A. P., & Teboul, J. (2016) · Critical Care Medicine · doi:10.1097/ccm.0000000000002183 · Cited by 240
Roughly balanced bagging for imbalanced data
Hido, S., Kashima, H., & Takahashi, Y. (2009) · Statistical Analysis and Data Mining The ASA Data Science Journal · doi:10.1002/sam.10061 · Cited by 244
Hypothesis testing of diagnostic accuracy for multiple readers and multiple tests an anova approach with dependent observations
Obuchowski, N. A., & Rockette, H. E. (1995) · Communications in Statistics - Simulation and Computation · doi:10.1080/03610919508813243 · Cited by 234
Breast Cancer: Diffusion Kurtosis MR Imaging—Diagnostic Accuracy and Correlation with Clinical-Pathologic Factors
Sun, K., Chen, X., Chai, W., Fei, X., Fu, C., Yan, X., Zhan, Y., Chen, K., Shen, K., & Yan, F. (2015) · Radiology · doi:10.1148/radiol.15141625 · Cited by 233
Diagnostic Classification of the Instantaneous Wave-Free Ratio Is Equivalent to Fractional Flow Reserve and Is Not Improved With Adenosine Administration
Sen, S., Asrress, K., Nijjer, S. S., Petraco, R., Malik, I. S., Foale, R. A., Mikhail, G., Foin, N., Broyd, C., Hadjiloizou, N., et al. (2013) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2013.01.034 · Cited by 233
The 3‐month readmission rate remains unacceptably high in a large North American cohort of patients with cirrhosis
Bajaj, J. S., Reddy, K. R., Tandon, P., Wong, F. S.-H., Kamath, P. S., García–Tsao, G., Maliakkal, B. J., Biggins, S. W., Thuluvath, P. J., Fallon, M. B., et al. (2015) · Hepatology · doi:10.1002/hep.28414 · Cited by 233
High Neutrophil-to-Lymphocyte Ratio Predicts Stroke-Associated Pneumonia
Nam, K., Kim, T. J., Lee, J. S., Kwon, H., Lee, Y.-S., Ko, S., & Yoon, B. (2018) · Stroke · doi:10.1161/strokeaha.118.021228 · Cited by 233
Reliabilität und Validität der Wender-Utah-Rating-Scale-Kurzform: Retrospektive Erfassung von Symptomen aus dem Spektrum der Aufmerksamkeitsdefizit/Hyperaktivitätsstörung
Retz-Junginger, P., Retz, W., Blocher, D., Stieglitz, R.-D., Georg, T., Supprian, T., Wender, P. H., & Rösler, M. (2003) · Der Nervenarzt · doi:10.1007/s00115-002-1447-4 · Cited by 233
Prevalence of and Factors Related to Discordance About Prognosis Between Physicians and Surrogate Decision Makers of Critically Ill Patients
White, D. B., Ernecoff, N., Buddadhumaruk, P., Hong, S., Weissfeld, L., Curtis, J. R., Luce, J. M., & Lo, B. (2016) · JAMA · doi:10.1001/jama.2016.5351 · Cited by 232
CT Features Associated with Epidermal Growth Factor Receptor Mutation Status in Patients with Lung Adenocarcinoma
Liu, Y., Kim, J., Qu, F., Liu, S., Wang, H. X., Balagurunathan, Y., Ye, Z., & Gillies, R. J. (2016) · Radiology · doi:10.1148/radiol.2016151455 · Cited by 237
Prognostic value of combined clinical and myocardial perfusion imaging data using machine learning
Betancur, J., Otaki, Y., Motwani, M., Fish, M., Lemley, M., Dey, D., Gransar, H., Tamarappoo, B., Germano, G., Sharir, T., et al. (2017) · JACC Cardiovascular Imaging · doi:10.1016/j.jcmg.2017.07.024 · Cited by 220
N-Terminal Pro–B-Type Natriuretic Peptide for Risk Assessment in Patients With Atrial Fibrillation
Hijazi, Z., Wallentin, L., Siegbahn, A., Andersson, U., Christersson, C., Ezekowitz, J., Gersh, B. J., Hanna, M., Hohnloser, S., Horowitz, J., et al. (2013) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2012.11.082 · Cited by 231
Predicting Late Myocardial Recovery and Outcomes in the Early Hours of ST-Segment Elevation Myocardial Infarction
Larose, É., Rodés‐Cabau, J., Pîbarot, P., Rinfret, S., Proulx, G., Nguyen, C. M., Déry, J., Gleeton, O., Roy, L., Noël, B. M., et al. (2010) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2010.02.033 · Cited by 231
Growth Differentiation Factor 15 for Risk Stratification and Selection of an Invasive Treatment Strategy in Non–ST-Elevation Acute Coronary Syndrome
Wollert, K. C., Kempf, T., Lagerqvist, B., Lindahl, B., Olofsson, S., Allhoff, T., Peter, T., Siegbahn, A., Venge, P., Drexler, H., et al. (2007) · Circulation · doi:10.1161/circulationaha.107.697714 · Cited by 231
Prospective validation of a 1-hour algorithm to rule-out and rule-in acute myocardial infarction using a high-sensitivity cardiac troponin T assay
Reichlin, T., Twerenbold, R., Wildi, K. S., Giménez, M. R., Bergsma, N., Haaf, P., Druey, S., Puelacher, C., Moehring, B., Freese, M., et al. (2015) · Canadian Medical Association Journal · doi:10.1503/cmaj.141349 · Cited by 231

Authors: DeLong, E. R., DeLong, D. M., & Clarke-Pearson, D. L.
Affiliations: IQVIA (United States) (US)
IQVIA (United Kingdom) (GB)
ORCID: DeLong, Elizabeth R.
Venue: Biometrics
Publisher: JSTOR
Published: 1988-09
Volume: 44
Issue: 3
Pages: 837
ISSN: 0006-341X (ISSN-L)
DOI: 10.2307/2531595
OpenAlex: W2328176404
Cited by: 22,649
References: 12
Citations by year: 2026: 1897 · 2025: 1546 · 2024: 1644 · 2023: 1628 · 2022: 1861 · 2021: 1946 · 2020: 1651 · 2019: 1364 · 2018: 1193 · 2017: 1076 · 2016: 1001 · 2015: 924 · 2014: 775 · 2013: 745 · 2012: 621
FWCI: 4.52
Type: journal article
Language: en
Open access: closed
Locations: Biometrics (publishedVersion)
PubMed (publishedVersion)
Funders: Health Services Research and Development
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
Keywords: receiver operating characteristic, area under the ROC curve, diagnostic test comparison, nonparametric covariance estimation, sensitivity and false positive rate
MeSH terms: Algorithms, Analysis of Variance, Female, Humans, Intestinal Obstruction, Ovarian Neoplasms, Predictive Value of Tests, ROC Curve, Models, Statistical

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Comparing the Areas under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric Approach

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks