Cited by 22,649Imbalanced Data Classification TechniquesStatistical Methods in EpidemiologyStatistical Methods in Clinical Trials

The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.

Comparing the Areas under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric Approach

DeLong, Elizabeth R., DeLong, David M., Clarke-Pearson, Daniel L.·1988·Biometrics

DOI 10.2307/2531595

Open this paper in CoChat — free account

Abstract

Methods of evaluating and comparing the performance of diagnostic tests are of increasing importance as new tests are developed and marketed. When a test is based on an observed variable that lies on a continuous or graded scale, an assessment of the overall value of the test can be made through the use of a receiver operating characteristic (ROC) curve. The curve is constructed by varying the cutpoint used to determine which values of the observed variable will be considered abnormal and then plotting the resulting sensitivities against the corresponding false positive rates. When two or more empirical curves are constructed based on tests performed on the same individuals, statistical analysis on differences between curves must take into account the correlated nature of the data. This paper presents a nonparametric approach to the analysis of areas under correlated ROC curves, by using the theory on generalized U-statistics to generate an estimated covariance matrix.

Integrity signals

Registry

Crossref · 10.2307/2531595

Cites

12 of 12 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

The meaning and use of the area under a receiver operating characteristic (ROC) curve.
Hanley, J. A., & McNeil, B. J. (1982) · Radiology · doi:10.1148/radiology.143.1.7063747 · Cited by 21,856
Basic principles of ROC analysis
Metz, C. E. (1978) · Seminars in Nuclear Medicine · doi:10.1016/s0001-2998(78)80014-2 · Cited by 6,122
A Class of Statistics with Asymptotically Normal Distribution
Hoeffding, W. (1992) · Springer series in statistics · doi:10.1007/978-1-4612-0919-5_20 · Cited by 1,898
The area above the ordinal dominance graph and the area below the receiver operating characteristic graph
Bamber, D. (1975) · Journal of Mathematical Psychology · doi:10.1016/0022-2496(75)90001-2 · Cited by 1,452
Untitled work
(1982) · Cited by 1,288
Maximum-likelihood estimation of parameters of signal-detection theory and determination of confidence intervals—Rating-method data
Dorfman, D. D., & Alf, E. F. (1969) · Journal of Mathematical Psychology · doi:10.1016/0022-2496(69)90019-4 · Cited by 878
A New Approach for Testing the Significance of Differences Between ROC Curves Measured from Correlated Data
Metz, C. E., Wang, P.-L., & Kronman, H. B. (1984) · Information Processing in Medical Imaging · doi:10.1007/978-94-009-6045-9_25 · Cited by 359
Jackknifing $U$-Statistics
Arvesen, J. N. (1969) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177697287 · Cited by 286
On Some Convergence Properties of UStatistics
Sen, P. K. (1960) · Calcutta Statistical Association Bulletin · doi:10.1177/0008068319600101 · Cited by 182
Surgical management of bowel obstruction in advanced ovarian carcinoma.
Krebs, H. B., & Goplerud, D. R. (1983) · PubMed · Cited by 132

Flagged first, then by citations

Show all 12 references

Sorted by citations

Items 601–700 of 1,197

Utility of a novel serum tumor biomarker HE4 in patients with endometrioid adenocarcinoma of the uterus
Moore, R. G., Brown, A., Miller, M. C., Badgwell, D. B., Lü, Z., Allard, W. J., Granai, C. O., Bast, R. C., & Lu, K. (2008) · Gynecologic Oncology · doi:10.1016/j.ygyno.2008.04.002 · Cited by 230
Prognostic Value of Echocardiography in Normotensive Patients With Acute Pulmonary Embolism
Pruszczyk, P., Goliszek, S., Lichodziejewska, B., Kostrubiec, M., Ciurzyński, M., Kurnicka, K., Dzikowska‐Diduch, O., Palczewski, P., & Wyzgał, A. (2014) · JACC. Cardiovascular imaging · doi:10.1016/j.jcmg.2013.11.004 · Cited by 230
Contributions of symptoms, signs, erythrocyte sedimentation rate, and C-reactive protein to a diagnosis of pneumonia in acute lower respiratory tract infection.
Hopstaken, R., Muris, J. W. M., Knottnerus, J. A., Kester, A. D. M., Rinkens, P. E. L. M., & Dinant, G. (2003) · Europe PMC (PubMed Central) · Cited by 230
History, Evolution and Future of Big Data and Analytics: A Bibliometric Analysis of Its Relationship to Performance in Organizations
Batistič, S., & van der Laken, P. A. (2019) · British Journal of Management · doi:10.1111/1467-8551.12340 · Cited by 247
Lung Ultrasound Integrated with Clinical Assessment for the Diagnosis of Acute Decompensated Heart Failure in the Emergency Department: A Randomized Controlled Trial
Pivetta, E. E., Goffi, A., Nazerian, P., Castagno, D., Tozzetti, C., Tizzani, P., Tizzani, M., Porrino, G., Ferreri, E., Busso, V., et al. (2019) · European Journal of Heart Failure · doi:10.1002/ejhf.1379 · Cited by 230
Detection of Glaucoma With Scanning Laser Polarimetry
Weinreb, R. N. (1998) · Archives of Ophthalmology · doi:10.1001/archopht.116.12.1583 · Cited by 230
Machine learning model to predict mental health crises from electronic health records
Garriga, R., Mas, J., Abraha, S., Nolan, J., Harrison, O., Tadros, G., & Matic, A. (2022) · Nature Medicine · doi:10.1038/s41591-022-01811-5 · Cited by 229
Prognostic Value of B-Type Natriuretic Peptides in Patients With Stable Coronary Artery Disease
Omland, T., Sabatine, M. S., Jablonski, K. A., Rice, M. M., Hsia, J. A., Wergeland, R., Landaas, S., Rouleau, J. L., DOMANSKI, M. J., Hall, C., et al. (2007) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2007.03.038 · Cited by 230
Near-infrared hyperspectral reflectance imaging for detection of bruises on pickling cucumbers
Ariana, D. P., Lu, R., & Guyer, D. E. (2006) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2006.04.001 · Cited by 229
Evaluation of biomarker panels for early stage ovarian cancer detection and monitoring for disease recurrence
Havrilesky, L. J., Whitehead, C. M., Rubatt, J. M., Cheek, R., Groelke, J. W., He, Q., Malinowski, D. P., Fischer, T. J., & Berchuck, A. (2008) · Gynecologic Oncology · doi:10.1016/j.ygyno.2008.04.041 · Cited by 229
Efficacy of Using Cognitive Status in Predicting Psychosis: A 7-Year Follow-Up
Riecher‐Rössler, A., Pflueger, M. O., Aston, J., Borgwardt, S., Brewer, W. J., Gschwandtner, U., & Stieglitz, R. (2009) · Biological Psychiatry · doi:10.1016/j.biopsych.2009.07.020 · Cited by 229
Arterial blood pressure during early sepsis and outcome
Dünser, M. W., Takala, J. A., Ulmer, H., Mayr, V. D., Luckner, G., Jochberger, S., Daudel, F., Lepper, P. M., Hasibeder, W. R., & Jakob, S. M. (2009) · Intensive Care Medicine · doi:10.1007/s00134-009-1427-2 · Cited by 233
Advanced Fibrosis in Nonalcoholic Fatty Liver Disease: Noninvasive Assessment with MR Elastography
Kim, D., Kim, W. R., Talwalkar, J. A., Kim, H. J., & Ehman, R. L. (2013) · Radiology · doi:10.1148/radiol.13121193 · Cited by 229
The value of big data for credit scoring: Enhancing financial inclusion using mobile phone data and social network analytics
Óskarsdóttir, M., Bravo, C., Sarraute, C., Vanthienen, J., & Baesens, B. (2018) · Applied Soft Computing · doi:10.1016/j.asoc.2018.10.004 · Cited by 229
Comparing the Mini-BESTest with the Berg Balance Scale to Evaluate Balance Disorders in Parkinson's Disease
King, L. A., Priest, K. C., Salarian, A., Pierce, D., & Horak, F. B. (2011) · Parkinson s Disease · doi:10.1155/2012/375419 · Cited by 228
A Prospective Analysis of the Prognostic Value of Biomarkers (FibroTest) in Patients with Chronic Hepatitis C
Ngo, Y., Munteanu, M., Messous, D., Charlotte, F., Imbert‐Bismut, F., Thabut, D., Lebray, P., Thibault, V., Benhamou, Y., Moussalli, J., et al. (2006) · Clinical Chemistry · doi:10.1373/clinchem.2006.070961 · Cited by 228
Enhanced diagnosis of advanced fibrosis and cirrhosis in individuals with NAFLD using FibroScan-based Agile scores
Sanyal, A. J., Foucquier, J., Younossi, Z. M., Harrison, S. A., Newsome, P. N., Chan, W., Yilmaz, Y. N., de Lédinghen, V., Costentin, C. E., Zheng, M., et al. (2022) · Journal of Hepatology · doi:10.1016/j.jhep.2022.10.034 · Cited by 228
Prediction of Hyperbilirubinemia in Near-Term and Term Infants
Stevenson, D. K., Fanaroff, A. A., Maisels, M. J., Young, B. W., Wong, R. J., Vreman, H. J., MacMahon, J. R., Yeung, C. Y., Seidman, D. S., Gale, R., et al. (2001) · PEDIATRICS · doi:10.1542/peds.108.1.31 · Cited by 228
Validation of a Machine Learning Model That Outperforms Clinical Risk Scoring Systems for Upper Gastrointestinal Bleeding
Shung, D. L., Au, B., Taylor, R. A., Tay, J. K., Laursen, S. B., Stanley, A. J., Dalton, H. R., Ngu, J., Schultz, M., & Laine, L. (2020) · Gastroenterology · doi:10.1053/j.gastro.2019.09.009 · Cited by 227
“For external validation, the AUC was calculated and then compared using a 2-tailed nonparametric method.(35) McNemar’s matched pairs test was used to compare specificities.”
Method
Comparison of machine learning and traditional classifiers in glaucoma diagnosis
Chan, K., Lee, T.-W., Sample, P. A., Goldbaum, M. H., Weinreb, R. N., & B, T. J. S. A. (2002) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2002.802012 · Cited by 230
Use of Procalcitonin Assays to Predict Serious Bacterial Infection in Young Febrile Infants
Milcent, K., Faesch, S., Guen, C. G., Dubos, F., Poulalhon, C., Badier, I., Marc, E., Laguille, C., de Pontual, L., Mosca, A., et al. (2015) · Archives of Pediatrics and Adolescent Medicine · doi:10.1001/jamapediatrics.2015.3210 · Cited by 227
Evaluation of Machine-Learning Algorithms for Predicting Opioid Overdose Risk Among Medicare Beneficiaries With Opioid Prescriptions
Lo-Ciganic, W.-H., Huang, J. L., Zhang, H. H., Weiss, J. C., Wu, Y., Kwoh, C. K., Donohue, J. M., Cochran, G., Gordon, A. J., Malone, D. C., et al. (2019) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.0968 · Cited by 228
The APPLE score: a novel and simple score for the prediction of rhythm outcomes after catheter ablation of atrial fibrillation
Kornej, J., Hindricks, G., Shoemaker, M. B., Husser, D., Arya, A., Sommer, P., Rolf, S., Saavedra, P. J., Kanagasundram, A. N., Whalen, S. P., et al. (2015) · Clinical Research in Cardiology · doi:10.1007/s00392-015-0856-x · Cited by 227
Infarct Volume Is a Pivotal Biomarker After Intra-Arterial Stroke Therapy
Yoo, A. J., Chaudhry, Z. A., Nogueira, R. G., Lev, M. H., Schaefer, P. W., Schwamm, L. H., Hirsch, J. A., & González, R. G. (2012) · Stroke · doi:10.1161/strokeaha.111.639401 · Cited by 226
A Novel Approach to Contrast-Enhanced Breast Magnetic Resonance Imaging for Screening
Mann, R. M., Mus, R. D. M., van Zelst, J. C. M., Geppert, C., Karssemeijer, N., & Platel, B. (2014) · Investigative Radiology · doi:10.1097/rli.0000000000000057 · Cited by 226
Quantity and Location of Aortic Valve Complex Calcification Predicts Severity and Location of Paravalvular Regurgitation and Frequency of Post-Dilation After Balloon-Expandable Transcatheter Aortic Valve Replacement
Khalique, O. K., Hahn, R. T., Gada, H., Nazif, T. M., Vahl, T. P., George, I., Kalesan, B., Forster, M., Williams, M. B., Leon, M. B., et al. (2014) · JACC: Cardiovascular Interventions · doi:10.1016/j.jcin.2014.03.007 · Cited by 226
The value of vital sign trends for detecting clinical deterioration on the wards
Churpek, M. M., Adhikari, R., & Edelson, D. P. (2016) · Resuscitation · doi:10.1016/j.resuscitation.2016.02.005 · Cited by 226
GALAD Score for Hepatocellular Carcinoma Detection in Comparison with Liver Ultrasound and Proposal of GALADUS Score
Yang, J. D., Addissie, B. D., Mara, K. C., Harmsen, W. S., Dai, J., Zhang, N., Wongjarupong, N., Ali, H. M., Ali, H. A., Hassan, F., et al. (2018) · Cancer Epidemiology Biomarkers & Prevention · doi:10.1158/1055-9965.epi-18-0281 · Cited by 230
A Deep Learning Approach for Assessment of Regional Wall Motion Abnormality From Echocardiographic Images
Kusunose, K., Abe, T., Haga, A., Fukuda, D., Yamada, H., Harada, M., & Sata, M. (2019) · JACC. Cardiovascular imaging · doi:10.1016/j.jcmg.2019.02.024 · Cited by 230
Inhospital and 1-year mortality of patients who develop worsening renal function following acute ST-elevation myocardial infarction
Goldberg, A., Hammerman, H., Petcherski, S., Zdorovyak, A., Yalonetsky, S., Kapeliovich, M. R., Agmon, Y., Markiewicz, W., & Aronson, D. (2005) · American Heart Journal · doi:10.1016/j.ahj.2004.09.055 · Cited by 225
On the statistical analysis of ROC curves
Thompson, M. L., & Zucchini, W. (1989) · Statistics in Medicine · doi:10.1002/sim.4780081011 · Cited by 225
Replication and extension of genome-wide association study results for obesity in 4923 adults from northern Sweden
Renström, F., Payne, F., Nordström, A., Brito, E. C., Rolandsson, O., Hallmans, G., Barroso, I., Nordström, P., & Franks, P. W. (2009) · Human Molecular Genetics · doi:10.1093/hmg/ddp041 · Cited by 225
Incremental Prognostic Value of Cardiac Computed Tomography in Coronary Artery Disease Using CONFIRM
Chow, B. J. W., Small, G. R., Yam, Y., Chen, L., Achenbach, S. S., Al-Mallah, M. H., Berman, D. S., Budoff, M. J., Cademartiri, F., Callister, T. Q., et al. (2011) · Circulation Cardiovascular Imaging · doi:10.1161/circimaging.111.964155 · Cited by 227
Absolute quantification of donor-derived cell-free DNA as a marker of rejection and graft injury in kidney transplantation: Results from a prospective observational study
Oellerich, M., Shipkova, M., Asendorf, T., Walson, P. D., Schauerte, V., Mettenmeyer, N., Kabakchiev, M., Hasche, G., Gröne, H., Friede, T., et al. (2019) · American Journal of Transplantation · doi:10.1111/ajt.15416 · Cited by 225
Biomarkers for staging fibrosis and non-alcoholic steatohepatitis in non-alcoholic fatty liver disease (the LITMUS project): a comparative diagnostic accuracy study
Vali, Y., Lee, J., Boursier, J., Petta, S., Wonders, K., Tiniakos, D., Bedossa, P., Geier, A., Francque, S., Allison, M., et al. (2023) · The Lancet Gastroenterology & Hepatology · doi:10.1016/s2468-1253(23)00017-1 · Cited by 224
Automated Quantification of 18F-Flutemetamol PET Activity for Categorizing Scans as Negative or Positive for Brain Amyloid: Concordance with Visual Image Reads
Thurfjell, L., Lilja, J., Lundqvist, R., Buckley, C. J., Smith, A. P. L., Vandenberghe, R. R., & Sherwin, P. F. (2014) · Journal of Nuclear Medicine · doi:10.2967/jnumed.114.142109 · Cited by 225
Comprehensive cancer-gene panels can be used to estimate mutational load and predict clinical benefit to PD-1 blockade in clinical practice
Campesato, L. F., Barroso‐Sousa, R., Jimenez, L., Corrêa, B., Sabbaga, J., Hoff, P. M., Reis, L. F. L., Galante, P. A. F., & Camargo, A. A. (2015) · Oncotarget · doi:10.18632/oncotarget.5950 · Cited by 224
Assessment of a Deep Learning Model Based on Electronic Health Record Data to Forecast Clinical Outcomes in Patients With Rheumatoid Arthritis
Norgeot, B., Glicksberg, B. S., Trupin, L. S., Lituiev, D., Gianfrancesco, M. A., Oskotsky, B., Schmajuk, G., Yazdany, J., & Butte, A. J. (2019) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.0606 · Cited by 228
Native T1-mapping detects the location, extent and patterns of acute myocarditis without the need for gadolinium contrast agents
Ferreira, V. M., Piechnik, S. K., Dall’Armellina, E., Karamitsos, T. D., Francis, J. M., Ntusi, N., Holloway, C. J., Choudhury, R. P., Kardos, A., Robson, M. D., et al. (2014) · Journal of Cardiovascular Magnetic Resonance · doi:10.1186/1532-429x-16-36 · Cited by 225
Impact of azathioprine and tumour necrosis factor antagonists on the need for surgery in newly diagnosed Crohn's disease
Peyrin‐Biroulet, L., Oussalah, A., Williet, N., Pillot, C., Bresler, L., & Bigard, M. (2011) · Gut · doi:10.1136/gut.2010.227884 · Cited by 223
C-terminal provasopressin (copeptin) is a strong prognostic marker in patients with heart failure after an acute myocardial infarction: results from the OPTIMAAL study
Voors, A. A., von Haehling, S., Anker, S. D., Hillege, H. L., Struck, J., Hartmann, O., Bergmann, A., Squire, I., van Veldhuisen, D. J., Dickstein, K., et al. (2009) · European Heart Journal · doi:10.1093/eurheartj/ehp098 · Cited by 222
Physical Activity, Sedentary Time, and Obesity in an International Sample of Children
KATZMARZYK, P. T., BARREIRA, T. V., BROYLES, S. T., CHAMPAGNE, C. M., CHAPUT, J.-P., FOGELHOLM, M., HU, G., JOHNSON, W. D., KURIYAN, R., KURPAD, A., et al. (2015) · Medicine & Science in Sports & Exercise · doi:10.1249/mss.0000000000000649 · Cited by 222
A blood-based diagnostic test incorporating plasma Aβ42/40 ratio, ApoE proteotype, and age accurately identifies brain amyloid status: findings from a multi cohort validity analysis
West, T., Kirmess, K. M., Meyer, M. R., Holubasch, M. S., Knapik, S. S., Hu, Y., Contois, J. H., Jackson, E. N., Harpstrite, S. E., Bateman, R. J., et al. (2021) · Molecular Neurodegeneration · doi:10.1186/s13024-021-00451-6 · Cited by 222
Attention Strongly Modulates Reliability of Neural Responses to Naturalistic Narrative Stimuli
Ki, J. J., Kelly, S. P., & Parra, L. C. (2016) · The Journal of Neuroscience · doi:10.1523/jneurosci.2942-15.2016 · Cited by 221
Screening for Primary Aldosteronism in Essential Hypertension: Diagnostic Accuracy of the Ratio of Plasma Aldosterone Concentration to Plasma Renin Activity
Schwartz, G. L., & Turner, S. T. (2005) · Clinical Chemistry · doi:10.1373/clinchem.2004.041780 · Cited by 221
Prediction of Prostate Cancer Risk: The Role of Prostate Volume and Digital Rectal Examination in the ERSPC Risk Calculators
Roobol, M. J., van Vugt, H. A., Loeb, S., Zhu, X., Bul, M., Bangma, C. H., van Leenders, A. G. J. L. H., Steyerberg, E. W., & Schröder, F. H. (2011) · European Urology · doi:10.1016/j.eururo.2011.11.012 · Cited by 221
Statistical and data reporting guidelines for the European Journal of Cardio-Thoracic Surgery and the Interactive CardioVascular and Thoracic Surgery
Hickey, G. L., Dunning, J., Seifert, B., Sodeck, G. H., Carr, M., Burger, H. U., & Beyersdorf, F. (2015) · European Journal of Cardio-Thoracic Surgery · doi:10.1093/ejcts/ezv168 · Cited by 221
Cardiac Magnetic Resonance Myocardial Feature Tracking for Optimized Prediction of Cardiovascular Events Following Myocardial Infarction
Eitel, I., Stiermaier, T., Lange, T., Rommel, K., Koschalka, A., Kowallick, J. T., Lotz, J., Kutty, S., Gutberlet, M., Hasenfuss, G. P., et al. (2018) · JACC. Cardiovascular imaging · doi:10.1016/j.jcmg.2017.11.034 · Cited by 221
An annotation-free whole-slide training approach to pathological classification of lung cancer types using deep learning
Chen, C., Chen, C., Yu, W.-H., Chen, S. S., Chang, Y., Hsu, T., Hsiao, M., Yeh, C., & Chen, C. (2021) · Nature Communications · doi:10.1038/s41467-021-21467-y · Cited by 221
Cirrhotic patients in the medical intensive care unit: Early prognosis and long-term survival*
Das, V., Boëlle, P., Galbois, A., Guidet, B., Maury, É., Carbonell, N., Moreau, R., & Offenstadt, G. (2010) · Critical Care Medicine · doi:10.1097/ccm.0b013e3181f3dea9 · Cited by 220
Comparison Between the Four-kallikrein Panel and Prostate Health Index for Predicting Prostate Cancer
Nordström, T., Vickers, A. J., Assel, M. J., Lilja, H. G., Grönberg, H., & Eklund, M. (2014) · European Urology · doi:10.1016/j.eururo.2014.08.010 · Cited by 220
Predicting urinary tract infections in the emergency department with machine learning
Taylor, R. A., Moore, C. L., Cheung, K., & Brandt, C. (2018) · PLoS ONE · doi:10.1371/journal.pone.0194085 · Cited by 224
Identification and validation of an explainable prediction model of acute kidney injury with prognostic implications in critically ill children: a prospective multicenter cohort study
Hu, J., Xu, J., Li, M., Jiang, Z., Mao, J., Feng, L., Miao, K., Li, H., Chen, J., Bai, Z., et al. (2024) · EClinicalMedicine · doi:10.1016/j.eclinm.2023.102409 · Cited by 226
Diagnostic accuracy of procalcitonin, neutrophil-lymphocyte count ratio, C-reactive protein, and lactate in patients with suspected bacterial sepsis
Ljungström, L., Pernestig, A.-K., Jacobsson, G., Andersson, R., Usener, B., & Tilevik, D. (2017) · PLOS ONE · doi:10.1371/journal.pone.0181704 · Cited by 219
VERIFY (VERification of Instantaneous Wave-Free Ratio and Fractional Flow Reserve for the Assessment of Coronary Artery Stenosis Severity in EverydaY Practice)
Berry, C., Veer, M. V. ’., Witt, N., Kala, P., Boček, O., Pyxaras, S. A., McClure, J., Fearon, W. F., Barbato, E., Tonino, P. A. L., et al. (2013) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2012.09.065 · Cited by 219
Refining Angiographic Biomarkers of Revascularization
Yoo, A. J., Simonsen, C. Z., Prabhakaran, S., Chaudhry, Z. A., Issa, M. A., Fugate, J. E., Linfante, I., Liebeskind, D. S., Khatri, P., Jovin, T. G., et al. (2013) · Stroke · doi:10.1161/strokeaha.113.001990 · Cited by 219
Association of Intraoperative Transcranial Doppler Monitoring Variables With Stroke From Carotid Endarterectomy
Ackerstaff, R. G. A., Moons, K. G. M., van de Vlasakker, C. J. W., Moll, F. L., Vermeulen, F. E. E., Algra, A., & Spencer, M. P. (2000) · Stroke · doi:10.1161/01.str.31.8.1817 · Cited by 219
Oxidation-Specific Biomarkers, Prospective 15-Year Cardiovascular and Stroke Outcomes, and Net Reclassification of Cardiovascular Events
Tsimikas, S., Willeit, P., Willeit, J., Santer, P., Mayr, M., Xu, Q., Mayr, A., Witztum, J. L., & Kiechl, S. J. (2012) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2012.08.979 · Cited by 219
Development and Validation of a Deep Learning System for Staging Liver Fibrosis by Using Contrast Agent–enhanced CT Images in the Liver
Choi, K. J., Jang, J. K., Lee, S. S., Sung, Y. S., Shim, W. H., Kim, H. S., Yun, J., Choi, J., Lee, Y., Kang, B.-K., et al. (2018) · Radiology · doi:10.1148/radiol.2018180763 · Cited by 226
Digital Assessment of Endothelial Function and Ischemic Heart Disease in Women
Matsuzawa, Y., Sugiyama, S., Sugamura, K., Nozaki, T., Ohba, K., Konishi, M., Matsubara, J., Sumida, H., Kaikita, K., Kojima, S., et al. (2010) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2009.10.073 · Cited by 219
Predicting Bacteremia in Patients with Community-Acquired Pneumonia
Metersky, M. L., Ma, A., Bratzler, D. W., & Houck, P. M. (2003) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.200309-1248oc · Cited by 218
Risk factors for complicated parapneumonic effusion and empyema on presentation to hospital with community-acquired pneumonia
Chalmers, J. D., Singanayagam, A., Murray, M., Scally, C., Fawzi, A., & Hill, A. T. (2009) · Thorax · doi:10.1136/thx.2008.105080 · Cited by 218
Detecting money laundering transactions with machine learning
Jullum, M., Løland, A., Huseby, R. B., Ånonsen, G., & Lorentzen, J. (2020) · Journal of Money Laundering Control · doi:10.1108/jmlc-07-2019-0055 · Cited by 157
A distribution-free procedure for comparing receiver operating characteristic curves for a paired experiment
Venkatraman, E. S. (1996) · Biometrika · doi:10.1093/biomet/83.4.835 · Cited by 218
Consistent accuracy of the Australian weed risk assessment system across varied geographies
Gordon, D. R., Onderdonk, D. A., Fox, A. M., & Stocker, R. K. (2008) · Diversity and Distributions · doi:10.1111/j.1472-4642.2007.00460.x · Cited by 217
Growth Differentiation Factor 15, ST2, High-Sensitivity Troponin T, and N-Terminal Pro Brain Natriuretic Peptide in Heart Failure with Preserved vs. Reduced Ejection Fraction
Santhanakrishnan, R., Chong, J. P. C., Ng, T. P., Ling, L. H., Sim, D., Leong, K. T. G., Yeo, P. S. D., Ong, H. Y., Jaufeerally, F. R., Wong, R. C.-C., et al. (2012) · European Journal of Heart Failure · doi:10.1093/eurjhf/hfs130 · Cited by 217
Impact of Creatinine Calibration on Performance of GFR Estimating Equations in a Pooled Individual Patient Database
Stevens, L. A., Manzi, J., Levey, A. S., Chen, J., Deysher, A. E., Greene, T., Poggio, E. D., Schmid, C. H., Steffes, M. W., Zhang, Y., et al. (2007) · American Journal of Kidney Diseases · doi:10.1053/j.ajkd.2007.04.004 · Cited by 217
Early detection of Alzheimer's disease using M RI hippocampal texture
Sørensen, L., Igel, C., Hansen, N. L., Osler, M., Lauritzen, M., Rostrup, E., Nielsen, M., & Initiative, A. D. N. (2015) · Human Brain Mapping · doi:10.1002/hbm.23091 · Cited by 225
A comparative analysis of data preparation algorithms for customer churn prediction: A case study in the telecommunication industry
Coussement, K., Lessmann, S., & Verstraeten, G. (2016) · Decision Support Systems · doi:10.1016/j.dss.2016.11.007 · Cited by 235
Global habitat suitability of cold‐water octocorals
Yesson, C., Taylor, M. L., Tittensor, D. P., Davies, A. J., Guinotte, J., Baco, A. R., Black, J., Hall-Spencer, J. M., & Rogers, A. D. (2012) · Journal of Biogeography · doi:10.1111/j.1365-2699.2011.02681.x · Cited by 216
Detection of Rheumatoid Arthritis–Interstitial Lung Disease Is Enhanced by Serum Biomarkers
Doyle, T. J., Patel, A. S., Hatabu, H., Nishino, M., Wu, G., Osorio, J. C., Golzarri, M. F., Traslosheros, A., Chu, S. G., Frits, M. L., et al. (2015) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.201411-1950oc · Cited by 219
The New Injury Severity Score: A More Accurate Predictor of In-Hospital Mortality than the Injury Severity Score
Lavoie, A., Moore, L., LeSage, N., Liberman, M., & Sampalis, J. S. (2004) · The Journal of Trauma: Injury, Infection, and Critical Care · doi:10.1097/01.ta.0000075342.36072.ef · Cited by 216
A Molecular Host Response Assay to Discriminate Between Sepsis and Infection-Negative Systemic Inflammation in Critically Ill Patients: Discovery and Validation in Independent Cohorts
McHugh, L. C., Seldon, T. A., Brandon, R. A., Kirk, J. T., Rapisarda, A., Sutherland, A. J., Presneill, J., Venter, D. J., Lipman, J., Thomas, M., et al. (2015) · PLoS Medicine · doi:10.1371/journal.pmed.1001916 · Cited by 222
Hepatocellular Carcinoma Detection by Plasma Methylated DNA: Discovery, Phase I Pilot, and Phase II Clinical Validation
Kisiel, J. B., Dukek, B. A., Kanipakam, R. V. S. R., Ghoz, H. M., Yab, T. C., Berger, C. K., Taylor, W. R., Foote, P. H., Giama, N. H., Onyirioha, K., et al. (2018) · Hepatology · doi:10.1002/hep.30244 · Cited by 219
A comparison of droplet digital polymerase chain reaction (PCR), quantitative PCR and metabarcoding for species‐specific detection in environmental DNA
Wood, S. A., Pochon, X., Laroche, O., von Ammon, U., Adamson, J. E., & Zaiko, A. (2019) · Molecular Ecology Resources · doi:10.1111/1755-0998.13055 · Cited by 229
A Review of Analysis and Sample Size Calculation Considerations for Wilcoxon Tests
Divine, G., Norton, H. J., Hunt, R., & Dienemann, J. A. (2013) · Anesthesia & Analgesia · doi:10.1213/ane.0b013e31827f53d7 · Cited by 216
CT Signs of Right Ventricular Dysfunction
Kang, D. K., Thilo, C., Schoepf, U. J., Barraza, J. M., Nance, J. W., Bastarrika, G., Abro, J. A., Ravenel, J. G., Costello, P., & Goldhaber, S. Z. (2011) · JACC. Cardiovascular imaging · doi:10.1016/j.jcmg.2011.04.013 · Cited by 215
Optimal Cutoff Levels of More Sensitive Cardiac Troponin Assays for the Early Diagnosis of Myocardial Infarction in Patients With Renal Dysfunction
Twerenbold, R., Wildi, K. S., Jaeger, C., Giménez, M. R., Reiter, M., Reichlin, T., Walukiewicz, A., Gugala, M., Krivoshei, L., Marti, N., et al. (2015) · Circulation · doi:10.1161/circulationaha.114.014245 · Cited by 215
Prognostic Value of Changes in Galectin-3 Levels Over Time in Patients With Heart Failure
van der Velde, A. R., Gullestad, L. L., Ueland, T., Aukrust, P., Guo, Y. F., Adourian, A. S., Muntendam, P., van Veldhuisen, D. J., & de Boer, R. A. (2013) · Circulation Heart Failure · doi:10.1161/circheartfailure.112.000129 · Cited by 215
Improved Early Detection of Sepsis in the ED With a Novel Monocyte Distribution Width Biomarker
Crouser, E. D., Parrillo, J. E., Seymour, C. W., Angus, D. C., Bicking, K., Tejidor, L., Magari, R. T., Careaga, D., Williams, J., Closser, D. R., et al. (2017) · CHEST Journal · doi:10.1016/j.chest.2017.05.039 · Cited by 215
T‐MoCA: A valid phone screen for cognitive impairment in diverse community samples
Katz, M. J., Wang, C., Nester, C. O., Derby, C. A., Zimmerman, M. E., Lipton, R. B., Sliwinski, M. J., & Rabin, L. A. (2021) · Alzheimer's & Dementia: Diagnosis, Assessment & Disease Monitoring · doi:10.1002/dad2.12144 · Cited by 214
A Novel Tool for Reliable and Accurate Prediction of Renal Complications in Patients Undergoing Percutaneous Coronary Intervention
Gurm, H. S., Seth, M., Kooiman, J., & Share, D. A. (2013) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2013.03.026 · Cited by 215
Keratoconus Diagnosis with Optical Coherence Tomography Pachymetry Mapping
Li, Y., Meisler, D. M., Tang, M., Lu, A. T. H., Thakrar, V., Reiser, B. J., & Huang, D. (2008) · Ophthalmology · doi:10.1016/j.ophtha.2008.08.004 · Cited by 214
The European Network for the Study of Adrenal Tumors staging system is prognostically superior to the international union against cancer-staging system: A North American validation
Giovanni Lughezzani, A., Sun, M., Perrotte, P., Claudio Jeldres, A., Alasker, A., Hendrik Isbarn, A., Budäus, L., Shariat, S. F., Guazzoni, G. F., Montorsi, F., et al. (2010) · European Journal of Cancer · doi:10.1016/j.ejca.2009.12.007 · Cited by 214
Non-invasive myocardial work index identifies acute coronary occlusion in patients with non-ST-segment elevation-acute coronary syndrome
Boe, E., Russell, K. E., Eek, C. H., Eriksen, M., Remme, E. W., Smiseth, O. A., & Skulstad, H. (2015) · European Heart Journal - Cardiovascular Imaging · doi:10.1093/ehjci/jev078 · Cited by 214
Data augmentation using Generative Adversarial Networks (GANs) for GAN-based detection of Pneumonia and COVID-19 in chest X-ray images
Motamed, S., Rogalla, P., & Khalvati, F. (2021) · Informatics in Medicine Unlocked · doi:10.1016/j.imu.2021.100779 · Cited by 225
Advances in statistical methodology for the evaluation of diagnostic and laboratory tests
Campbell, G. D. (1994) · Statistics in Medicine · doi:10.1002/sim.4780130513 · Cited by 214
Simultaneous Prediction of New Morbidity, Mortality, and Survival Without New Morbidity From Pediatric Intensive Care
Pollack, M. M., Holubkov, R., Funai, T., BERGER, J. T., Clark, A. E., Meert, K. L., Berg, R. A., Carcillo, J. A., WESSEL, D. L., Moler, F. W., et al. (2015) · Critical Care Medicine · doi:10.1097/ccm.0000000000001081 · Cited by 214
Spatial Navigation in Preclinical Alzheimer’s Disease
Allison, S. L., Fagan, A. M., Morris, J. C., & Head, D. M. (2016) · Journal of Alzheimer s Disease · doi:10.3233/jad-150855 · Cited by 237
Serum Isoform [−2]proPSA Derivatives Significantly Improve Prediction of Prostate Cancer at Initial Biopsy in a Total PSA Range of 2–10 ng/ml: A Multicentric European Study
Lazzeri, M., Haese, A., TAILLE, A. D. L., Palou, J., McNicholas, T., Giovanni Lughezzani, A., Scattoni, V., Bini, V., Freschi, M., Sussman, A. N., et al. (2013) · European Urology · doi:10.1016/j.eururo.2013.01.011 · Cited by 212
Risk of Bacteremia for Febrile Young Children in the Post–Haemophilus influenzae Type b Era
Lee, G. M., & Harper, M. B. (1998) · Archives of Pediatrics and Adolescent Medicine · doi:10.1001/archpedi.152.7.624 · Cited by 212
Use of administrative claims data for identifying patients with cirrhosis
Nehra, M. S., Ma, Y., Clark, C., Amarasingham, R., Rockey, D. C., & Singal, A. G. (2013) · Journal of Clinical Gastroenterology · doi:10.1097/mcg.0b013e3182688d2f · Cited by 211
Oculomotor function in frontotemporal lobar degeneration, related disorders and Alzheimer's disease
Garbutt, S., Matlin, A., Hellmuth, J. M., Schenk, A. K., Johnson, J. K., Rosen, H. J., Dean, D., Kramer, J. H., Neuhaus, J. M., Miller, B. L., et al. (2008) · Brain · doi:10.1093/brain/awn047 · Cited by 216
Direct comparison of high-sensitivity-cardiac troponin I vs. T for the early diagnosis of acute myocardial infarction
Giménez, M. R., Twerenbold, R., Reichlin, T., Wildi, K. S., Haaf, P., Schaefer, M., Zellweger, C., Moehring, B., Stallone, F., Sou, S. M., et al. (2014) · European Heart Journal · doi:10.1093/eurheartj/ehu188 · Cited by 210
Inflammatory and metabolic biomarkers and risk of liver and biliary tract cancer
Aleksandrova, K., Boeing, H., Nöthlings, U., Jenab, M., Fedirko, V., Kaaks, R., Lukanova, A., Trichopoulou, A., Trichopoulos, D., Boffetta, P., et al. (2014) · Hepatology · doi:10.1002/hep.27016 · Cited by 217
Hyaluronic acid, an accurate serum marker for severe hepatic fibrosis in patients with non‐alcoholic fatty liver disease
Suzuki, A., Angulo, P., Lymp, J. F., Li, D., Satomura, S., & Lindor, K. D. (2005) · Liver International · doi:10.1111/j.1478-3231.2005.01064.x · Cited by 213
Serial Sampling of ST2 Predicts 90-Day Mortality Following Destabilized Heart Failure
Boisot, S., Beede, J., Isakson, S. R., Chiu, A., Clopton, P., Januzzi, J. L., Maisel, A. S., & Fitzgerald, R. L. (2008) · Journal of Cardiac Failure · doi:10.1016/j.cardfail.2008.06.415 · Cited by 210
A Computer-Aided Diagnosis System Using Artificial Intelligence for the Diagnosis and Characterization of Thyroid Nodules on Ultrasound: Initial Clinical Assessment
Choi, Y. J., Baek, J. H., Park, H. S., Shim, W. H., Kim, T. Y., Shong, Y. K., & Lee, J. H. (2017) · Thyroid · doi:10.1089/thy.2016.0372 · Cited by 210
An integrated magneto-electrochemical device for the rapid profiling of tumour extracellular vesicles from blood plasma
Park, J., Park, J. S., Huang, C.-H., Jo, A., Cook, K. A., Wang, R., Lin, H.-Y., Deun, J. V., Li, H., Min, J., et al. (2021) · Nature Biomedical Engineering · doi:10.1038/s41551-021-00752-7 · Cited by 212
Insulin resistance assessed by estimated glucose disposal rate and risk of incident cardiovascular diseases among individuals without diabetes: findings from a nationwide, population based, prospective cohort study
Zhang, Z., Zhao, L. Y., Lu, Y., Xiao, Y. H., & Zhou, X. (2024) · Cardiovascular Diabetology · doi:10.1186/s12933-024-02256-5 · Cited by 225

Authors: DeLong, E. R., DeLong, D. M., & Clarke-Pearson, D. L.
Affiliations: IQVIA (United States) (US)
IQVIA (United Kingdom) (GB)
ORCID: DeLong, Elizabeth R.
Venue: Biometrics
Publisher: JSTOR
Published: 1988-09
Volume: 44
Issue: 3
Pages: 837
ISSN: 0006-341X (ISSN-L)
DOI: 10.2307/2531595
OpenAlex: W2328176404
Cited by: 22,649
References: 12
Citations by year: 2026: 1897 · 2025: 1546 · 2024: 1644 · 2023: 1628 · 2022: 1861 · 2021: 1946 · 2020: 1651 · 2019: 1364 · 2018: 1193 · 2017: 1076 · 2016: 1001 · 2015: 924 · 2014: 775 · 2013: 745 · 2012: 621
FWCI: 4.52
Type: journal article
Language: en
Open access: closed
Locations: Biometrics (publishedVersion)
PubMed (publishedVersion)
Funders: Health Services Research and Development
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
Keywords: receiver operating characteristic, area under the ROC curve, diagnostic test comparison, nonparametric covariance estimation, sensitivity and false positive rate
MeSH terms: Algorithms, Analysis of Variance, Female, Humans, Intestinal Obstruction, Ovarian Neoplasms, Predictive Value of Tests, ROC Curve, Models, Statistical

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Comparing the Areas under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric Approach

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks