The DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Combining independent studies of a diagnostic test into a summary roc curve: Data‐analytic approaches and some additional considerations
Abstract
We consider how to combine several independent studies of the same diagnostic test, where each study reports an estimated false positive rate )FPR( and an estimated true positive rate )TPR(. We propose constructing a summary receiver operating characteristic )ROC( curve by the following steps. )i( Convert each FPR to its logistic transform U and each TPR to its logistic transform V after increasing each observed frequency by adding 1/2. )ii( For each study calculate D = V – U, which is the log odds ratio of TPR and FPR, and S = V + U, an implied function of test threshold; then plot each study's point )S i, D i(. )iii( Fit a robust‐resistant regression line to these points )or an equally weighted least‐squares regression line(, with V – U as the dependent variable. )iv( Back‐transform the line to ROC space. To avoid model‐dependent extrapolation from irrelevant regions of ROC space we propose defining a priori a value of FPR so large that the test simply would not be used at that FPR, and a value of TPR so low that the test would not be used at that TPR. Then )a( only data points lying in the thus defined north‐west rectangle of the unit square are used in the data analysis, and )b( the estimated summary ROC is depicted only within that subregion of the unit square. We illustrate the methods using simulated and real data sets, and we point to ways of comparing different tests and of taking into account the effects of covariates.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Statistical Aspects of the Analysis of Data From Retrospective Studies of Disease
(1959) · JNCI: Journal of the National Cancer Institute · doi:10.1093/jnci/22.4.719 · Cited by 14,689
Plackett, R. L., & Cox, D. R. (1971) · Journal of the Royal Statistical Society. Series A (General) · doi:10.2307/2344247 · Cited by 2,264
Extra-Binomial Variation in Logistic Linear Models
Williams, D. A. (1982) · Applied Statistics · doi:10.2307/2347977 · Cited by 798
Extra-Poisson Variation in Log-Linear Models
Breslow, N. E. (1984) · Applied Statistics · doi:10.2307/2347661 · Cited by 746
Computing an Exact Confidence Interval for the Common Odds Ratio in Several 2×2 Contingency Tables
Mehta, C. R., Patel, N. R., & Gray, R. (1985) · Journal of the American Statistical Association · doi:10.1080/01621459.1985.10478212 · Cited by 328
Regression of Area Mortality Rates on Explanatory Variables: What Weighting is Appropriate?
Pocock, S., Cook, D. G., & Beresford, S. A. A. (1981) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.2307/2346353 · Cited by 158
Computing an Exact Confidence Interval for the Common Odds Ratio in Several 2 × 2 Contingency Tables
Mehta, C. R., Patel, N. R., & Gray, R. M. (1985) · Journal of the American Statistical Association · doi:10.2307/2288562 · Cited by 155
Kardaun, J. W. P. F., & Kardaun, O. J. W. F. (1990) · Methods of Information in Medicine · doi:10.1055/s-0038-1634759 · Cited by 74
Mehta, C. R., & Walsh, S. J. (1992) · The American Statistician · doi:10.1080/00031305.1992.10475871 · Cited by 50
Sensitivities and Specificities of Diagnostic Tests
Swets, J. A. (1982) · JAMA · doi:10.1001/jama.1982.03330050032020 · Cited by 34
Cited by
Items 101–162 of 162
Onishi, A., Sugiyama, D., Kogata, Y., Saegusa, J., Sugimoto, T., Kawano, S., Morinobu, A., Nishimura, K., & Kumagai, S. (2012) · Journal of Clinical Microbiology · doi:10.1128/jcm.05267-11 · Cited by 328
Does This Patient Have Strep Throat?
Ebell, M. H. (2000) · JAMA · doi:10.1001/jama.284.22.2912 · Cited by 327
Bivariate Random Effects Meta-Analysis of ROC Curves
Arends, L. R., Hamza, T. H., van Houwelingen, J. C., Heijenbrok-Kal, M. H., Hunink, M. G. M., & Stijnen, T. (2008) · Medical Decision Making · doi:10.1177/0272989x08319957 · Cited by 325
A Systematic Review of the Diagnostic Accuracy of Natriuretic Peptides for Heart Failure
Doust, J. A., Glasziou, P. P., Pietrzak, E., & Dobson, A. J. (2004) · Archives of Internal Medicine · doi:10.1001/archinte.164.18.1978 · Cited by 323
Radiological Evaluation of Lymph Node Metastases in Patients With Cervical Cancer: A Meta-analysis
Scheidler, J. (1997) · JAMA · doi:10.1001/jama.1997.03550130070040 · Cited by 320
Lee, A., Fan, L. T. Y., Gin, T., Karmakar, M. K., & Ngan Kee, W. D. (2006) · Anesthesia & Analgesia · doi:10.1213/01.ane.0000217211.12232.55 · Cited by 319
Park, D. I., Ryu, S., Kim, Y.-H., Lee, S.-H., Lee, C. K., Eun, C. S., & Han, D. S. (2010) · American Journal of Gastroenterology · doi:10.1038/ajg.2010.179 · Cited by 318
Choi, H. J., Ju, W., Myung, S. K., & Kim, Y. (2010) · Cancer Science · doi:10.1111/j.1349-7006.2010.01532.x · Cited by 317
Honest, H., Bachmann, L. M., Gupta, J. K., Kleijnen, J., & Khan, K. S. (2002) · BMJ · doi:10.1136/bmj.325.7359.301 · Cited by 316
Arbyn, M., Sankaranarayanan, R., Muwonge, R., Keita, N., Dolo, A., Mbalawa, C. G., Nouhou, H., Sakande, B., Wesley, R., Somanathan, T., et al. (2008) · International Journal of Cancer · doi:10.1002/ijc.23489 · Cited by 311
Maw, A. M., Hassanin, A., Ho, P. M., McInnes, M. D. F., Moss, A., Juarez-Colunga, E., Soni, N. J., Miglioranza, M. H., Platz, E., DeSanto, K., et al. (2019) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.0703 · Cited by 309
Benefits and Costs of Using HPV Testing to Screen for Cervical Cancer
Mandelblatt, J. S. (2002) · JAMA · doi:10.1001/jama.287.18.2372 · Cited by 308
Lahaye, M. J., Engelen, S. M. E., Nelemans, P. J., Beets, G. L., van de Velde, C. J. H., van Engelshoven, J. M. A., & Beets-Tan, R. G. H. (2005) · Seminars in Ultrasound, CT and MRI · doi:10.1053/j.sult.2005.04.005 · Cited by 307
Role of US in Detection of Crohn Disease: Meta-Analysis
Fraquelli, M., Colli, A., Casazza, G., Paggi, S., Colucci, A., Massironi, S., Duca, P., & Conte, D. (2005) · Radiology · doi:10.1148/radiol.2361040799 · Cited by 307
Lamoth, F., Cruciani, M., Mengoli, C., Castagnola, E., Lortholary, O., Richardson, M., Marchetti, O., & (ECIL-3), O. B. O. T. T. E. C. O. I. I. L. (2012) · Clinical Infectious Diseases · doi:10.1093/cid/cir897 · Cited by 302
Systematic Review: Noninvasive Testing for Chlamydia trachomatis and Neisseria gonorrhoeae
Cook, R. L., Hutchison, S. L., Østergaard, L., Braithwaite, R. S., & Ness, R. B. (2005) · Annals of Internal Medicine · doi:10.7326/0003-4819-142-11-200506070-00010 · Cited by 300
Westerterp, M., van Westreenen, H. L., Reitsma, J. B., Hoekstra, O. S., Stoker, J., Fockens, P., Jager, P. L., Van Eck-Smit, B. L. F., Plukker, J. T. M., van Lanschot, J. J. B., et al. (2005) · Radiology · doi:10.1148/radiol.2363041042 · Cited by 299
Diagnostic Performance of Low-Dose CT for the Detection of Urolithiasis: A Meta-Analysis
Niemann, T., Kollmann, T., & Bongartz, G. (2008) · American Journal of Roentgenology · doi:10.2214/ajr.07.3414 · Cited by 296
Cochrane diagnostic test accuracy reviews
Leeflang, M. M., Deeks, J. J., Takwoingi, Y., & Macaskill, P. (2013) · Systematic Reviews · doi:10.1186/2046-4053-2-82 · Cited by 292
Cruciani, M., Scarparo, C., Malena, M., Bosco, O., Serpelloni, G., & Mengoli, C. (2004) · Journal of Clinical Microbiology · doi:10.1128/jcm.42.5.2321-2325.2004 · Cited by 291
Thornton, G. D., McPhail, M. J. W., Nayagam, S., Hewitt, M. J., Vlavianos, P., & Monahan, K. J. (2013) · Pancreatology · doi:10.1016/j.pan.2012.11.313 · Cited by 289
Identifying depression in primary care: a literature synthesis of case-finding instruments
Williams, J. W., Pignone, M., Ramirez, G., & Perez Stellato, C. (2002) · General Hospital Psychiatry · doi:10.1016/s0163-8343(02)00195-0 · Cited by 289
Traditional markers of excessive alcohol use
Conigrave, K. M., Davies, P., Haber, P., & Whitfield, J. B. (2003) · Addiction · doi:10.1046/j.1359-6357.2003.00581.x · Cited by 287
Collins, R., Burch, J., Cranny, G., Aguiar-Ibáñez, R., Craig, D., Wright, K., Berry, E., Gough, M., Kleijnen, J., & Westwood, M. (2007) · BMJ · doi:10.1136/bmj.39217.473275.55 · Cited by 287
Birim, Ö., Kappetein, A. P., Stijnen, T., & Bogers, A. J. J. C. (2005) · The Annals of Thoracic Surgery · doi:10.1016/j.athoracsur.2004.06.041 · Cited by 284
Dinh, M. T., Abad, C. L., & Safdar, N. (2008) · Clinical Infectious Diseases · doi:10.1086/590011 · Cited by 284
Freeman, S. C., Kerby, C. R., Patel, A., Cooper, N. J., Quinn, T., & Sutton, A. J. (2019) · BMC Medical Research Methodology · doi:10.1186/s12874-019-0724-x · Cited by 283
Prediction of sepsis patients using machine learning approach: a meta-analysis
Islam, M. M., Nasrin, T., Walther, B. A., Wu, C.-C., Yang, H.-C., & Li, Y.-C. (2019) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2018.12.027 · Cited by 259
Kinkel, K., Lu, Y., Mehdizade, A., Pelte, M.-F., & Hricak, H. (2005) · Radiology · doi:10.1148/radiol.2361041618 · Cited by 280
Iliodromiti, S., Kelsey, T. W., Anderson, R. A., & Nelson, S. M. (2013) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jc.2013-1393 · Cited by 280
Evennett, N. J., Petrov, M. S., Mittal, A., & Windsor, J. A. (2009) · World Journal of Surgery · doi:10.1007/s00268-009-0074-7 · Cited by 270
Arida, A., Kyprianou, M., Kanakis, M., & Sfikakis, P. P. (2010) · BMC Musculoskeletal Disorders · doi:10.1186/1471-2474-11-44 · Cited by 269
Hamon, M., Fau, G., Née, G., Ehtisham, J., Morello, R., & Hamon, M. (2010) · Journal of Cardiovascular Magnetic Resonance · doi:10.1186/1532-429x-12-29 · Cited by 268
Predictors of common bile duct stones prior to cholecystectomy: a meta-analysis
Abboud, P.-A. C., Malet, P. F., Berlin, J. A., Staroscik, R., Cabana, M. D., Clarke, J. R., Shea, J. A., Schwartz, J. S., & Williams, S. V. (1996) · Gastrointestinal Endoscopy · doi:10.1016/s0016-5107(96)70098-6 · Cited by 267
Reese, G. E., Constantinides, V. A., Simillis, C., Darzi, A. W., Orchard, T. R., Fazio, V. W., & Tekkis, P. P. (2006) · The American Journal of Gastroenterology · doi:10.1111/j.1572-0241.2006.00840.x · Cited by 267
CA 125, PET alone, PET–CT, CT and MRI in diagnosing recurrent ovarian carcinoma
Gu, P., Pan, L.-L., Wu, S.-Q., Sun, L., & Huang, G. (2009) · European Journal of Radiology · doi:10.1016/j.ejrad.2008.02.019 · Cited by 265
The Test of Lasègue: Systematic Review of the Accuracy in Diagnosing Herniated Discs
Devillé, W. L. J. M., van der Windt, D. A. W. M., Dzaferagić, A., Bezemer, P. D., & Bouter, L. M. (2000) · Spine · doi:10.1097/00007632-200005010-00016 · Cited by 264
Mortality predictions in the intensive care unit: Comparing physicians with scoring systems*
Sinuff, T., Adhikari, N. K. J., Cook, D. J., Schünemann, H. J., Griffith, L. E., Rocker, G., & Walter, S. D. (2006) · Critical Care Medicine · doi:10.1097/01.ccm.0000201881.58644.41 · Cited by 262
Makrydimas, G., Sotiriadis, A., & Ioannidis, J. P. A. (2003) · American Journal of Obstetrics and Gynecology · doi:10.1067/s0002-9378(03)00645-8 · Cited by 258
Yang, X., & Du, B. (2014) · Critical Care · doi:10.1186/s13054-014-0650-6 · Cited by 258
Performance of methods for meta-analysis of diagnostic test accuracy with few studies or sparse data
Takwoingi, Y., Guo, B., Riley, R. D., & Deeks, J. J. (2017) · Statistical Methods in Medical Research · doi:10.1177/0962280215592269 · Cited by 257
Statistics in medical journals: some recent trends
Altman, D. G. (2000) · Statistics in Medicine · doi:10.1002/1097-0258(20001215)19:23<3275::aid-sim626>3.0.co;2-m · Cited by 254
Noninvasive Fetal Sex Determination Using Cell-Free Fetal DNA: A Systematic Review and Meta-analysis
Devaney, S. A., Palomaki, G. E., Scott, J. A., & Bianchi, D. W. (2011) · JAMA · doi:10.1001/jama.2011.1114 · Cited by 247
Diagnostic test accuracy: methods for systematic review and meta-analysis
Campbell, J. M., Klugar, M., Ding, S., Carmody, D. P., Hakonsen, S. J., Jadotte, Y. T., White, S., & Munn, Z. (2015) · International Journal of Evidence-Based Healthcare · doi:10.1097/xeb.0000000000000061 · Cited by 193
(2003) · PubMed · Cited by 190
A systematic review of the diagnosis of occupational asthma
Beach, J., Russell, K., Blitz, S., Hooton, N., Spooner, C., Lemiere, C., Tarlo, S. M., & Rowe, B. H. (2007) · Chest · doi:10.1378/chest.06-0492 · Cited by 128
Kang, S., Wu, Y., & Li, X. (2004) · Atherosclerosis · doi:10.1016/j.atherosclerosis.2004.08.005 · Cited by 126
Meng, X., Xie, L., & Song, W. (2013) · Ultrasound in Medicine & Biology · doi:10.1016/j.ultrasmedbio.2013.05.017 · Cited by 118
Dendukuri, N., Schiller, I., Joseph, L., & Pai, M. (2012) · Biometrics · doi:10.1111/j.1541-0420.2012.01773.x · Cited by 117
A system for rating the stability and strength of medical evidence
Treadwell, J. R., Tregear, S. J., Reston, J. T., & Turkelson, C. M. (2006) · BMC Medical Research Methodology · doi:10.1186/1471-2288-6-52 · Cited by 93
Dodd, S. R., Lancaster, G. A., Craig, J. V., Smyth, R. L., & Williamson, P. R. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2005.10.004 · Cited by 90
Accuracy of the GLIM criteria for diagnosing malnutrition: A systematic review and meta-analysis
Huo, Z., Chong, F., Yin, L., Lu, Z., Liu, J., & Xu, H. (2022) · Clinical Nutrition · doi:10.1016/j.clnu.2022.04.005 · Cited by 88
Suzuki, R., Thosani, N., Annangi, S., Guha, S., & Bhutani, M. S. (2014) · Pancreatology · doi:10.1016/j.pan.2014.07.006 · Cited by 84
de Sousa, M. R., Morillo, C. A., Rabelo, F. T., Filho, A. M. N., & Ribeiro, A. L. P. (2008) · European Journal of Heart Failure · doi:10.1016/j.ejheart.2008.07.002 · Cited by 72
Toddler Screening for Autism Spectrum Disorder: A Meta-Analysis of Diagnostic Accuracy
Sánchez-García, A. B., Galindo-Villardón, P., Nieto-Librero, A. B., Martín-Rodero, H., & Robins, D. L. (2019) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-018-03865-2 · Cited by 66
JOSEPH, S., SELVARAJ, J., MANI, I., KUMARAGURUPARI, T., SHANG, X., MUDGIL, P., RAVILLA, T., & HE, M. (2024) · American Journal of Ophthalmology · doi:10.1016/j.ajo.2024.02.012 · Cited by 69
Xu, Y., Ding, W., Wang, Y., Tan, Y., Xi, C., Ye, N., Wu, D., & Xu, X. (2021) · PLOS ONE · doi:10.1371/journal.pone.0246892 · Cited by 42
Yu, D., Sui, L., & Zhang, N. (2020) · Journal of Ultrasound in Medicine · doi:10.1002/jum.15123 · Cited by 33
Michelet, D., Julien-Marsollier, F., Vacher, T., Bellon, M., Skhiri, A., Bruneau, B., Fournier, J., Diallo, T., Luce, V., Brasher, C., et al. (2019) · Sleep Medicine · doi:10.1016/j.sleep.2018.09.027 · Cited by 22
Ban, J.-W., Emparanza, J. I., Urreta, I., & Burls, A. (2016) · PLOS ONE · doi:10.1371/journal.pone.0145779 · Cited by 18
Meta-analysis and structured literature review in radiology1
Doria, A. S. (2005) · Academic Radiology · doi:10.1016/j.acra.2005.01.007 · Cited by 9
Yu, D., Zhang, N., Sun, H., & Liu, M. (2023) · Journal of Clinical Ultrasound · doi:10.1002/jcu.23507 · Cited by 5
Metadata
- Authors
- Moses, L. E., Shapiro, D., & Littenberg, B.
- Venue
- Statistics in Medicine
- Publisher
- Wiley
- Published
- 1993-07-30
- Volume
- 12
- Issue
- 14
- Pages
- 1293-1316
- ISSN
- 0277-6715 (ISSN-L)
- 1097-0258
- OpenAlex
- W2113009004
- Cited by
- 1,405
- References
- 13
- FWCI
- 12.15
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.