Evaluating the Yield of Medical Tests
Abstract
A method is presented for evaluating the amount of information a medical test provides about individual patients. Emphasis is placed on the role of a test in the evaluation of patients with a chronic disease. In this context, the yield of a test is best interpreted by analyzing the prognostic information it furnishes. Information from the history, physical examination, and routine procedures should be used in assessing the yield of a new test. As an example, the method is applied to the use of the treadmill exercise test in evaluating the prognosis of patients with suspected coronary artery disease. The treadmill test is shown to provide surprisingly little prognostic information beyond that obtained from basic clinical measurements.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
For binary outcome prediction models, the c-statistic is a reparameterization of Somer's D and can be derived from the distribution of the linear predictor, Cohen's effect size, or several correlation indices.
“For instance, for prediction models with a binary outcome, the c-statistic is a reparameterization of Somer’s D,19 and can also be derived from the distribution of the linear predictor,20 Cohen’s effect size,20 or several correlation indices.21”
From A framework for meta-analysis of prediction model studies with binary and time-to-event outcomes (2019) · cites this paper as [21]
Harrell's version of the c-statistic appears most widespread and recommended for time-to-event outcomes.
“Although validation studies do not commonly report the definition and estimation method of presented c-statistics, Harrell’s version21 appears to be most widespread and recommended.22”
From A framework for meta-analysis of prediction model studies with binary and time-to-event outcomes (2019) · cites this paper as [21]
Discrimination can be assessed across all time points using Harrell's c statistic.
“Also, discrimination can be assessed across all time points using Harrell's c statistic.”
From Developing clinical prediction models: a step-by-step guide (2024) · cites this paper as [83]
Cites
Regression Models and Life-Tables
Cox, D. R. (1992) · Springer Series in Statistics · doi:10.1007/978-1-4612-4380-9_37 · Cited by 26,297 · see citing works
Experimental studies of physical fitness in relation to age
Robinson, S. (1938) · European Journal of Applied Physiology · doi:10.1007/bf02011412 · Cited by 672 · see citing works
The role of the exercise test in the evaluation of patients for ischemic heart disease.
McNeer, J. F., Margolis, J. R., Lee, K. L., KISSLO, J. A., Peter, R. H., Kong, Y., Behar, V. S., Wallace, A. G., McCants, C. B., & Rosati, R. A. (1978) · Circulation · doi:10.1161/01.cir.57.1.64 · Cited by 497 · see citing works
Survival in medically treated coronary artery disease.
Harris, P. J., Harrell, F. E., Lee, K. L., Behar, V. S., & Rosati, R. A. (1979) · Circulation · doi:10.1161/01.cir.60.6.1259 · Cited by 377 · see citing works
Self-citationSurvival in medically treated coronary artery disease.
JHarris, P., EHarrell, F., LLee, K., SBehar, V., & ARosati, R. (1979) · Circulation · Cited by 252 · see citing works
A New Information System for Medical Practice
Rosati, R. A. (1975) · Archives of Internal Medicine · doi:10.1001/archinte.1975.00330080019003 · Cited by 219 · see citing works
Morris, S. N., Phillips, J. F., Jordan, J. W., et al. (1978) · The American Journal of Cardiology · doi:10.1016/0002-9149(78)90160-1 · Cited by 132 · see citing works
Udall, J. A., & Ellestad, M. (1977) · Circulation · doi:10.1161/01.cir.56.6.985 · Cited by 125 · see citing works
Podrid, P. J., Graboys, T. B., & Lown, B. (1981) · New England Journal of Medicine · doi:10.1056/nejm198111053051903 · Cited by 114 · see citing works
Diamond, G. A., Hirsch, M., Forrester, J. S., et al. (1981) · Circulation · doi:10.1161/01.cir.63.4.915 · Cited by 107 · see citing works
Fisher, L. D., Kennedy, J. W., CHAITMAN, B. R., et al. (1981) · Circulation · doi:10.1161/01.cir.63.5.987 · Cited by 90 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
New insights into the genetic etiology of Alzheimer’s disease and related dementias
Bellenguez, C., Küçükali, F., Jansen, I. E., Kleineidam, L., Moreno-Grau, S., Amin, N., Naj, A. C., Campos-Martin, R., Grenier-Boley, B., Andrade, V., et al. (2022) · Nature Genetics · doi:10.1038/s41588-022-01024-z · Cited by 2,498 · see citing works
Uno, H., Cai, T., Pencina, M. J., D'Agostino, R. B., & Wei, L. J. (2011) · Statistics in Medicine · doi:10.1002/sim.4154 · Cited by 1,535 · see citing works
Predicting cancer outcomes from histology and genomics using convolutional networks
Mobadersany, P., Yousefi, S., Amgad, M., Gutman, D. A., Barnholtz-Sloan, J. S., Velázquez Vega, J. E., Brat, D. J., & Cooper, L. A. D. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1717139115 · Cited by 1,084 · see citing works
Sartore-Bianchi, A., Trusolino, L., Martino, C., Bencardino, K., Lonardi, S., Bergamo, F., Zagonel, V., Leone, F., Depetris, I., Martinelli, E., et al. (2016) · The Lancet Oncology · doi:10.1016/s1470-2045(16)00150-9 · Cited by 1,016 · see citing works
Molecular International Prognostic Scoring System for Myelodysplastic Syndromes
Bernard, E., Tuechler, H., Greenberg, P. L., Hasserjian, R. P., Arango Ossa, J. E., Nannya, Y., Devlin, S. M., Creignou, M., Pinel, P., Monnier, L., et al. (2022) · NEJM Evidence · doi:10.1056/evidoa2200008 · Cited by 867 · see citing works
CT-based radiomic signature predicts distant metastasis in lung adenocarcinoma
Coroller, T. P., Grossmann, P., Hou, Y., Rios Velazquez, E., Leijenaar, R. T. H., Hermann, G., Lambin, P., Haibe-Kains, B., Mak, R. H., & Aerts, H. J. W. L. (2015) · Radiotherapy and Oncology · doi:10.1016/j.radonc.2015.02.015 · Cited by 695 · see citing works
Collins, G. S., Ogundimu, E. O., & Altman, D. G. (2016) · Statistics in Medicine · doi:10.1002/sim.6787 · Cited by 665 · see citing works
Developing clinical prediction models: a step-by-step guide
Efthimiou, O., Seo, M., Chalkou, K., Debray, T., Egger, M., & Salanti, G. (2024) · BMJ · doi:10.1136/bmj-2023-078276 · Cited by 544 · see citing works
A tutorial on calibration measurements and calibration models for clinical prediction models
Huang, Y., Li, W., Macheret, F., Gabriel, R. A., & Ohno-Machado, L. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz228 · Cited by 486 · see citing works
Ganna, A., & Ingelsson, E. (2015) · The Lancet · doi:10.1016/s0140-6736(15)60175-1 · Cited by 411 · see citing works
Parameters behind “Nonparametric” Statistics: Kendall's tau, Somers’ D and Median Differences
Newson, R. (2002) · The Stata Journal: Promoting communications on statistics and Stata · doi:10.1177/1536867x0200200103 · Cited by 394 · see citing works
Kosiborod, M., Inzucchi, S. E., Krumholz, H. M., Xiao, L., Jones, P. G., Fiske, S., Masoudi, F. A., Marso, S. P., & Spertus, J. A. (2008) · Circulation · doi:10.1161/circulationaha.107.740498 · Cited by 385 · see citing works
Defining the biological basis of radiomic phenotypes in lung cancer
Grossmann, P., Stringfield, O., El-Hachem, N., Bui, M. M., Rios Velazquez, E., Parmar, C., Leijenaar, R. T., Haibe-Kains, B., Lambin, P., Gillies, R. J., et al. (2017) · eLife · doi:10.7554/elife.23421 · Cited by 365 · see citing works
Gaziano, T. A., Young, C. R., Fitzmaurice, G., Atwood, S., & Gaziano, J. M. (2008) · The Lancet · doi:10.1016/s0140-6736(08)60418-3 · Cited by 366 · see citing works
Buxton, A. E., Lee, K. L., Hafley, G. E., Pires, L. A., Fisher, J. D., Gold, M. R., Josephson, M. E., Lehmann, M. H., & Prystowsky, E. N. (2007) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2007.04.095 · Cited by 356 · see citing works
Extreme lipoprotein(a) levels and improved cardiovascular risk prediction
Kamstrup, P. R., Tybjærg-Hansen, A., & Nordestgaard, B. G. (2013) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2012.12.023 · Cited by 284 · see citing works
Deep learning-based survival prediction for multiple cancer types using histopathology images
Wulczyn, E., Steiner, D. F., Xu, Z., Sadhwani, A., Wang, H., Flament-Auvigne, I., Mermel, C. H., Chen, P.-H. C., Liu, Y., & Stumpe, M. C. (2020) · PLOS ONE · doi:10.1371/journal.pone.0233678 · Cited by 266 · see citing works
A framework for meta-analysis of prediction model studies with binary and time-to-event outcomes
Debray, T. P., Damen, J. A., Riley, R. D., Snell, K., Reitsma, J. B., Hooft, L., Collins, G. S., & Moons, K. G. (2019) · Statistical Methods in Medical Research · doi:10.1177/0962280218785504 · Cited by 278 · see citing works
“For instance, for prediction models with a binary outcome, the c-statistic is a reparameterization of Somer’s D,(19) and can also be derived from the distribution of the linear predictor,(20) Cohen’s effect size,(20) or several correlation indices.(21) An overview of relevant equations for this derivation of the c-sta…”
MethodLong-term cancer survival prediction using multimodal deep learning
Vale-Silva, L. A., & Rohr, K. (2021) · Scientific Reports · doi:10.1038/s41598-021-92799-4 · Cited by 268 · see citing works
Vergouwe, Y., Moons, K. G. M., & Steyerberg, E. W. (2010) · American Journal of Epidemiology · doi:10.1093/aje/kwq223 · Cited by 243 · see citing works
Rim, T. H., Lee, C. J., Tham, Y.-C., Cheung, N., Yu, M., Lee, G., Kim, Y., Ting, D. S. W., Chong, C. C. Y., Choi, Y. S., et al. (2021) · The Lancet Digital Health · doi:10.1016/s2589-7500(21)00043-1 · Cited by 234 · see citing works
Interpretable survival prediction for colorectal cancer using deep learning
Wulczyn, E., Steiner, D. F., Moran, M., Plass, M., Reihs, R., Tan, F., Flament-Auvigne, I., Brown, T., Regitnig, P., Chen, P.-H. C., et al. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00427-2 · Cited by 231 · see citing works
Boursier, J., Hagström, H., Ekstedt, M., Moreau, C., Bonacci, M., Cure, S., Ampuero, J., Nasr, P., Tallab, L., Canivet, C. M., et al. (2022) · Journal of Hepatology · doi:10.1016/j.jhep.2021.12.031 · Cited by 206 · see citing works
Aktas, M. K. (2004) · JAMA · doi:10.1001/jama.292.12.1462 · Cited by 193 · see citing works
Collins, G. S., Ogundimu, E. O., Cook, J. A., Manach, Y. L., & Altman, D. G. (2016) · Statistics in Medicine · doi:10.1002/sim.6986 · Cited by 185 · see citing works
McGirt, M. J., Bydon, M., Archer, K. R., Devin, C. J., Chotai, S., Parker, S. L., Nian, H., Harrell, F. E., Speroff, T., Dittus, R. S., et al. (2017) · Journal of Neurosurgery: Spine · doi:10.3171/2016.11.spine16526 · Cited by 176 · see citing works
Self-citationThe index of prediction accuracy: an intuitive measure useful for evaluating risk prediction models
Kattan, M. W., & Gerds, T. A. (2018) · Diagnostic and Prognostic Research · doi:10.1186/s41512-018-0029-2 · Cited by 149 · see citing works
Ioachimescu, A. G., Brennan, D. M., Hoar, B. M., & Hoogwerf, B. J. (2010) · Obesity · doi:10.1038/oby.2009.453 · Cited by 147 · see citing works
Ali, S., Rhodes, L., Moreea, O., McMillan, D., Gilbody, S., Leach, C., Lucock, M., Lutz, W., & Delgadillo, J. (2017) · Behaviour Research and Therapy · doi:10.1016/j.brat.2017.04.006 · Cited by 139 · see citing works
Verbakel, J. Y., Steyerberg, E. W., Uno, H., De Cock, B., Wynants, L., Collins, G. S., & Van Calster, B. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.01.028 · Cited by 138 · see citing works
Validation of probabilistic predictions
Miller, M. E., Langefeld, C. D., Tierney, W. M., Hui, S. L., & McDonald, C. J. (1993) · Medical Decision Making · doi:10.1177/0272989x9301300107 · Cited by 129 · see citing works
Coronary Artery Calcium and Long-Term Risk of Death, Myocardial Infarction, and Stroke
Mitchell, J. D., Paisley, R., Moon, P., Novak, E., & Villines, T. C. (2018) · JACC: Cardiovascular Imaging · doi:10.1016/j.jcmg.2017.09.003 · Cited by 107 · see citing works
Group lasso regularized deep learning for cancer prognosis from multi-omics and clinical features
Xie, G., Dong, C., Kong, Y., Zhong, J. F., Li, M., & Wang, K. (2019) · Genes · doi:10.3390/genes10030240 · Cited by 102 · see citing works
Validation of prediction models in the presence of competing risks: a guide through modern methods
van Geloven, N., Giardiello, D., Bonneville, E. F., Teece, L., Ramspek, C. L., van Smeden, M., Snell, K. I. E., van Calster, B., Pohar-Perme, M., Riley, R. D., et al. (2022) · BMJ · doi:10.1136/bmj-2021-069249 · Cited by 106 · see citing works
A meta-learning approach for genomic survival analysis
Qiu, Y. L., Zheng, H., Devos, A., Selby, H., & Gevaert, O. (2020) · Nature Communications · doi:10.1038/s41467-020-20167-3 · Cited by 96 · see citing works
Kim, E. S. H., Ishwaran, H., Blackstone, E., & Lauer, M. S. (2007) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2007.08.003 · Cited by 77 · see citing works
Machine learning in prostate MRI for prostate cancer: current status and future opportunities
Li, H., Lee, C. H., Chia, D., Lin, Z., Huang, W., & Tan, C. H. (2022) · Diagnostics · doi:10.3390/diagnostics12020289 · Cited by 74 · see citing works
Leroux, A., Xu, S., Kundu, P., Muschelli, J., Smirnova, E., Chatterjee, N., & Crainiceanu, C. (2021) · The Journals of Gerontology: Series A · doi:10.1093/gerona/glaa250 · Cited by 66 · see citing works
Hippisley-Cox, J., & Coupland, C. (2015) · BMJ · doi:10.1136/bmj.h5441 · Cited by 63 · see citing works
Radiomics-Based Outcome Prediction for Pancreatic Cancer Following Stereotactic Body Radiotherapy
Parr, E., Du, Q., Zhang, C., Lin, C., Kamal, A., McAlister, J., Liang, X., Bavitz, K., Rux, G., Hollingsworth, M., et al. (2020) · Cancers · doi:10.3390/cancers12041051 · Cited by 61 · see citing works
Lessons and tips for designing a machine learning study using EHR data
Arbet, J., Brokamp, C., Meinzen-Derr, J., Trinkley, K. E., & Spratt, H. M. (2021) · Journal of Clinical and Translational Science · doi:10.1017/cts.2020.513 · Cited by 46 · see citing works
A new concordance measure for risk prediction models in external validation settings
van Klaveren, D., Gönen, M., Steyerberg, E. W., & Vergouwe, Y. (2016) · Statistics in Medicine · doi:10.1002/sim.6997 · Cited by 48 · see citing works
Assessing the incremental role of novel and emerging risk factors.
Cook, N. R. (2010) · Current Cardiovascular Risk Reports · doi:10.1007/s12170-010-0084-x · Cited by 39 · see citing works
Carrillo-Perez, F., Pizurica, M., Zheng, Y., Nandi, T. N., Madduri, R., Shen, J., & Gevaert, O. (2024) · Nature Biomedical Engineering · doi:10.1038/s41551-024-01193-8 · Cited by 37 · see citing works
Yasodhara, A., Dong, V., Azhie, A., Goldenberg, A., & Bhat, M. (2021) · Liver Transplantation · doi:10.1002/lt.25930 · Cited by 37 · see citing works
Time‐to‐event prediction using survival analysis methods for Alzheimer's disease progression
Sharma, R., Anand, H., Badr, Y., & Qiu, R. G. (2021) · Alzheimer's & Dementia: Translational Research & Clinical Interventions · doi:10.1002/trc2.12229 · Cited by 19 · see citing works
Federated transfer learning with differential privacy for multi-omics survival analysis
Wen, G., & Li, L. (2025) · Briefings in Bioinformatics · doi:10.1093/bib/bbaf166 · Cited by 17 · see citing works
Ledbetter, M. K., Tabacu, L., Leroux, A., Crainiceanu, C. M., & Smirnova, E. (2022) · Preventive Medicine · doi:10.1016/j.ypmed.2022.107303 · Cited by 16 · see citing works
Covariate‐adjusted measures of discrimination for survival data
White, I. R., Rapsomaniki, E., & for the Emerging Risk Factors Collaboration (2015) · Biometrical Journal · doi:10.1002/bimj.201400061 · Cited by 13 · see citing works
Shen, M., Xie, Q., Zhang, R., Yu, C., & Xiao, P. (2023) · Frontiers in Pharmacology · doi:10.3389/fphar.2023.1175021 · Cited by 11 · see citing works
Lee, H. G., Sholle, E., Beecy, A., Al’Aref, S., & Peng, Y. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.358 · Cited by 7 · see citing works
Segar, M. W., Lambeth, K. D., Postalian, A., Coulter, S., Gilge, J., Razvi, N., Saeed, M., Paisley, R. D., Pandey, A., & Razavi, M. (2026) · European Heart Journal - Digital Health · doi:10.1093/ehjdh/ztag018 · Cited by 2 · see citing works
- Authors
- Harrell, F. E.
- Corresponding author
- Harrell, Frank E.
- ORCID
- Harrell, Frank E.
- Venue
- JAMA: The Journal of the American Medical Association
- Publisher
- American Medical Association (AMA)
- Published
- 1982-05-14
- Volume
- 247
- Issue
- 18
- Pages
- 2543
- ISSN
- 0002-9955 (ISSN-L)
- 0098-7484
- OpenAlex
- W2084139018
- Cited by
- 3,490
- References
- 11
- Citations by year
- 2026: 249 · 2025: 276 · 2024: 313 · 2023: 298 · 2022: 301 · 2021: 295 · 2020: 237 · 2019: 168 · 2018: 121 · 2017: 130 · 2016: 124 · 2015: 118 · 2014: 118 · 2013: 95 · 2012: 98
- FWCI
- 4.01
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- JAMA (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Cardiac Imaging and Diagnostics
- Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
- Keywords
- prognostic value, treadmill exercise testing, coronary artery disease, clinical history and physical examination, diagnostic testing
- MeSH terms
- Catheterization, Diagnostic Services, Evaluation Studies as Topic, Exercise Test, Heart Diseases, Hospital Bed Capacity, 500 and over, Humans, Medical History Taking, North Carolina, Physical Examination
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
