Use of reclassification for assessment of improved prediction: an empirical evaluation
The DOI matches its Crossref record; 93 of the 93 references it cites were found in a registry or an index.
Abstract
BACKGROUND: An increasing number of studies evaluate the ability of predictors to change risk stratification and alter medical decisions, i.e. reclassification performance. We examined the reported design and analysis of recent studies of reclassification and the robustness of their claims for improved reclassification. METHODS: Two independent investigators searched PubMed and citations to the article that introduced the currently most popular reclassification metric (net reclassification index, NRI) to identify studies performing reclassification analysis (January 2006-January 2010). We focused on articles that included any analyses comparing the performance of a baseline predictive model vs the baseline model plus some additional predictor for a prospectively assessed outcome. We recorded information on the baseline model used, outcomes assessed, choice of risk thresholds and features of reclassification analyses. RESULTS: Of 58 baseline models used in 51 eligible papers, only 14 (24%) were previously described, used as described and had same outcomes as originally intended. Calibration was examined in 53% of the studies. Sixteen studies (31%) provided a reference for the choice of risk thresholds and only six used the previously proposed categories or justified the use of alternative thresholds. Only 14 studies (27%) stated that the chosen risk thresholds had different therapeutic intervention implications. NRI was calculated in 38 studies and was smaller in studies with adequately referenced or justified risk thresholds vs others (P < 0.0001). CONCLUSIONS: Reclassification studies would benefit from more rigorous methodological standards; otherwise claims for improved reclassification may remain spurious.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“While making revisions, conscious efforts were made to harmonize our recommendations with other reporting guidelines, and where possible we chose the same or similar wording for items (37,39,41,45,46).”
From Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): The TRIPOD Statement (2015) · cites this paper as [41]
In the citing paper
“the focus of the REMARK checklist is primarily on prognostic factors and not prediction models, whereas the GRIPS statement is aimed at risk prediction using genetic risk factors and the specific methodological issues around handling large numbers of genetic variants.”
From Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): The TRIPOD Statement (2015) · cites this paper as [41]
In the citing paper
“The reporting of studies evaluating the incremental value of specific predictors, beyond established predictors or even beyond existing prediction models [18,42], also fits entirely within the remit of TRIPOD (see the accompanying explanation and elaboration document [43]).”
From Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD Statement (2015) · cites this paper as [42]
In the citing paper
“An ever-increasing number of studies are evaluating the incremental value of specific predictors, beyond established predictors or even beyond existing prediction models [18,42].”
From Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD Statement (2015) · cites this paper as [42]
Citations
Cites
Grundy, S. M., Bilheimer, D. W., Chait, A., Clark, L. T., Denke, M. A., Havel, R. J., Hazzard, W. R., Hulley, S. B., Hunninghake, D. B., Kreisberg, R. A., et al. (1993) · JAMA · doi:10.1001/jama.269.23.3015 · Cited by 27,947
Expert Panel on Detection, E. (2001) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.285.19.2486 · Cited by 26,011
Chobanian, A. V. (2003) · JAMA · doi:10.1001/jama.289.19.2560 · Cited by 25,386
Detection, evaluation, and treatment of high blood cholesterol in adults (Adult Treatment Panel III)
Grundy, S. M., Becker, D. M., Clark, L. T., Cooper, R., Denke, M. A., Howard, J. L., Hunninghake, D. B., Illingworth, D. R., Luepker, R. V., McBride, P. E., et al. (2002) · Civil War Book Review · Cited by 12,198
Prediction of coronary heart disease using risk factor categories
Wilson, P. W. F., D’Agostino, R. B., Levy, D., Belanger, A. M., Silbershatz, H., & Kannel, W. B. (1998) · Circulation · doi:10.1161/01.cir.97.18.1837 · Cited by 9,581
Le Gall, J.-R. (1993) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.270.24.2957 · Cited by 7,773
Pencina, M. J., D' Agostino, R. B., D' Agostino, R. B., & Vasan, R. S. (2008) · Statistics in Medicine · doi:10.1002/sim.2929 · Cited by 6,493
Estimation of ten-year risk of fatal cardiovascular disease in Europe: the SCORE project
Conroy, R. (2003) · European Heart Journal · doi:10.1016/s0195-668x(03)00114-3 · Cited by 5,290
A Model to Predict Survival in Patients With End–Stage Liver Disease
Kamath, P. S., Wiesner, R. H., Malinchoc, M., Kremers, W., Therneau, T. M., Kosberg, C. L., D'Amico, G., Dickson, R. E., & Kim, R. W. (2001) · Hepatology · doi:10.1053/jhep.2001.22172 · Cited by 4,975
The APACHE III Prognostic System
Knaus, W. A., Wagner, D. P., Draper, E. A., Zimmerman, J. E., Bergner, M., Bastos, P. G., Sirio, C. A., Murphy, D. J., Lotring, T., Damiano, A., et al. (1991) · Chest · doi:10.1378/chest.100.6.1619 · Cited by 3,439
Gail, M. H., Brinton, L. A., Byar, D. P., Corle, D. K., Green, S. B., Schairer, C., & Mulvihill, J. J. (1989) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/81.24.1879 · Cited by 3,334
The APACHE III Prognostic System
Teres, D., & Lemeshow, S. (1992) · Chest · doi:10.1378/chest.102.6.1919b · Cited by 2,880
Quantitative Synthesis in Systematic Reviews
Lau, J., Ioannidis, J. P. A., & Schmid, C. H. (1997) · Annals of Internal Medicine · doi:10.7326/0003-4819-127-9-199711010-00008 · Cited by 2,517
Self-citationDiagnosis of osteoporosis and assessment of fracture risk
Kanis, J. A. (2002) · The Lancet · doi:10.1016/s0140-6736(02)08761-5 · Cited by 2,260
General and abdominal adiposity and risk of death in Europe
Pischon, T., Boeing, H., Hoffmann, K., Bergmann, M., Schulze, M. B., Overvad, K., van der Schouw, Y. T., Spencer, E., Moons, K. G. M., Tjønneland, A., et al. (2008) · New England Journal of Medicine · doi:10.1056/nejmoa0801891 · Cited by 2,155
Use and Misuse of the Receiver Operating Characteristic Curve in Risk Prediction
Cook, N. R. (2007) · Circulation · doi:10.1161/circulationaha.106.672402 · Cited by 2,112
Hyponatremia and Mortality among Patients on the Liver-Transplant Waiting List
Kim, W. R., Biggins, S. W., Kremers, W. K., Wiesner, R. H., Kamath, P. S., Benson, J. T., Edwards, E., & Therneau, T. M. (2008) · New England Journal of Medicine · doi:10.1056/nejmoa0801209 · Cited by 1,486
Prognosis and prognostic research: what, why, and how?
Moons, K. G. M., Royston, P., Vergouwe, Y., Grobbee, D. E., & Altman, D. G. (2009) · BMJ · doi:10.1136/bmj.b375 · Cited by 1,337
Prognosis and prognostic research: Developing a prognostic model
Royston, P., Moons, K. G. M., Altman, D. G., & Vergouwe, Y. (2009) · BMJ · doi:10.1136/bmj.b604 · Cited by 1,172
Schnabel, R. B., Sullivan, L. M., Levy, D., Pencina, M. J., Massaro, J. M., D'Agostino, R. B., Newton-Cheh, C., Yamamoto, J. F., Magnani, J. W., Tadros, T. M., et al. (2009) · The Lancet · doi:10.1016/s0140-6736(09)60443-8 · Cited by 1,041
Assessing Prostate Cancer Risk: Results from the Prostate Cancer Prevention Trial
Thompson, I. M., Ankerst, D. P., Chi, C., Goodman, P. J., Tangen, C. M., Lucia, M. S., Feng, Z., Parnes, H. L., & Coltman, C. A. (2006) · JNCI: Journal of the National Cancer Institute · doi:10.1093/jnci/djj131 · Cited by 938
Clinical risk factors, DNA variants, and the development of type 2 diabetes
Lyssenko, V., Jonsson, A., Almgren, P., Pulizzi, N., Isomaa, B., Tuomi, T., Berglund, G., Altshuler, D., Nilsson, P., & Groop, L. (2008) · New England Journal of Medicine · doi:10.1056/nejmoa0801869 · Cited by 938
Prognosis and prognostic research: application and impact of prognostic models in clinical practice
Moons, K. G. M., Altman, D. G., Vergouwe, Y., & Royston, P. (2009) · BMJ · doi:10.1136/bmj.b606 · Cited by 934
Use of Multiple Biomarkers to Improve the Prediction of Death from Cardiovascular Causes
Zethelius, B., Berglund, L., Sundström, J., Ingelsson, E., Basu, S., Larsson, A., Venge, P., & Ärnlöv, J. (2008) · New England Journal of Medicine · doi:10.1056/nejmoa0707064 · Cited by 889
Yeboah, J., Folsom, A. R., Burke, G. L., Johnson, C., Polak, J. F., Post, W., Lima, J. A., Crouse, J. R., & Herrington, D. M. (2009) · Circulation · doi:10.1161/circulationaha.109.864801 · Cited by 888
Predicting the 30-Year Risk of Cardiovascular Disease: The Framingham Heart Study
Pencina, M. J., D'Agostino, R. B., Larson, M. G., Massaro, J. M., & Vasan, R. S. (2009) · Circulation · doi:10.1161/circulationaha.108.816694 · Cited by 870
Ridker, P. M., Paynter, N. P., Rifai, N., Gaziano, J. M., & Cook, N. R. (2008) · Circulation · doi:10.1161/circulationaha.108.814251 · Cited by 841
Genotype score in addition to common risk factors for prediction of type 2 diabetes
Meigs, J. B., Shrader, P., Sullivan, L. M., McAteer, J. B., Fox, C. S., Dupuis, J., Manning, A. K., Florez, J. C., Wilson, P. W. F., D'Agostino, R. B., et al. (2008) · New England Journal of Medicine · doi:10.1056/nejmoa0804742 · Cited by 759
Polymorphisms Associated with Cholesterol and Risk of Cardiovascular Events
Kathiresan, S., Melander, O., Anevski, D., Guiducci, C., Burtt, N. P., Roos, C., Hirschhorn, J. N., Berglund, G., Hedblad, B., Groop, L., et al. (2008) · New England Journal of Medicine · doi:10.1056/nejmoa0706728 · Cited by 696
The APACHE III Prognostic System
Knaus, W. A. (1992) · Chest · doi:10.1016/s0012-3692(16)40910-4 · Cited by 626
Ingelsson, E., Schaefer, E. J., Contois, J. H., McNamara, J. R., Sullivan, L., Keyes, M. J., Pencina, M. J., Schoonmaker, C., Wilson, P. W. F., D’Agostino, R. B., et al. (2007) · JAMA · doi:10.1001/jama.298.7.776 · Cited by 608
Novel and Conventional Biomarkers for Prediction of Incident Cardiovascular Events in the Community
Melander, O. (2009) · JAMA · doi:10.1001/jama.2009.943 · Cited by 551
The effect of including C-reactive protein in cardiovascular risk prediction models for women
Cook, N. R., Buring, J. E., & Ridker, P. M. (2006) · Annals of Internal Medicine · doi:10.7326/0003-4819-145-1-200607040-00128 · Cited by 504
Mora, S., Otvos, J. D., Rifai, N., Rosenson, R. S., Buring, J. E., & Ridker, P. M. (2009) · Circulation · doi:10.1161/circulationaha.108.816181 · Cited by 490
Cook, N. R., & Ridker, P. M. (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-150-11-200906020-00007 · Cited by 458
Reasons or excuses for avoiding meta-analysis in forest plots
Ioannidis, J. P. A., Patsopoulos, N. A., & Rothstein, H. R. (2008) · BMJ · doi:10.1136/bmj.a117 · Cited by 457
Self-citationCounsell, C., Dennis, M., McDowall, M., & Warlow, C. (2002) · Stroke · doi:10.1161/hs0402.105909 · Cited by 319
Talmud, P. J., Hingorani, A. D., Cooper, J. A., Marmot, M. G., Brunner, E. J., Kumari, M., Kivimaki, M., & Humphries, S. E. (2010) · BMJ · doi:10.1136/bmj.b4838 · Cited by 296
Hallan, S. (2007) · Archives of Internal Medicine · doi:10.1001/archinte.167.22.2490 · Cited by 284
Whiteley, W., Jackson, C., Lewis, S., Lowe, G., Rumley, A., Sandercock, P., Wardlaw, J., Dennis, M., & Sudlow, C. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000145 · Cited by 279
Development of a nomogram for individualizing hip fracture risk in men and women
Nguyen, N. D., Frost, S. A., Center, J. R., Eisman, J. A., & Nguyen, T. V. (2007) · Osteoporosis International · doi:10.1007/s00198-007-0362-8 · Cited by 279
Assessment of Claims of Improved Prediction Beyond the Framingham Risk Score
Tzoulaki, I. (2009) · JAMA · doi:10.1001/jama.2009.1757 · Cited by 276
Self-citationPaynter, N. P., Chasman, D. I., Buring, J. E., Shiffman, D., Cook, N. R., & Ridker, P. M. (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-150-2-200901200-00003 · Cited by 262
Assessing the Value of Risk Predictions by Using Risk Stratification Tables
Janes, H., Pepe, M. S., & Gu, W. (2008) · Annals of Internal Medicine · doi:10.7326/0003-4819-149-10-200811180-00009 · Cited by 249
Shah, T., Casas, J. P., Cooper, J. A., Tzoulaki, I., Sofat, R., McCormack, V., Smeeth, L., Deanfield, J. E., Lowe, G. D., Rumley, A., et al. (2009) · International Journal of Epidemiology · doi:10.1093/ije/dyn217 · Cited by 238
Self-citationAlmost all articles on cancer prognostic markers report statistically significant results
Kyzas, P. A., Denaxa-Kyza, D., & Ioannidis, J. P. A. (2007) · European Journal of Cancer · doi:10.1016/j.ejca.2007.08.030 · Cited by 231
Self-citationSeveral methods to assess improvement in risk prediction models: Extension to survival analysis
Chambless, L. E., Cummiskey, C. P., & Cui, G. (2011) · Statistics in Medicine · doi:10.1002/sim.4026 · Cited by 194
Barr, E. L. M., Boyko, E. J., Zimmet, P. Z., Wolfe, R., Tonkin, A. M., & Shaw, J. E. (2009) · Diabetologia · doi:10.1007/s00125-008-1246-y · Cited by 167
Talmud, P. J., Cooper, J. A., Palmen, J., Lovering, R., Drenos, F., Hingorani, A. D., & Humphries, S. E. (2008) · Clinical Chemistry · doi:10.1373/clinchem.2007.095489 · Cited by 147
Rahman, M., Simmons, R. K., Harding, A.-H., Wareham, N. J., & Griffin, S. J. (2008) · Family Practice · doi:10.1093/fampra/cmn024 · Cited by 126
How much does HDL cholesterol add to risk estimation? A report from the SCORE Investigators
Cooney, M. T., Dudina, A., De Bacquer, D., Fitzgerald, A., Conroy, R., Sans, S., Menotti, A., De Backer, G., Jousilahti, P., Keil, U., et al. (2009) · European Journal of Cardiovascular Prevention & Rehabilitation · doi:10.1097/hjr.0b013e3283213140 · Cited by 121
Holme, I., Aastveit, A. H., Jungner, I., & Walldius, G. (2008) · Journal of Internal Medicine · doi:10.1111/j.1365-2796.2008.01925.x · Cited by 106
Improving Global Vascular Risk Prediction With Behavioral and Anthropometric Factors
Sacco, R. L., Khatri, M., Rundek, T., Xu, Q., Gardener, H., Boden-Albala, B., Di Tullio, M. R., Homma, S., Elkind, M. S. V., & Paik, M. C. (2009) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2009.07.047 · Cited by 105
Kyzas, P. A., Denaxa-Kyza, D., & Ioannidis, J. P. A. (2007) · JNCI: Journal of the National Cancer Institute · doi:10.1093/jnci/djk032 · Cited by 94
Self-citationPrognostic Value of Biomarkers During and After Non–ST-Segment Elevation Acute Coronary Syndrome
Eggers, K. M., Lagerqvist, B., Venge, P., Wallentin, L. C., & Lindahl, B. (2009) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2009.03.056 · Cited by 89
Is personalized medicine finally arriving?
Allison, M. (2008) · Nature Biotechnology · doi:10.1038/nbt0508-509 · Cited by 88
Reclassification Calculations for Persons With Incomplete Follow-up
Steyerberg, E. W., & Pencina, M. J. (2010) · Annals of Internal Medicine · doi:10.7326/0003-4819-152-3-201002020-00019 · Cited by 88
Evaluation of risk prediction updates from commercial genome-wide scans
Mihăescu, R., van Hoek, M., Sijbrands, E. J. G., Uitterlinden, A. G., Witteman, J. C. M., Albert Hofman, D., van Duijn, C. M., & Janssens, A. C. J. W. (2009) · Genetics in Medicine · doi:10.1097/gim.0b013e3181b13a4f · Cited by 84
Chien, K., Su, T., Jeng, J., Hsu, H., Chang, W., Chen, M.-F., Lee, Y., & Hu, F. B. (2008) · PLoS ONE · doi:10.1371/journal.pone.0003435 · Cited by 82
What Makes a Good Predictor?: The Evidence Applied to Coronary Artery Calcium Score
Ioannidis, J. P. A., & Tzoulaki, I. (2010) · JAMA · doi:10.1001/jama.2010.503 · Cited by 81
Self-citationSimmons, R. K. (2008) · Archives of Internal Medicine · doi:10.1001/archinte.168.11.1209 · Cited by 75
Lin, H.-J., Lee, B.-C., Ho, Y.-L., Lin, Y.-H., Chen, C.-Y., Hsu, H.-C., Lin, M.-S., Chien, K.-L., & Chen, M.-F. (2009) · Diabetes Care · doi:10.2337/dc08-2337 · Cited by 73
Anemia for Risk Assessment of Patients With Acute Coronary Syndromes
Meneveau, N. F., Schiele, F., Seronde, M., Descotes‐Genon, V., Oettinger, J., Chopard, R., Ecarnot, F., & Bassand, J. (2008) · The American Journal of Cardiology · doi:10.1016/j.amjcard.2008.10.023 · Cited by 69
Mihaescu, R., van Zitteren, M., van Hoek, M., Sijbrands, E. J. G., Uitterlinden, A. G., Witteman, J. C. M., Hofman, A., Hunink, M. G. M., van Duijn, C. M., & Janssens, A. C. J. W. (2010) · American Journal of Epidemiology · doi:10.1093/aje/kwq122 · Cited by 69
Hsich, E., Gorodeski, E. Z., Starling, R. C., Blackstone, E. H., Ishwaran, H., & Lauer, M. S. (2009) · Circulation · doi:10.1161/circulationaha.109.848382 · Cited by 69
Prediction of Incident Hypertension Risk in Women with Currently Normal Blood Pressure
Paynter, N. P., Cook, N. R., Everett, B. M., Sesso, H. D., Buring, J. E., & Ridker, P. M. (2009) · The American Journal of Medicine · doi:10.1016/j.amjmed.2008.10.034 · Cited by 68
Prognostic Value of Biomarkers in Heart Failure: Application of Novel Methods in the Community
Dunlay, S. M., Gerber, Y., Weston, S. A., Killian, J. M., Redfield, M. M., & Roger, V. L. (2009) · Circulation Heart Failure · doi:10.1161/circheartfailure.109.849299 · Cited by 68
Empana, J. P., Canouï‐Poitrine, F., Luc, G., Juhan-Vague, I., Morange, P., Arveiler, D., Ferrières, J., Amouyel, P., Bingham, A., Montaye, M., et al. (2008) · European Heart Journal · doi:10.1093/eurheartj/ehn331 · Cited by 64
Mann, D. M., Bertoni, A. G., Shimbo, D., Carnethon, M. R., Chen, H., Jenny, N. S., & Muntner, P. (2010) · American Journal of Epidemiology · doi:10.1093/aje/kwq030 · Cited by 58
Rana, J. S., Cote, M., Després, J.-P., Sandhu, M. S., Talmud, P. J., Ninio, E., Wareham, N. J., Kastelein, J. J. P., Zwinderman, A. H., Khaw, K.-T., et al. (2009) · Heart · doi:10.1136/hrt.2009.170134 · Cited by 58
Holme, I., Cater, N. B., Faergeman, O., Kastelein, J. J. P., Olsson, A. G., Tikkanen, M. J., Lytken Larsen, M., Lindahl, C., Pedersen, T. R., & Decrea, O. B. O. T. I. (2008) · Annals of Medicine · doi:10.1080/07853890801964955 · Cited by 53
Sun, Y., Chien, K., Hsu, H., Su, T., Chen, M.-F., & Lee, Y. (2009) · Circulation Journal · doi:10.1253/circj.cj-08-1077 · Cited by 51
The Potential of Genes and Other Markers to Inform about Risk
PEPE, M. S., Gu, J., & Morris, D. E. (2010) · Cancer Epidemiology Biomarkers & Prevention · doi:10.1158/1055-9965.epi-09-0510 · Cited by 50
Hamer, M., Chida, Y., & Stamatakis, E. A. (2009) · Clinical Chemistry · doi:10.1373/clinchem.2009.130740 · Cited by 47
Moayyeri, A., Kaptoge, S. K., Dalzell, N., Luben, R., Wareham, N. J., Bingham, S. A., REEVE, J., & Khaw, K. (2009) · Bone · doi:10.1016/j.bone.2009.05.001 · Cited by 35
Limits to forecasting in personalized medicine: An overview
Ioannidis, J. P. A. (2009) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2009.05.003 · Cited by 35
Self-citationCortigiani, L., Bombardini, T., Corbisiero, A. M., Mazzoni, A., Bovenzi, F., & Picano, E. (2009) · Heart · doi:10.1136/hrt.2008.161752 · Cited by 35
PEPE, M. S., Janes, H., & Gu, J. (2007) · Circulation · doi:10.1161/circulationaha.107.709253 · Cited by 30
Somsouk, M., Guy, J. E., Biggins, S. W., Vittinghoff, E., Kohn, M. A., & Inadomi, J. M. (2009) · Alimentary Pharmacology & Therapeutics · doi:10.1111/j.1365-2036.2009.04096.x · Cited by 29
Common genetic markers and prediction of recurrent events after ischemic stroke in young adults
Pezzini, A., Grassi, M., Zotto, E. D., Lodigiani, C., Ferrazzi, P., Spalloni, A., Patella, R., Giossi, A., Volonghi, I., Iacoviello, L., et al. (2009) · Neurology · doi:10.1212/wnl.0b013e3181b59aaf · Cited by 29
Woodward, M., Tunstall‐Pedoe, H., Rumley, A., & Lowe, G. D. O. (2009) · British Journal of Haematology · doi:10.1111/j.1365-2141.2009.07778.x · Cited by 26
Enhancement of Absolute Fracture Risk Prognosis with Genetic Marker: The Collagen I Alpha 1 Gene
Tran, B. N. H., Nguyen, N. D., Eisman, J. A., & Nguyen, T. V. (2009) · Calcified Tissue International · doi:10.1007/s00223-009-9296-9 · Cited by 26
Gorodeski, E. Z., Ishwaran, H., Blackstone, E. H., & Lauer, M. S. (2009) · American Heart Journal · doi:10.1016/j.ahj.2009.04.015 · Cited by 22
Robinson, D., Sandblom, G., Johansson, R., Garmo, H., Aus, G., Hedlund, P. O., & Varenhorst, E. (2008) · Urology · doi:10.1016/j.urology.2008.05.026 · Cited by 20
Correia, L. C. L., Rocha, M. D. S., Bittencourt, A. P. S. D. V., Freitas, R., Souza, A. C., de Almeida, M. C., & Esteves, J. P. (2009) · Clinica Chimica Acta · doi:10.1016/j.cca.2009.09.026 · Cited by 17
Impact of Gender on Risk Stratification by Stress Echocardiography
Cortigiani, L., Sicari, R., Bigi, R., Landi, P., Bovenzi, F., & Picano, E. (2009) · The American Journal of Medicine · doi:10.1016/j.amjmed.2008.11.003 · Cited by 16
Holme, I., Strandberg, T. E., Færgeman, O., Kastelein, J. J. P., OLSSON, A. G., Tikkanen, M. J., Larsen, M. L., Lindahl, C., & Pedersen, T. R. (2009) · Atherosclerosis · doi:10.1016/j.atherosclerosis.2009.01.023 · Cited by 15
Kizer, J. R., Krauser, D. G., Rodeheffer, R. J., Burnett, J. C. J., Okin, P. M., Roman, M. J., Umans, J. G., Best, L. G., Lee, E. T., & Devereux, R. B. (2009) · The American Journal of Cardiology · doi:10.1016/j.amjcard.2009.03.026 · Cited by 13
Prediction of Incident Diabetes Mellitus in Middle-aged Adults: The Framingham Offspring Study
Wilson, P. W. F. (2007) · Archives of Internal Medicine · doi:10.1001/archinte.167.10.1068 · Cited by 4
Are Risk Stratification Tables the Best Way to Evaluate Model Performance?
Stern, R. H., & Smith, D. E. (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-150-6-200903170-00021 · Cited by 1
The Predictive Value of the ELISpot-Based Interferon-γ–Release Assay for Tuberculosis Disease
Hesseling, A. C., Gie, R. P., & Mandalakas, A. M. (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-150-6-200903170-00023 · Cited by 1
Are Risk Stratification Tables the Best Way to Evaluate Model Performance?
Janes, H. E., Pepe, M. S., & Gu, W. (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-150-6-200903170-00022 · Cited by 1
Cited by
Moons, K. G. M., Altman, D. G., Reitsma, J. B., Ioannidis, J. P. A., Macaskill, P., Steyerberg, E. W., Vickers, A. J., Ransohoff, D. F., & Collins, G. S. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0698 · Cited by 5,147
Self-citationCollins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · BMJ · doi:10.1136/bmj.g7594 · Cited by 4,388
“The reporting of studies evaluating the incremental value of specific predictors, beyond established predictors or even beyond existing prediction models (18, 42), also fits entirely within the remit of TRIPOD (see the accompanying explanation and elaboration document [43], available at www .”
Collins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · British Journal of Cancer · doi:10.1038/bjc.2014.639 · Cited by 3,293
“An ever-increasing number of studies are evaluating the incremental value of specific predictors, beyond established predictors or even beyond existing prediction models (Tzoulaki et al, 2011; Steyerberg et al, 2012).”
InfluentialCollins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0697 · Cited by 3,267
“The reporting of studies evaluating the incremental value of specific predictors, beyond established predictors or even beyond existing prediction models [18,42]; also fits entirely within the remit of TRIPOD (see the accompanying explanation and elaboration document [43]; available at www.”
BackgroundCollins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. (2015) · BMC Medicine · doi:10.1186/s12916-014-0241-z · Cited by 1,927
“The reporting of studies evaluating the incremental value of specific predictors, beyond established predictors or even beyond existing prediction models [18,42], also fits entirely within the remit of TRIPOD (see the accompanying”
Metadata
- Authors
- Tzoulaki, I., Liberopoulos, G., & Ioannidis, J. P. A.
- Affiliations
- Imperial College London (GB)
- Tufts University (US)
- Corresponding author
- Ioannidis, John P A
- Venue
- International Journal of Epidemiology
- Publisher
- Oxford University Press (OUP)
- Published
- 2011-02-16
- Volume
- 40
- Issue
- 4
- Pages
- 1094-1105
- ISSN
- 0300-5771 (ISSN-L)
- 1464-3685
- OpenAlex
- W2106597074
- Cited by
- 70
- References
- 93
- Citations by year
- 2026: 2 · 2025: 3 · 2024: 2 · 2023: 3 · 2022: 7 · 2020: 1 · 2018: 4 · 2017: 2 · 2016: 2 · 2015: 16 · 2014: 5 · 2013: 9 · 2012: 11
- FWCI
- 7.86
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- International Journal of Epidemiology (publishedVersion)
- PubMed (publishedVersion)
- HighWire Press Open Archive (submittedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Keywords
- reclassification, net reclassification improvement, risk stratification, risk thresholds, calibration, risk prediction model, methodological quality, prediction model evaluation
- MeSH terms
- Chronic Disease, Humans, Risk Factors, Models, Statistical, Treatment Outcome, Risk Assessment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.