Cited by 195Statistical Methods in EpidemiologyMeta-analysis and systematic reviewsInflammatory Biomarkers in Disease Prognosis

The DOI matches its Crossref record; 113 of the 113 references it cites were found in a registry or an index.

Reporting performance of prognostic models in cancer: a review.

Open access

Mallett, Susan, Royston, Patrick, Waters, Rachel, Dutton, Susan, Altman, Douglas G·2010·BMC Medicine

DOI 10.1186/1741-7015-8-21PMC PMC2857810

Open this paper in CoChat — free account Open PDF

Abstract

BACKGROUND: Appropriate choice and use of prognostic models in clinical practice require the use of good methods for both model development, and for developing prognostic indices and risk groups from the models. In order to assess reliability and generalizability for use, models need to have been validated and measures of model performance reported. We reviewed published articles to assess the methods and reporting used to develop and evaluate performance of prognostic indices and risk groups from prognostic models. METHODS: We developed a systematic search string and identified articles from PubMed. Forty-seven articles were included that satisfied the following inclusion criteria: published in 2005; aiming to predict patient outcome; presenting new prognostic models in cancer with outcome time to an event and including a combination of at least two separate variables; and analysing data using multivariable analysis suitable for time to event data. RESULTS: In 47 studies, Cox models were used in 94% (44), but the coefficients or hazard ratios for the variables in the final model were reported in only 72% (34). The reproducibility of the derived model was assessed in only 11% (5) of the articles. A prognostic index was developed from the model in 81% (38) of the articles, but researchers derived the prognostic index from the final prognostic model in only 34% (13) of the studies; different coefficients or variables from those in the final model were used in 50% (19) of models and the methods used were unclear in 16% (6) of the articles. Methods used to derive prognostic groups were also poor, with researchers not reporting the methods used in 39% (14 of 36) of the studies and data derived methods likely to bias estimates of differences between risk groups being used in 28% (10) of the studies. Validation of their models was reported in only 34% (16) of the studies. In 15 studies validation used data from the same population and in five studies from a different population. Including reports of validation with external data from publications up to four years following model development, external validation was attempted for only 21% (10) of models. Insufficient information was provided on the performance of models in terms of discrimination and calibration. CONCLUSIONS: Many published prognostic models have been developed using poor methods and many with poor reporting, both of which compromise the reliability and clinical relevance of models, prognostic indices and risk groups derived from them.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1186/1741-7015-8-21

Cites

113 of 113 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Regression Modeling Strategies
Harrell, F. E. (2001) · Springer series in statistics · doi:10.1007/978-1-4757-3462-1 · Cited by 9,270
Evaluating the added predictive ability of a new marker: From area under the ROC curve to reclassification and beyond
Pencina, M. J., D' Agostino, R. B., D' Agostino, R. B., & Vasan, R. S. (2008) · Statistics in Medicine · doi:10.1002/sim.2929 · Cited by 6,493
Decision curve analysis: a novel method for evaluating prediction models
Vickers, A. J., & Elkin, E. B. (2006) · Medical Decision Making · doi:10.1177/0272989x06295361 · Cited by 6,341
Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating
Steyerberg, E. W. (2019) · Statistics for Biology and Health · doi:10.1007/978-3-030-16399-0 · Cited by 3,281
Dichotomizing continuous predictors in multiple regression: a bad idea
Royston, P., Altman, D. G., & Sauerbrei, W. (2006) · Statistics in Medicine · doi:10.1002/sim.2331 · Cited by 2,312
Hepatogastroenterology
Xu, X., Yuan, G., Li, Q., Shan, S.-L., & Jiang, S. (2012) · Hepatogastroenterology · doi:10.5754/hge12404 · Cited by 1,825
Clinical Prediction Models
Steyerberg, E. W. (2009) · Statistics for Biology and Health · doi:10.1007/978-0-387-77244-8 · Cited by 1,491
Prognosis and prognostic research: validating a prognostic model
Altman, D. G., Vergouwe, Y., Royston, P., & Moons, K. G. M. (2009) · BMJ · doi:10.1136/bmj.b605 · Cited by 1,436
Prognosis and prognostic research: what, why, and how?
Moons, K. G. M., Royston, P., Vergouwe, Y., Grobbee, D. E., & Altman, D. G. (2009) · BMJ · doi:10.1136/bmj.b375 · Cited by 1,337
Comments on ‘Evaluating the added predictive ability of a new marker: From area under the ROC curve to reclassification and beyond’ by M. J. Pencina, R. B. D'Agostino Sr, R. B. D'Agostino Jr, R. S. Vasan, Statistics in Medicine (DOI: 10.1002/sim.2929)
Greenland, P. (2007) · Statistics in Medicine · doi:10.1002/sim.2976 · Cited by 1,300
Assessing the Generalizability of Prognostic Information
Justice, A. C., Covinsky, K. E., & Berlin, J. A. (1999) · Annals of Internal Medicine · doi:10.7326/0003-4819-130-6-199903160-00016 · Cited by 1,160
Dangers of Using "Optimal" Cutpoints in the Evaluation of Prognostic Factors
Altman, D. G., Lausen, B., Sauerbrei, W., & Schumacher, M. (1994) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/86.11.829 · Cited by 1,143
Multivariable Model‐Building
Royston, P. J., & Sauerbrei, W. (2008) · Wiley series in probability and statistics · doi:10.1002/9780470770771 · Cited by 1,002
Prognosis and prognostic research: application and impact of prognostic models in clinical practice
Moons, K. G. M., Altman, D. G., Vergouwe, Y., & Royston, P. (2009) · BMJ · doi:10.1136/bmj.b606 · Cited by 934
Inappropriate use of bivariable analysis to screen risk factors for use in multivariable analysis
Sun, G.-W., Shook, T. L., & Kay, G. L. (1996) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(96)00025-x · Cited by 933
Combining estimates of interest in prognostic modelling studies after multiple imputation: current practice and guidelines
Marshall, A., Altman, D. G., HOLDER, R. L., & Royston, P. J. (2009) · BMC Medical Research Methodology · doi:10.1186/1471-2288-9-57 · Cited by 883
THE UNIVERSITY OF CALIFORNIA, SAN FRANCISCO CANCER OF THE PROSTATE RISK ASSESSMENT SCORE: A STRAIGHTFORWARD AND RELIABLE PREOPERATIVE PREDICTOR OF DISEASE RECURRENCE AFTER RADICAL PROSTATECTOMY
Cooperberg, M. R., Pasta, D. J., Elkin, E. P., Litwin, M. S., Latini, D. M., CHANE, J. D., & Carroll, P. R. (2005) · The Journal of Urology · doi:10.1097/01.ju.0000158155.33890.e7 · Cited by 774
Postoperative Nomogram Predicting the 10-Year Probability of Prostate Cancer Recurrence After Radical Prostatectomy
Stephenson, A. J., Scardino, P. T., Eastham, J. A., Bianco, F. J., Dotan, Z., DiBlasio, C. J., Reuther, A. M., Klein, E. A., & Kattan, M. W. (2005) · Journal of Clinical Oncology · doi:10.1200/jco.2005.01.867 · Cited by 663
Logistic regression in the medical literature:
Bagley, S. C., White, H., & Golomb, B. A. (2001) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00372-9 · Cited by 616 · cited 2×
“Other research has found that a large range, 0% to 52%, of articles where a prediction model was developed, included either internal or external validation [93,95-98].”
BackgroundMethod
Internal and external validation of predictive models: A simulation study of bias and precision in small samples
Steyerberg, E. W., Bleeker, S. E., Moll, H. A., Grobbee, D. E., & Moons, K. G. M. (2003) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(03)00047-7 · Cited by 611
Multivariable Model-Building: A Pragmatic Approach to Regression Analysis based on Fractional Polynomials for Modelling Continuous Variables
Royston, P. J., & Sauerbrei, W. (2008) · Cited by 601
Validation, updating and impact of clinical prediction rules: A review
Toll, D. B., Janssen, K. J. M., Vergouwe, Y., & Moons, K. G. M. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.04.008 · Cited by 592
Validation and Extension of the Memorial Sloan-Kettering Prognostic Factors Model for Survival in Patients With Previously Untreated Metastatic Renal Cell Carcinoma
Mekhail, T. M., Abou-Jawde, R. M., BouMerhi, G., Malhi, S., Wood, L. S., Elson, P. J., & Bukowski, R. M. (2005) · Journal of Clinical Oncology · doi:10.1200/jco.2005.05.179 · Cited by 558
A POSTOPERATIVE PROGNOSTIC NOMOGRAM PREDICTING RECURRENCE FOR PATIENTS WITH CONVENTIONAL CLEAR CELL RENAL CELL CARCINOMA
Sorbellini, M., Kattan, M. W., Snyder, M. E., Reuter, V. E., Motzer, R. J., Goetzl, M. A., McKIERNAN, J., & Russo, P. (2005) · The Journal of Urology · doi:10.1097/01.ju.0000148261.19532.2c · Cited by 527
Prognostic Index for Adult Patients With Acute Myeloid Leukemia in First Relapse
Breems, D. A., van Putten, W. L. J., Huijgens, P. C., Ossenkoppele, G. J., Verhoef, G. E. G., Verdonck, L. F., Vellenga, E., de Greef, G. E., Jacky, E., der Lelie, J. V., et al. (2005) · Journal of Clinical Oncology · doi:10.1200/jco.2005.06.027 · Cited by 510
A new measure of prognostic separation in survival data
Royston, P., & Sauerbrei, W. (2004) · Statistics in Medicine · doi:10.1002/sim.1621 · Cited by 450 · cited 1×
“Several measures of discrimination have been developed including the R squared [3], D statistic [67], c-index [4], SEP and PSEP [68,69], K [70], NRI [6], IDI [6] and decision curve analysis [71].”
Method
Outcome After Pathologic Complete Eradication of Cytologically Proven Breast Cancer Axillary Node Metastases Following Primary Chemotherapy
Hennessy, B. T., Hortobágyi, G. N., Rouzier, R., Kuerer, H. M., Sneige, N., Buzdar, A. U., Kau, S., Fornage, B. D., Şahin, A. A., Broglio, K. R., et al. (2005) · Journal of Clinical Oncology · doi:10.1200/jco.2005.02.5023 · Cited by 420
Prediction of survival in patients with metastases in the spinal column
van der Linden, Y. M., Dijkstra, P. D. S., Vonk, E. J. A., Marijnen, C. A. M., Leer, J. W. H., & for The Dutch Bone Metastasis Study Group (2004) · Cancer · doi:10.1002/cncr.20756 · Cited by 396
Elevated neutrophil and monocyte counts in peripheral blood are associated with poor survival in patients with metastatic melanoma: a prognostic model
Schmidt, H., Bastholt, L., Geertsen, P. F., Christensen, I. J., Larsen, S., Gehl, J., & von der Maase, H. (2005) · British Journal of Cancer · doi:10.1038/sj.bjc.6602702 · Cited by 327
Bootstrap investigation of the stability of a cox regression model
Altman, D. G., & Andersen, P. K. (1989) · Statistics in Medicine · doi:10.1002/sim.4780080702 · Cited by 311
Predictive value of seven preoperative prognostic scoring systems for spinal metastases
Leithner, A., Radl, R., Gruber, G., Hochegger, M., Leithner, K., Welkerling, H., Rehak, P., & Windhager, R. H. (2008) · European Spine Journal · doi:10.1007/s00586-008-0763-1 · Cited by 304
Development and validation of a prediction model with missing predictor data: a practical approach
Vergouwe, Y., Royston, P. J., Moons, K. G. M., & Altman, D. G. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.03.017 · Cited by 288
Prognostic factors and a scoring system for patients with skeletal metastasis
Katagiri, H., Takahashi, M., Wakai, K., Sugiura, H., Kataoka, T., & Nakanishi, K. (2005) · Journal of Bone and Joint Surgery - British Volume · doi:10.1302/0301-620x.87b5.15185 · Cited by 278
Proposal of a new prognostic model for hepatocellular carcinoma: an analysis of 403 patients
Tateishi, R., Yoshida, H., Shiina, S., Imamura, H., Hasegawa, K., Teratani, T., Obi, S., Sato, S., Koike, Y., Fujishima, T., et al. (2005) · Gut · doi:10.1136/gut.2003.035055 · Cited by 273
Breast carcinoma malignancy grading by Bloom–Richardson system vs proliferation index: reproducibility of grade and advantages of proliferation index
Meyer, J. S., Alvarez, C., Milikowski, C., Olson, N., Russo, I. H., Russo, J., Glass, A. G., Zehnbauer, B. A., Lister, K., & Parwaresch, R. M. (2005) · Modern Pathology · doi:10.1038/modpathol.3800388 · Cited by 264
Clinical Model to Predict Survival in Chemonaive Patients With Advanced Non–Small-Cell Lung Cancer Treated With Third-Generation Chemotherapy Regimens Based on Eastern Cooperative Oncology Group Data
HOANG, T. N., Xu, R., Schiller, J. H., Bonomi, P., & Johnson, D. H. (2004) · Journal of Clinical Oncology · doi:10.1200/jco.2005.04.177 · Cited by 262
USING TUMOR MARKERS TO PREDICT THE SURVIVAL OF PATIENTS WITH METASTATIC RENAL CELL CARCINOMA
Kim, H. L., Seligson, D. B., Liu, X., Janzen, N. K., BUI, M. H. T., Yu, H., Shi, T., Belldegrun, A. S., Horvath, S., & Figlin, R. A. (2005) · The Journal of Urology · doi:10.1097/01.ju.0000154351.37249.f0 · Cited by 216
A SCORING ALGORITHM TO PREDICT SURVIVAL FOR PATIENTS WITH METASTATIC CLEAR CELL RENAL CELL CARCINOMA: A STRATIFICATION TOOL FOR PROSPECTIVE CLINICAL TRIALS
Leibovich, B. C., Cheville, J. C., Lohse, C. M., Zincke, H., Frank, I., Kwon, E. D., Merchan, J. R., & Blute, M. L. (2005) · The Journal of Urology · doi:10.1097/01.ju.0000177487.64651.3a · Cited by 216
Survival in Patients Operated on for Pathologic Fracture: Implications for End-of-Life Orthopedic Care
Nathan, S. S., Healey, J. H., Mellano, D., Hoang, B. H., Lewis, I., Morris, C. D., Athanasian, E. A., & Boland, P. J. (2005) · Journal of Clinical Oncology · doi:10.1200/jco.2005.08.104 · Cited by 213
A new simplified comorbidity score as a prognostic factor in non-small-cell lung cancer patients: description and comparison with the Charlson's index
for the oncoLR health network, Colinet, B., Jacot, W., Bertrand, D., Lacombe, S., Bozonnat, M., Daurès, J., & Pujol, J.-L. (2005) · British Journal of Cancer · doi:10.1038/sj.bjc.6602836 · Cited by 211
Validation and adaptation of a nomogram for predicting the survival of patients with extremity soft tissue sarcoma using a three‐grade system
Mariani, L., Miceli, R., Kattan, M. W., Brennan, M. F., Colecchia, M., Fiore, M., Casali, P. G., & Gronchi, A. (2004) · Cancer · doi:10.1002/cncr.20778 · Cited by 211
Reporting methods in studies developing prognostic models in cancer: a review
Mallett, S., Royston, P., Dutton, S., Waters, R., & Altman, D. G. (2010) · BMC Medicine · doi:10.1186/1741-7015-8-20 · Cited by 206 · cited 4×
“Details of the literature search, inclusion criteria and methods in data extraction are reported in the companion paper (Mallett et al [7]).”
MethodInfluentialSelf-citation
An independent external validation and evaluation of QRISK cardiovascular risk prediction: a prospective open cohort study
Collins, P. G. S., & Altman, D. G. (2009) · BMJ · doi:10.1136/bmj.b2584 · Cited by 192
A systematic review finds methodological improvements necessary for prognostic models in determining traumatic brain injury outcomes
Mushkudiani, N. A., Hukkelhoven, C. W. P. M., Hernández, A. V., Murray, G. D., Choi, S. C., Maas, A. I. R., & Steyerberg, E. W. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.06.011 · Cited by 177
Prediction models in reproductive medicine: a critical appraisal†
Leushuis, E., van der Steeg, J. W., Steures, P., Bossuyt, P. M., Eijkemans, M. J. C., van der Veen, F., Mol, B. W., & Hompes, P. G. A. (2009) · Human Reproduction Update · doi:10.1093/humupd/dmp013 · Cited by 173
Risk factors for subsequent development of bladder cancer after primary transitional cell carcinoma of the upper urinary tract
Matsui, Y., Utsunomiya, N., Ichioka, K., Ueda, N., Yoshimura, K., Terai, A., & Arai, Y. (2005) · Urology · doi:10.1016/j.urology.2004.09.021 · Cited by 161
Dealing with Missing Predictor Values When Applying Clinical Prediction Models
Janssen, K. J. M., Vergouwe, Y., Donders, A. R. T., Harrell, F. E., Chen, Q., Grobbee, D. E., & Moons, K. G. M. (2009) · Clinical Chemistry · doi:10.1373/clinchem.2008.115345 · Cited by 152
Assessing New Biomarkers and Predictive Models for Use in Clinical Practice
McGeechan, K., Macaskill, P., Irwig, L., Liew, G., & Wong, T. Y. (2008) · Archives of Internal Medicine · doi:10.1001/archinte.168.21.2304 · Cited by 132
Prognostic Models: A Methodological Framework and Review of Models for Breast Cancer
Altman, D. G. (2009) · Cancer Investigation · doi:10.1080/07357900802572110 · Cited by 129
Risk‐group definition by recursive partitioning analysis of patients with squamous cell head and neck carcinoma treated with surgery and postoperative radiotherapy
Langendijk, J. A., Slotman, B. J., van der Waal, I., Doornaert, P., Berkof, J., & Leemans, C. R. (2005) · Cancer · doi:10.1002/cncr.21340 · Cited by 128
Surgical management of spinal metastases: analysis of prognostic factors during a 10-year experience
North, R. B., LaRocca, V. R., Schwartz, J., North, C. A., ZAHURAK, M. L., Davis, R. F., & McAfee, P. C. (2005) · Journal of Neurosurgery Spine · doi:10.3171/spi.2005.2.5.0564 · Cited by 128
A Prognostic Index for Systemic AIDS-Related Non-Hodgkin Lymphoma Treated in the Era of Highly Active Antiretroviral Therapy
Bower, M., Gazzard, B., Mandalia, S., Newsom‐Davis, T., Thirlwell, C., Dhillon, T., Young, A. M., Powles, T., Gaya, A. M., Nelson, M. R., et al. (2005) · Annals of Internal Medicine · doi:10.7326/0003-4819-143-4-200508160-00007 · Cited by 123
A review of two journals found that articles using multivariable logistic regression frequently did not report commonly recommended assumptions
Ottenbacher, K. J., Ottenbacher, H. R., Tooth, L. R., & Ostir, G. V. (2004) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2003.05.003 · Cited by 121
Quality of life and comorbidity score as prognostic determinants in non-small-cell lung cancer patients
Jacot, W., Colinet, B., Bertrand, D., Lacombe, S., Bozonnat, M. C., Daurès, J., & Pujol, J.-L. (2008) · Annals of Oncology · doi:10.1093/annonc/mdn064 · Cited by 105
Patient subsets with T1-T2, node-negative breast cancer at high locoregional recurrence risk after mastectomy
Truong, P. T., Lesperance, M., Culhaci, A., Kader, H. A., Speers, C. H., & Olivotto, I. A. (2005) · International Journal of Radiation Oncology*Biology*Physics · doi:10.1016/j.ijrobp.2004.09.013 · Cited by 102
Comments on ‘Evaluating the added predictive ability of a new marker: From area under the ROC curve to reclassification and beyond’ by M. J. Pencina et al., Statistics in Medicine (DOI: 10.1002/sim.2929)
Cook, N. R. (2007) · Statistics in Medicine · doi:10.1002/sim.2987 · Cited by 102
Post hoc choice of cut points introduced bias to diagnostic research
Ewald, B. D. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2005.11.025 · Cited by 94
Time to Detectable Metastatic Disease in Patients with Rising Prostate-Specific Antigen Values following Surgery or Radiation Therapy
Slovin, S. F., Wilton, A. S., Heller, G., & Scher, H. I. (2005) · Clinical Cancer Research · doi:10.1158/1078-0432.ccr-05-1668 · Cited by 92
Stent placement or brachytherapy for palliation of dysphagia from esophageal cancer: a prognostic model to guide treatment selection
Steyerberg, E. W., Homs, M. Y. V., Stokvis, A., Essink‐Bot, M., & Siersema, P. D. (2005) · Gastrointestinal Endoscopy · doi:10.1016/s0016-5107(05)01587-7 · Cited by 90
LYMPHOVASCULAR INVASION INDEPENDENTLY PREDICTS INCREASED DISEASE SPECIFIC SURVIVAL IN PATIENTS WITH TRANSITIONAL CELL CARCINOMA OF THE UPPER URINARY TRACT
Kikuchi, E., Horiguchi, Y., Nakashima, J., HATAKEYAMA, N., Matsumoto, M., Nishiyama, T., & Murai, M. (2005) · The Journal of Urology · doi:10.1097/01.ju.0000181801.22474.8b · Cited by 84
A new system for substaging pT1 papillary bladder cancer: a prognostic evaluation
VANDERAA, M., VANLEENDERS, G., Steyerberg, E. W., VANRHIJN, B., Jöbsis, A. C., Zwarthoff, E. C., & van der Kwast, T. (2005) · Human Pathology · doi:10.1016/j.humpath.2005.06.017 · Cited by 79
Applicability of staging systems for patients with hepatocellular carcinoma is dependent on treatment method – Analysis of 2010 Taiwanese patients
Chen, C., Hu, F.-C., Huang, G., Lee, P., Tsang, Y.-M., Cheng, A., Chen, D., Wang, J., & Sheu, J. (2009) · European Journal of Cancer · doi:10.1016/j.ejca.2008.12.025 · Cited by 74
Validity of the CAPRA Score to Predict Biochemical Recurrence-Free Survival After Radical Prostatectomy. Results From a European Multicenter Survey of 1,296 Patients
May, M., Knoll, N., Siegsmund, M. J., Fahlenkamp, D., Vogler, H., Hoschke, B., & Gralla, O. (2007) · The Journal of Urology · doi:10.1016/j.juro.2007.07.043 · Cited by 73
Prognostic factors and prediction of prognosis by the CTCL Severity Index in mycosis fungoides and Sezary syndrome
Klemke, C., Mansmann, U., Poenitz, N., Dippel, E., & Goerdt, S. (2005) · British Journal of Dermatology · doi:10.1111/j.1365-2133.2005.06676.x · Cited by 66
The Cutaneous B-Cell Lymphoma Prognostic Index: A Novel Prognostic Index Derived From a Population-Based Registry
Smith, B. D., Smith, G. L., COOPER, D. L., & Wilson, L. D. (2005) · Journal of Clinical Oncology · doi:10.1200/jco.2005.08.137 · Cited by 64
Prognosis research: Why is Dr. Lydgate still waiting?
Hemingway, H. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.02.005 · Cited by 62
Improvement of breast cancer relapse prediction in high risk intervals using artificial neural networks
Jerez, J. M., Franco, L., Alba, E., Llombart‐Cussac, A., Lluch, A. M., Ribelles, N., Munárriz, B., & Martín, M. (2005) · Breast Cancer Research and Treatment · doi:10.1007/s10549-005-9013-y · Cited by 62
Lymphopenia: A new independent prognostic factor for survival in patients treated with whole brain radiotherapy for brain metastases from breast carcinoma
Claude, L., Pérol, D., Ray-Coquard, I. L., Petit, T., Blay, J., Carrié, C., & Bachelot, T. (2005) · Radiotherapy and Oncology · doi:10.1016/j.radonc.2005.06.004 · Cited by 61
External Validation of University of California, San Francisco, Cancer of the Prostate Risk Assessment Score
Zhao, K., Hernandez, D. J., Han, M., Humphreys, E. B., Mangold, L. A., & Partin, A. W. (2008) · Urology · doi:10.1016/j.urology.2007.11.165 · Cited by 59
Comparison of three current staging systems for hepatocellular carcinoma: Japan integrated staging score, new Barcelona Clinic Liver Cancer staging classification, and Tokyo score
Chung, H., Kudo, M., Takahashi, S., Hagiwara, S., Sakaguchi, Y., Inoue, T., Minami, Y., Ueshima, K., Fukunaga, T., & Matsunaga, T. (2007) · Journal of Gastroenterology and Hepatology · doi:10.1111/j.1440-1746.2007.05075.x · Cited by 50
Linear discriminant models for unbalanced longitudinal data
Marshall, G., & Bar�n, A. E. (2000) · Statistics in Medicine · doi:10.1002/1097-0258(20000815)19:15 · Cited by 50
A model for predicting outcomes in patients with differentiated thyroid cancer and model performance in comparison with other classification systems
Yıldırım, E. (2005) · Journal of the American College of Surgeons · doi:10.1016/j.jamcollsurg.2004.10.031 · Cited by 49
Prognostic factors for survival in patients with advanced renal cell carcinoma undergoing nonmyeloablative allogeneic stem cell transplantation
Peccatori, J., Barkholt, L., Demirer, T., Sormani, M. P., Bruzzi, P., Ciceri, F., Zambelli, A., Prada, G. A. D., Pedrazzoli, P., Siena, S., et al. (2005) · Cancer · doi:10.1002/cncr.21477 · Cited by 48
Evaluation of performance
Steyerberg, E. W. (2008) · Statistics in the health sciences · doi:10.1007/978-0-387-77244-8_15 · Cited by 45
Staging systems for follicular thyroid carcinoma: application to 171 consecutive patients treated in a tertiary referral centre
Lang, B. H., Lo, C., Chan, W.-F., Lam, K.-Y., & Wan, K. Y. (2007) · Endocrine Related Cancer · doi:10.1677/erc.1.01284 · Cited by 45
Validation of Prediction Models
Steyerberg, E. W. (2008) · Statistics in the health sciences · doi:10.1007/978-0-387-77244-8_17 · Cited by 42
The use of prednisolone in the treatment of disseminated breast carcinoma
Kofman, S., Nagamani, D., Buenger, R. E., & Taylor, S. G. (1958) · Cancer · doi:10.1002/1097-0142(195801/02)11:1 · Cited by 41
Evaluation of Logistic Regression Reporting in Current Obstetrics and Gynecology Literature
Mikolajczyk, R., DiSilvesto, A., & Zhang, J. (2008) · Obstetrics and Gynecology · doi:10.1097/aog.0b013e318160f38e · Cited by 36
A gender-based score system predicts the clinical outcome of patients with early B-cell chronic lymphocytic leukemia
Molica, S., Mauro, F. R., Callea, V., Gentile, M., Giannarelli, D., Lopez, M., Lauria, F., Rotoli, B., Montanaro, M., Cortelezzi, A., et al. (2005) · Leukemia & lymphoma/Leukemia and lymphoma · doi:10.1080/10428190400029965 · Cited by 27
Validation of the European Prognostic Index for younger adult patients with acute myeloid leukaemia in first relapse
Giles, F. J., Verstovšek, S., Garcia‐Manero, G., Thomas, D. A., Ravandi, F., Wierda, W. G., Ferrajoli, A., Kornblau, S. M., Jabbour, E. J., Shan, J., et al. (2006) · British Journal of Haematology · doi:10.1111/j.1365-2141.2006.06106.x · Cited by 27
A prognostic model for prolonged event-free survival after autologous or allogeneic blood or marrow transplantation for relapsed and refractory Hodgkin's disease
Hahn, T., Beneklı, M., Wong, C., Moysich, K. B., Hyland, A. J., Michalek, A. M., Alam, A. R., Baer, M. R., Bambach, B., Czuczman, M. S., et al. (2005) · Bone Marrow Transplantation · doi:10.1038/sj.bmt.1704789 · Cited by 27
Prognostic factors in patients with advanced biliary tract cancer receiving chemotherapy.
Saisho, T., Okusaka, T., Ueno, H., Morizane, C., & Okada, S. (2005) · PubMed · Cited by 26
Do nomograms predict aggressive recurrence after radical prostatectomy more accurately than biochemical recurrence alone?
Schroeck, F. R., Aronson, W. J., Presti, J. C., Terris, M. K., Kane, C. J., Amling, C. L., & Freedland, S. J. (2008) · British Journal of Urology · doi:10.1111/j.1464-410x.2008.08118.x · Cited by 21
Selecting a Short-term Prognostic Model for Hepatocellular Carcinoma
Huo, T., Hsia, C., Huang, Y., Lin, H., Lee, P.-C., Lui, W., Chiang, J., Chiou, Y., Loong, C., & Lee, S. (2009) · Journal of Clinical Gastroenterology · doi:10.1097/mcg.0b013e31818dd962 · Cited by 21
Multicenter Validation of Recursive Partitioning Analysis Classification for Patients With Squamous Cell Head and Neck Carcinoma Treated With Surgery and Postoperative Radiotherapy
Jonkman, A., Kaanders, J. H. A. M., Terhaard, C. H. J., Hoebers, F. J. P., van den Ende, P. L. A., Wijers, O. B., Verhoef, L. C. G., de Jong, M. A., Leemans, C. R., & Langendijk, J. A. (2007) · International Journal of Radiation Oncology*Biology*Physics · doi:10.1016/j.ijrobp.2006.12.002 · Cited by 21
Prostate cancer risk assessment
Cooperberg, M. R. (2008) · Cancer · doi:10.1002/cncr.23920 · Cited by 20
A prognostic model for prolonged event-free survival after autologous or allogeneic blood or marrow transplantation for relapsed and refractory Hodgkin's disease
Tey, S., Butler, J., Durrant, S., Hill, G. R., Morton, J., & Kennedy, G. (2005) · Bone Marrow Transplantation · doi:10.1038/sj.bmt.1705044 · Cited by 20
A prognostic index (bladder prognostic index) for bilharzial-related invasive bladder cancer
Khaled, H. M., Hattab, O. E., Moneim, D. A., Kassem, H. A., Morsi, A., Sherif, G. M., Darwish, T., & Gaafar, R. M. (2005) · Urologic Oncology Seminars and Original Investigations · doi:10.1016/j.urolonc.2005.04.005 · Cited by 20
RISK GROUPS IN PATIENTS WITH BLADDER CANCER TREATED WITH RADICAL CYSTECTOMY: STATISTICAL AND CLINICAL MODEL IMPROVING HOMOGENEITY
Solsona, E., Iborra, I., Dumont, R., Rubio, J., Casanova, J. L., & Almenar, S. M. (2005) · The Journal of Urology · doi:10.1097/01.ju.0000173920.31824.2f · Cited by 19
Successful validation of a survival prediction model in patients with metastases in the spinal column
Chow, E., Harris, K., & Fung, K. (2006) · International Journal of Radiation Oncology*Biology*Physics · doi:10.1016/j.ijrobp.2006.03.013 · Cited by 19
Modelling to extract more information from clinical trials data: On some roles for the bootstrap
Sauerbrei, W., & Royston, P. J. (2007) · Statistics in Medicine · doi:10.1002/sim.2954 · Cited by 18
Comparison of models to predict clinical failure after radical prostatectomy
Eggener, S. E., Vickers, A. J., Serio, A. M., Donovan, M. J., Khan, F. M., Bayer‐Zubek, V., Verbel, D., Cordon‐Cardo, C., Reuter, V. E., Bianco, F. J., et al. (2008) · Cancer · doi:10.1002/cncr.24016 · Cited by 17
Study design for prediction models
Steyerberg, E. W. (2008) · Statistics in the health sciences · doi:10.1007/978-0-387-77244-8_3 · Cited by 16
Should post-mastectomy radiation be used for node-negative breast cancer patients?
Woodward, W. A. (2007) · The Women s Oncology Review · doi:10.3109/14733400600577479 · Cited by 16
Erratum: Breast carcinoma malignancy grading by Bloom–Richardson system vs proliferation index: reproducibility of grade and advantages of proliferation index
Meyer, J. S., Alvarez, C., Milikowski, C., Olson, N., Russo, I. H., Russo, J., Glass, A. G., Zehnbauer, B. A., Lister, K., & Parwaresch, R. M. (2005) · Modern Pathology · doi:10.1038/modpathol.3800479 · Cited by 13
A risk model for non-small cell lung cancer using clinicopathological variables, angiogenesis and oncoprotein expression.
Rubio, L. P., Vera-Sempere, F. J., López‐Guerrero, J. A., Padilla, J., & Moreno-Baylach, M. J. (2005) · PubMed · Cited by 12
Identifying Good Prognosis Group of Breast Cancer Patients with 1–3 Positive Axillary Nodes for Adjuvant Cyclophosphamide, Methotrexate and 5-Fluorouracil (CMF) Chemotherapy
Lin, Y., Chen, S., Chang, H., Hsueh, S., Tsai, C.-S., Lo, Y., Hwang, T., & Chen, M. (2005) · Japanese Journal of Clinical Oncology · doi:10.1093/jjco/hyi143 · Cited by 10
Prognostic model of event‐free survival for patients with androgen‐independent prostate carcinoma
Shulman, M. J., & Benaim, E. A. (2005) · Cancer · doi:10.1002/cncr.21054 · Cited by 9
A population survival model for breast cancer
Stracci, F., Rosa, F. L., Falsettini, E., Ricci, E., Aristei, C., Bellezza, G., Bolis, G. B., Fenocchio, D., Gori, S., Rulli, A., et al. (2004) · The Breast · doi:10.1016/j.breast.2004.08.011 · Cited by 9
Modified primary tumour/vessel tumour/nodal tumour classification for patients with invasive ductal carcinoma of the breast
Hasebe, T., Iwasaki, M., Akashi‐Tanaka, S., Hojo, T., Shibata, T., Sasajima, Y., Kinoshita, T., & Tsuda, H. (2011) · British Journal of Cancer · doi:10.1038/bjc.2011.279 · Cited by 8
A new staging system for colorectal carcinoma with liver metastasis.
Kato, H., Yoshimatsu, K., Ishibashi, K., Watanabe, K., Shiozawa, S., Tsuchiya, A., Yamada, R., Sakamoto, T., & Haga, S. (2005) · PubMed · Cited by 7
Epidemiologic Theory and Societal Patterns of Disease
Krieger, N. R. (1993) · Epidemiology · doi:10.1097/00001648-199305000-00016 · Cited by 7
Comparison of tumor- and comorbidity-related predictors of mortality after radical prostatectomy
Froehner, M., Koch, R., Litz, R. J., Haase, M., Klenk, U., Oehlschlaeger, S., Baretton, G. B., & Wirth, M. P. (2005) · Scandinavian Journal of Urology and Nephrology · doi:10.1080/00365590510031174 · Cited by 7
Primary tumour–vessel tumour–nodal tumour classification for patients with invasive ductal carcinoma of the breast
Hasebe, T., Sasaki, S., Imoto, S., Wada, N., Ishii, G., & Ochiai, A. (2005) · British Journal of Cancer · doi:10.1038/sj.bjc.6602353 · Cited by 7
Recurrent epithelial ovarian cancer: Validation and improvement of an established prognostic index
Grønlund, B., Christensen, I. J., Bülow-Lehnsby, A. L., Engelholm, S. A., Hansen, H. H., & Høgdall, C. K. (2005) · European Journal of Obstetrics & Gynecology and Reproductive Biology · doi:10.1016/j.ejogrb.2005.03.011 · Cited by 5
A Modified Prognostic Score for Patients with Curatively Resected Gastric Cancer
Mariani, L., Miceli, R., Lusa, L., Bartolomeo, M. D., & Bozzetti, F. (2005) · Tumori Journal · doi:10.1177/030089160509100302 · Cited by 5
Time-Varying Predictors for Clinical Surveillance of Small Hepatocellular Carcinoma
Liao, C., Yang, K.-C., Yen, M., & Chen, T. H.-H. (2005) · The Cancer Journal · doi:10.1097/00130404-200505000-00009 · Cited by 4
Is there really a conceptual difference in choosing the cutpoint for dichotomizing outcome and predictor variables?
Wartenberg, D. E., Northridge, M. E., & Rhoads, G. G. (1993) · PubMed · Cited by 1
The Author Replies
Ragland, D. R. (1993) · Epidemiology · doi:10.1097/00001648-199305000-00015 · Cited by 1
Untitled work
(1999) · doi:10.1002/(sici)1097-0258(19990915/30)18:17/18
Assessing the potential for bias in meta-analysis due to selective reporting of subgroup analyses within studies
Hahn, S., Williamson, P. R., Hutton, J. L., Garner, P., & Flynn, E. V. (2000) · Statistics in Medicine · doi:10.1002/1097-0258(20001230)19:24
What do we mean by validating a prognostic model?
(2000) · doi:10.1002/(sici)1097-0258(20000229)19:4
Untitled work
Harrell, F., Jr. (1996) · doi:10.1002/(sici)1097-0258(19960229)15:4

Research works printed but not linked: 1

Steyerberg EW: Clinical prediction models. 2008, New York: Springer

Sorted by citations

Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): Explanation and Elaboration
Moons, K. G. M., Altman, D. G., Reitsma, J. B., Ioannidis, J. P. A., Macaskill, P., Steyerberg, E. W., Vickers, A. J., Ransohoff, D. F., & Collins, G. S. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0698 · Cited by 5,147
Self-citation
Critical appraisal and data extraction for systematic reviews of prediction modelling studies: the CHARMS checklist
Moons, K. G. M., de Groot, J. A. H., Bouwmeester, W., Vergouwe, Y., Mallett, S., Altman, D. G., Reitsma, J. B., & Collins, G. S. (2014) · PLoS Medicine · doi:10.1371/journal.pmed.1001744 · Cited by 2,018
Self-citation
Prognosis Research Strategy (PROGRESS) 3: Prognostic Model Research
Steyerberg, E. W., Moons, K. G. M., van der Windt, D. A., Hayden, J. A., Perel, P., Schroter, S., Riley, R. D., Hemingway, H., Altman, D. G., & for the PROGRESS Group (2013) · PLoS Medicine · doi:10.1371/journal.pmed.1001381 · Cited by 1,508
Opportunities and challenges in developing risk prediction models with electronic health records data: A systematic review
Goldstein, B. A., Navar, A. M., Pencina, M. J., & Ioannidis, J. P. A. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw042 · Cited by 936
External validation of a Cox prognostic model: principles and methods
Royston, P. J., & Altman, D. G. (2013) · BMC Medical Research Methodology · doi:10.1186/1471-2288-13-33 · Cited by 911

See all 195

Authors: Mallett, S., Royston, P., Waters, R., Dutton, S., & Altman, D. G.
Affiliations: University of Oxford (GB)
MRC Clinical Trials Unit at UCL (GB)
Corresponding author: Mallett, Susan
ORCID: Mallett, Susan
Royston, Patrick
Dutton, Susan
Altman, Douglas G
Venue: BMC Medicine
Publisher: Springer Science and Business Media LLC
Published: 2010-03-30
Volume: 8
Issue: 1
Pages: 21-21
ISSN: 1741-7015 (ISSN-L)
DOI: 10.1186/1741-7015-8-21
PMC: PMC2857810
OpenAlex: W1988375742
Cited by: 195
References: 113
Citations by year: 2026: 3 · 2025: 6 · 2024: 14 · 2023: 11 · 2022: 17 · 2021: 11 · 2020: 14 · 2019: 12 · 2018: 13 · 2017: 10 · 2016: 16 · 2015: 14 · 2014: 18 · 2013: 11 · 2012: 13
FWCI: 1.59
Type: journal article
Language: en
Open access: gold
doi.org
Licence: cc-by
www.springer.com/tdm (unspecified)
Locations: BMC Medicine (publishedVersion, cc-by)
PubMed Central (publishedVersion, cc-by)
PubMed (publishedVersion)
DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
UCL Discovery (University College London) (submittedVersion)
University of Birmingham Research Portal (University of Birmingham) (submittedVersion, other-oa)
Funders: Cancer Research UK
Medical Research Council (MC_EX_G0800814, U.1228.06.001.00002.01)
Topics: Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Health Sciences › Medicine › Oncology › Inflammatory Biomarkers in Disease Prognosis
Keywords: prognostic model, cancer, reporting quality, prognostic score, risk groups, model validation, external validation, discrimination (statistics), calibration, Cox model, hazard ratio, reproducibility, time to event
MeSH terms: Evaluation Studies as Topic, Humans, Neoplasms, Prognosis, Statistics as Topic

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Reporting performance of prognostic models in cancer: a review.

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks