Reporting methods in studies developing prognostic models in cancer: a review
Open accessThe DOI matches its Crossref record; 88 of the 88 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Development of prognostic models enables identification of variables that are influential in predicting patient outcome and the use of these multiple risk factors in a systematic, reproducible way according to evidence based methods. The reliability of models depends on informed use of statistical methods, in combination with prior knowledge of disease. We reviewed published articles to assess reporting and methods used to develop new prognostic models in cancer. METHODS: We developed a systematic search string and identified articles from PubMed. Forty-seven articles were included that satisfied the following inclusion criteria: published in 2005; aiming to predict patient outcome; presenting new prognostic models in cancer with outcome time to an event and including a combination of at least two separate variables; and analysing data using multivariable analysis suitable for time to event data. RESULTS: In 47 studies, prospective cohort or randomised controlled trial data were used for model development in only 33% (15) of studies. In 30% (14) of the studies insufficient data were available, having fewer than 10 events per variable (EPV) used in model development. EPV could not be calculated in a further 40% (19) of the studies. The coding of candidate variables was only reported in 68% (32) of the studies. Although use of continuous variables was reported in all studies, only one article reported using recommended methods of retaining all these variables as continuous without categorisation. Statistical methods for selection of variables in the multivariate modelling were often flawed. A method that is not recommended, namely, using statistical significance in univariate analysis as a pre-screening test to select variables for inclusion in the multivariate model, was applied in 48% (21) of the studies. CONCLUSIONS: We found that published prognostic models are often characterised by both use of inappropriate methods for development of multivariable models and poor reporting. In addition, models are limited by the lack of studies based on prospective data of sufficient sample size to avoid overfitting. The use of poor methods compromises the reliability of prognostic models developed to provide objective probability estimates to complement clinical intuition of the physician and guidelines.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Many reviews have showed that the quality of reporting in published articles describing the development or validation of multivariable prediction models in medicine is poor [23–27,29].”
From Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD Statement (2015) · cites this paper as [26]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Good reporting of how candidate predictors were pre-selected in our review compares favourably with other reviews [15],[20],[21],[29],[33],[35],[36],[48].”
From Reporting and Methods in Clinical Prediction Research: A Systematic Review (2012) · cites this paper as [21]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Many reviews have showed that the quality of reporting in published articles describing the development or validation of multivariable prediction models in medicine is poor.23–27,29”
From Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD): The TRIPOD Statement (2015) · cites this paper as [26]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Many reviews have showed that the quality of reporting in published articles describing the development or validation of multivariable prediction models in medicine is poor (23)(24)(25)(26)(27)29).”
From Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): The TRIPOD Statement (2015) · cites this paper as [25]
The cited source says what the claim attributes to it.
In the citing paper
“However, the overwhelming evidence shows that the quality of reporting of prediction model studies is poor.”
From Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD): The TRIPOD Statement (2015) · cites this paper as [26]
3 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Harrell, F. E. (2001) · Springer Series in Statistics · doi:10.1007/978-1-4757-3462-1 · Cited by 9,195
Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating
Steyerberg, E. W. (2019) · Statistics for Biology and Health · doi:10.1007/978-3-030-16399-0 · Cited by 3,281
The cost of dichotomising continuous variables
Altman, D. G., & Royston, P. (2006) · BMJ · doi:10.1136/bmj.332.7549.1080 · Cited by 2,688
Self-citationDichotomizing continuous predictors in multiple regression: a bad idea
Royston, P., Altman, D. G., & Sauerbrei, W. (2006) · Statistics in Medicine · doi:10.1002/sim.2331 · Cited by 2,312 · cited 2×
“The practice of dichotomising continuous variables is not recommended, as it is an extreme form of rounding that causes loss of information and statistical power, equivalent to losing a third of the data [70].”
BackgroundSelf-citationPeduzzi, P., Concato, J., Feinstein, A. R., & Holford, T. R. (1995) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(95)00048-8 · Cited by 2,154 · cited 2×
“on estimates of the stability of coefficient estimates for individual variables in prognostic model [61-66].”
BackgroundChan, A.-W., Hróbjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004) · JAMA · doi:10.1001/jama.291.20.2457 · Cited by 1,845 · cited 1×
“A review of randomized controlled trials identified selective outcome reporting where the prespecified primary outcome of event -free survival was omitted from published reports [69].”
Self-citation(2012) · Hepatogastroenterology · doi:10.5754/hge12404 · Cited by 1,825
Regression modelling strategies for improved prognostic prediction
Harrell, F. E., Lee, K. L., Califf, R. M., Pryor, D. B., & Rosati, R. A. (1984) · Statistics in Medicine · doi:10.1002/sim.4780030207 · Cited by 1,798 · cited 1×
“on estimates of the stability of coefficient estimates for individual variables in prognostic model [61-66].”
BackgroundSteyerberg, E. W. (2009) · Statistics for Biology and Health · doi:10.1007/978-0-387-77244-8 · Cited by 1,491
Prognosis and prognostic research: validating a prognostic model
Altman, D. G., Vergouwe, Y., Royston, P., & Moons, K. G. M. (2009) · BMJ · doi:10.1136/bmj.b605 · Cited by 1,436 · cited 1×
“In any case, all newly proposed models need independent validation before being considered for clinical use [76].”
BackgroundSelf-citationPrognosis and prognostic research: what, why, and how?
Moons, K. G. M., Royston, P., Vergouwe, Y., Grobbee, D. E., & Altman, D. G. (2009) · BMJ · doi:10.1136/bmj.b375 · Cited by 1,337 · cited 5×
“The best study design to use for development of a prognostic model is a prospective cohort study [2], where baseline variables can be measured for all patients and selection bias due to incomplete availability of data or archival samples can be avoided [60].”
BackgroundInfluentialSelf-citationPrognosis and prognostic research: Developing a prognostic model
Royston, P., Moons, K. G. M., Altman, D. G., & Vergouwe, Y. (2009) · BMJ · doi:10.1136/bmj.b604 · Cited by 1,172 · cited 6×
“We assessed articles in random order using a piloted data extraction form based on key aspects of model design and development from the current literature [1,3,4,6,7].”
BackgroundMethodInfluentialSelf-citationDangers of Using "Optimal" Cutpoints in the Evaluation of Prognostic Factors
Altman, D. G., Lausen, B., Sauerbrei, W., & Schumacher, M. (1994) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/86.11.829 · Cited by 1,143 · cited 1×
“Furthermore choice of an optimal or minimal p-value cutpoint is even worse [72].”
BackgroundSelf-citationAssessment and comparison of prognostic classification schemes for survival data
Graf, E., Schmoor, C., Sauerbrei, W., & Schumacher, M. (1999) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19990915/30)18:17/18<2529::aid-sim274>3.0.co;2-5 · Cited by 975
Prognosis and prognostic research: application and impact of prognostic models in clinical practice
Moons, K. G. M., Altman, D. G., Vergouwe, Y., & Royston, P. (2009) · BMJ · doi:10.1136/bmj.b606 · Cited by 934
Self-citationInappropriate use of bivariable analysis to screen risk factors for use in multivariable analysis
Sun, G.-W., Shook, T. L., & Kay, G. L. (1996) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(96)00025-x · Cited by 933 · cited 2×
“Selection conditional on univariable tests in small sample sizes is likely to introduce error as the correlation between prognostic variables is not properly controlled for [74].”
BackgroundHudis, C. A., Barlow, W. E., Costantino, J. P., Gray, R. J., Pritchard, K. I., Chapman, J.-A. W., Sparano, J. A., Hunsberger, S., Enos, R. A., Gelber, R. D., et al. (2007) · Journal of Clinical Oncology · doi:10.1200/jco.2006.10.3523 · Cited by 864 · cited 3×
“Lack of transparency in the definition of outcomes has been shown to lead to lack of transparency and transportability in studies [67].”
BackgroundResultConcato, J., Peduzzi, P., Holford, T. R., & Feinstein, A. R. (1995) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(95)00510-2 · Cited by 797 · cited 12×
“For the 28 studies (60% of studies) where it could be calculated, the median number of events per candidate model variable (EPV) was 10 (IQR 5 to 33, range 2 to 95).”
BackgroundResultInfluentialCOOPERBERG, M. R., PASTA, D. J., ELKIN, E. P., LITWIN, M. S., LATINI, D. M., Du CHANE, J., & CARROLL, P. R. (2005) · Journal of Urology · doi:10.1097/01.ju.0000158155.33890.e7 · Cited by 772 · cited 2×
“Twenty-six percent (11) of Cox models included all variables in the multivariable analysis [13,28,33,35-37,39,40,46,50,56].”
MethodRoyston, P., & Sauerbrei, W. (2008) · Wiley Series in Probability and Statistics · doi:10.1002/9780470770771 · Cited by 712
Self-citationStephenson, A. J., Scardino, P. T., Eastham, J. A., Bianco, F. J., Dotan, Z. A., DiBlasio, C. J., Reuther, A., Klein, E. A., & Kattan, M. W. (2005) · Journal of Clinical Oncology · doi:10.1200/jco.2005.01.867 · Cited by 663 · cited 2×
“Our search string selected 2,076 articles from 681,530 articles with subject MeSH heading of neoplasm published in 2005, of which 47 met inclusion criteria for this study (Figure 1) [13-59].”
BackgroundMethodRoyston, P. J., & Sauerbrei, W. (2008) · Cited by 601
Self-citationStatistical aspects of prognostic factor studies in oncology
Simon, R., & Altman, D. (1994) · British Journal of Cancer · doi:10.1038/bjc.1994.192 · Cited by 582 · cited 1×
“Clear and transparent reporting of how variables are coded in the model is necessary so the model can be repeated and applied in clinical practice [73].”
BackgroundSelf-citationMekhail, T. M., Abou-Jawde, R. M., BouMerhi, G., Malhi, S., Wood, L., Elson, P., & Bukowski, R. (2005) · Journal of Clinical Oncology · doi:10.1200/jco.2005.05.179 · Cited by 559 · cited 4×
“In 21% (10) of articles models were based on prospective cohorts [16,21,22,26,34,43,45,46,56,59] (Table 1), two of which were based on phase II trials [16,45].”
BackgroundMethodInfluentialSORBELLINI, M., KATTAN, M. W., SNYDER, M. E., REUTER, V., MOTZER, R., GOETZL, M., McKIERNAN, J., & RUSSO, P. (2005) · Journal of Urology · doi:10.1097/01.ju.0000148261.19532.2c · Cited by 528 · cited 2×
“Twenty-six percent (11) of Cox models included all variables in the multivariable analysis [13,28,33,35-37,39,40,46,50,56].”
MethodPrognostic Index for Adult Patients With Acute Myeloid Leukemia in First Relapse
Breems, D. A., Van Putten, W. L. J., Huijgens, P. C., Ossenkoppele, G. J., Verhoef, G. E. G., Verdonck, L. F., Vellenga, E., De Greef, G. E., Jacky, E., Van der Lelie, J., et al. (2005) · Journal of Clinical Oncology · doi:10.1200/jco.2005.06.027 · Cited by 511 · cited 8×
“Patients recruited from RCTs in 11% (5) of the studies [14,38,40,44,47].”
BackgroundMethodInfluentialHennessy, B. T., Hortobagyi, G. N., Rouzier, R., Kuerer, H., Sneige, N., Buzdar, A. U., Kau, S. W., Fornage, B., Sahin, A., Broglio, K., et al. (2005) · Journal of Clinical Oncology · doi:10.1200/jco.2005.02.5023 · Cited by 420 · cited 6×
“Our search string selected 2,076 articles from 681,530 articles with subject MeSH heading of neoplasm published in 2005, of which 47 met inclusion criteria for this study (Figure 1) [13-59].”
BackgroundMethodResultInfluentialvan der Linden, Y. M., Dijkstra, S. P. D. S., Vonk, E. J. A., Marijnen, C. A. M., Leer, J. W. H., & for The Dutch Bone Metastasis Study Group (2005) · Cancer · doi:10.1002/cncr.20756 · Cited by 396
Schmidt, H., Bastholt, L., Geertsen, P., Christensen, I. J., Larsen, S., Gehl, J., & von der Maase, H. (2005) · British Journal of Cancer · doi:10.1038/sj.bjc.6602702 · Cited by 329
Chow, E., Wu, J. S. Y., Hoskin, P., Coia, L. R., Bentzen, S. M., & Blitzer, P. H. (2002) · Radiotherapy and Oncology · doi:10.1016/s0167-8140(02)00170-6 · Cited by 314 · cited 1×
“Studies in breast cancer [67] and treatment of bone metastasis [68] have found that outcome definitions are often unclear and inconsistent between studies.”
ResultBootstrap investigation of the stability of a cox regression model
Altman, D. G., & Andersen, P. K. (1989) · Statistics in Medicine · doi:10.1002/sim.4780080702 · Cited by 310 · cited 1×
“on estimates of the stability of coefficient estimates for individual variables in prognostic model [61-66].”
BackgroundSelf-citationWilczynski, N. L., Haynes, R. B., & the Hedges Team (2004) · BMC Medicine · doi:10.1186/1741-7015-2-23 · Cited by 291
Prognostic factors and a scoring system for patients with skeletal metastasis
Katagiri, H., Takahashi, M., Wakai, K., Sugiura, H., Kataoka, T., & Nakanishi, K. (2005) · The Journal of Bone and Joint Surgery. British volume · doi:10.1302/0301-620x.87b5.15185 · Cited by 279 · cited 3×
“Twenty-six percent (11) of Cox models included all variables in the multivariable analysis [13,28,33,35-37,39,40,46,50,56].”
BackgroundMethodProposal of a new prognostic model for hepatocellular carcinoma: an analysis of 403 patients
Tateishi, R., Yoshida, H., Shiina, S., Imamura, H., Hasegawa, K., Teratani, T., Obi, S., Sato, S., Koike, Y., Fujishima, T., et al. (2005) · Gut · doi:10.1136/gut.2003.035055 · Cited by 273 · cited 3×
“Ten of these studies had the same number of eligible patients as included patients, suggesting that complete patient data were an unspecified inclusion criterion [24,27,30-32,36,38,48,51,55].”
MethodResultHoang, T., Xu, R., Schiller, J. H., Bonomi, P., & Johnson, D. H. (2005) · Journal of Clinical Oncology · doi:10.1200/jco.2005.04.177 · Cited by 263 · cited 4×
“Patients recruited from RCTs in 11% (5) of the studies [14,38,40,44,47].”
BackgroundMethodInfluentialMeyer, J. S., Alvarez, C., Milikowski, C., Olson, N., Russo, I., Russo, J., Glass, A., Zehnbauer, B. A., Lister, K., & Parwaresch, R. (2005) · Modern Pathology · doi:10.1038/modpathol.3800388 · Cited by 262 · cited 2×
“In 7% (3) of the models authors did not report methods used to include variables in the final model [44,55, 57 ].”
MethodBurton, A., & Altman, D. G. (2004) · British Journal of Cancer · doi:10.1038/sj.bjc.6601907 · Cited by 219 · cited 2×
“Several search strings have been developed [9-11] but other studies have used hand-searching [12].”
LEIBOVICH, B. C., CHEVILLE, J. C., LOHSE, C. M., ZINCKE, H., FRANK, I., KWON, E. D., MERCHAN, J. R., & BLUTE, M. L. (2005) · Journal of Urology · doi:10.1097/01.ju.0000177487.64651.3a · Cited by 218
USING TUMOR MARKERS TO PREDICT THE SURVIVAL OF PATIENTS WITH METASTATIC RENAL CELL CARCINOMA
KIM, H. L., SELIGSON, D., LIU, X., JANZEN, N., BUI, M. H. T., YU, H., SHI, T., BELLDEGRUN, A. S., HORVATH, S., & FIGLIN, R. A. (2005) · Journal of Urology · doi:10.1097/01.ju.0000154351.37249.f0 · Cited by 216 · cited 4×
“Twenty-six percent (11) of Cox models included all variables in the multivariable analysis [13,28,33,35-37,39,40,46,50,56].”
BackgroundMethodResultInfluentialNathan, S. S., Healey, J. H., Mellano, D., Hoang, B., Lewis, I., Morris, C. D., Athanasian, E. A., & Boland, P. J. (2005) · Journal of Clinical Oncology · doi:10.1200/jco.2005.08.104 · Cited by 214 · cited 2×
“In 21% (10) of articles models were based on prospective cohorts [16,21,22,26,34,43,45,46,56,59] (Table 1), two of which were based on phase II trials [16,45].”
BackgroundMethodfor the oncoLR health network, Colinet, B., Jacot, W., Bertrand, D., Lacombe, S., Bozonnat, M.-C., Daurès, J.-P., & Pujol, J.-L. (2005) · British Journal of Cancer · doi:10.1038/sj.bjc.6602836 · Cited by 211 · cited 2×
“In 21% (10) of articles models were based on prospective cohorts [16,21,22,26,34,43,45,46,56,59] (Table 1), two of which were based on phase II trials [16,45].”
BackgroundMethodMariani, L., Miceli, R., Kattan, M. W., Brennan, M. F., Colecchia, M., Fiore, M., Casali, P. G., & Gronchi, A. (2005) · Cancer · doi:10.1002/cncr.20778 · Cited by 211 · cited 2×
“Our search string selected 2,076 articles from 681,530 articles with subject MeSH heading of neoplasm published in 2005, of which 47 met inclusion criteria for this study (Figure 1) [ 13-59 ].”
MethodReporting performance of prognostic models in cancer: a review.
Mallett, S., Royston, P., Waters, R., Dutton, S., & Altman, D. G. (2010) · BMC Medicine · doi:10.1186/1741-7015-8-21 · Cited by 195 · cited 5×
“In a companion paper, Mallett et al [8], we report the assessment of methods and reporting of multivariable analysis, numerical and graphical presentation of the model, crea-”
MethodInfluentialSelf-citationSearching for Clinical Prediction Rules in MEDLINE
Ingui, B. J., & Rogers, M. A. M. (2001) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2001.0080391 · Cited by 184
Matsui, Y., Utsunomiya, N., Ichioka, K., Ueda, N., Yoshimura, K., Terai, A., & Arai, Y. (2005) · Urology · doi:10.1016/j.urology.2004.09.021 · Cited by 161 · cited 3×
“Twenty-six percent (11) of Cox models included all variables in the multivariable analysis [13,28,33,35-37,39,40,46,50,56].”
MethodResultBower, M., Gazzard, B., Mandalia, S., Newsom‐Davis, T., Thirlwell, C., Dhillon, T., Young, A. M., Powles, T., Gaya, A. M., Nelson, M. R., et al. (2005) · Annals of Internal Medicine · doi:10.7326/0003-4819-143-4-200508160-00007 · Cited by 132 · cited 5×
“The authors of two articles appear confused as to the purpose of their models, and included adjustment for six to nine variables in a footnote to their model, while retaining only two significant variables in the predictive score [18,43].”
BackgroundMethodInfluentialPrognostic Models: A Methodological Framework and Review of Models for Breast Cancer
Altman, D. G. (2009) · Cancer Investigation · doi:10.1080/07357900802572110 · Cited by 129 · cited 1×
“We assessed articles in random order using a piloted data extraction form based on key aspects of model design and development from the current literature [1,3,4,6,7].”
MethodSelf-citationLangendijk, J. A., Slotman, B. J., van der Waal, I., Doornaert, P., Berkof, J., & Leemans, C. R. (2005) · Cancer · doi:10.1002/cncr.21340 · Cited by 128 · cited 1×
“Our search string selected 2,076 articles from 681,530 articles with subject MeSH heading of neoplasm published in 2005, of which 47 met inclusion criteria for this study (Figure 1) [13-59].”
MethodSurgical management of spinal metastases: analysis of prognostic factors during a 10-year experience
North, R. B., LaRocca, V. R., Schwartz, J., North, C. A., ZAHURAK, M. L., Davis, R. F., & McAfee, P. C. (2005) · Journal of Neurosurgery Spine · doi:10.3171/spi.2005.2.5.0564 · Cited by 128 · cited 1×
“Our search string selected 2,076 articles from 681,530 articles with subject MeSH heading of neoplasm published in 2005, of which 47 met inclusion criteria for this study (Figure 1) [13-59].”
MethodOttenbacher, K. J., Ottenbacher, H. R., Tooth, L. R., & Ostir, G. V. (2004) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2003.05.003 · Cited by 127 · cited 9×
“For the 28 studies (60% of studies) where it could be calculated, the median number of events per candidate model variable (EPV) was 10 (IQR 5 to 33, range 2 to 95).”
BackgroundInfluentialTruong, P. T., Lesperance, M., Culhaci, A., Kader, H. A., Speers, C. H., & Olivotto, I. A. (2005) · International Journal of Radiation Oncology*Biology*Physics · doi:10.1016/j.ijrobp.2004.09.013 · Cited by 105 · cited 2×
“Ten of these studies had the same number of eligible patients as included patients, suggesting that complete patient data were an unspecified inclusion criterion [24,27,30-32,36,38,48,51,55].”
MethodResultSteyerberg, E. W., Homs, M. Y. V., Stokvis, A., Essink‐Bot, M., & Siersema, P. D. (2005) · Gastrointestinal Endoscopy · doi:10.1016/s0016-5107(05)01587-7 · Cited by 103 · cited 9×
“Patients recruited from RCTs in 11% (5) of the studies [14,38,40,44,47].”
BackgroundMethodResultInfluentialWilliams, C., Brunskill, S., Altman, D., Briggs, A., Campbell, H., Clarke, M., Glanville, J., Gray, A., Harris, A., Johnston, K., et al. (2006) · Health Technology Assessment · doi:10.3310/hta10340 · Cited by 92 · cited 1×
“Several search strings have been developed [9-11] but other studies have used hand-searching [12].”
BackgroundSelf-citationSlovin, S. F., Wilton, A. S., Heller, G., & Scher, H. I. (2005) · Clinical Cancer Research · doi:10.1158/1078-0432.ccr-05-1668 · Cited by 92 · cited 4×
“Twenty-six percent (11) of Cox models included all variables in the multivariable analysis [13,28,33,35-37,39,40,46,50,56].”
BackgroundMethodInfluentialKikuchi, E., Horiguchi, Y., Nakashima, J., HATAKEYAMA, N., Matsumoto, M., Nishiyama, T., & Murai, M. (2005) · The Journal of Urology · doi:10.1097/01.ju.0000181801.22474.8b · Cited by 84 · cited 2×
“Twenty-six percent (11) of Cox models included all variables in the multivariable analysis [13,28,33,35-37,39,40,46,50,56].”
MethodA new system for substaging pT1 papillary bladder cancer: a prognostic evaluation
VANDERAA, M., VANLEENDERS, G., Steyerberg, E. W., VANRHIJN, B., Jöbsis, A. C., Zwarthoff, E. C., & van der Kwast, T. (2005) · Human Pathology · doi:10.1016/j.humpath.2005.06.017 · Cited by 79
Klemke, C.-D., Mansmann, U., Poenitz, N., Dippel, E., & Goerdt, S. (2005) · British Journal of Dermatology · doi:10.1111/j.1365-2133.2005.06676.x · Cited by 66 · cited 1×
“Our search string selected 2,076 articles from 681,530 articles with subject MeSH heading of neoplasm published in 2005, of which 47 met inclusion criteria for this study (Figure 1) [13-59].”
MethodPotential for Selection Bias with Tumor Tissue Retrieval in Molecular Epidemiology Studies
Hoppin, J. A. (2002) · Annals of Epidemiology · doi:10.1016/s1047-2797(01)00250-2 · Cited by 65 · cited 2×
“Most prognostic models, 68% in this study, are based on retrospective databases which have many inherent biases, often due to patchy availability of patient information [60].”
BackgroundSmith, B. D., Smith, G. L., COOPER, D. L., & Wilson, L. D. (2005) · Journal of Clinical Oncology · doi:10.1200/jco.2005.08.137 · Cited by 64 · cited 2×
“The authors of two articles appear confused as to the purpose of their models, and included adjustment for six to nine variables in a footnote to their model, while retaining only two significant variables in the predictive score [18,43].”
BackgroundMethodJerez, J. M., Franco, L., Alba, E., Llombart‐Cussac, A., Lluch, A. M., Ribelles, N., Munárriz, B., & Martín, M. (2005) · Breast Cancer Research and Treatment · doi:10.1007/s10549-005-9013-y · Cited by 64 · cited 1×
“Our search string selected 2,076 articles from 681,530 articles with subject MeSH heading of neoplasm published in 2005, of which 47 met inclusion criteria for this study (Figure 1) [13-59].”
MethodClaude, L., Pérol, D., Ray-Coquard, I. L., Petit, T., Blay, J., Carrié, C., & Bachelot, T. (2005) · Radiotherapy and Oncology · doi:10.1016/j.radonc.2005.06.004 · Cited by 61 · cited 3×
“Twenty-six percent (11) of Cox models included all variables in the multivariable analysis [13,28,33,35-37,39,40,46,50,56].”
BackgroundMethodYildirim, E. (2005) · Journal of the American College of Surgeons · doi:10.1016/j.jamcollsurg.2004.10.031 · Cited by 49 · cited 1×
“Our search string selected 2,076 articles from 681,530 articles with subject MeSH heading of neoplasm published in 2005, of which 47 met inclusion criteria for this study (Figure 1) [13-59].”
MethodPeccatori, J., Barkholt, L., Demirer, T., Sormani, M. P., Bruzzi, P., Ciceri, F., Zambelli, A., Prada, G. A. D., Pedrazzoli, P., Siena, S., et al. (2005) · Cancer · doi:10.1002/cncr.21477 · Cited by 48 · cited 2×
“Ten of these studies had the same number of eligible patients as included patients, suggesting that complete patient data were an unspecified inclusion criterion [24,27,30-32,36,38,48,51,55].”
MethodResultEvaluation of Logistic Regression Reporting in Current Obstetrics and Gynecology Literature
Mikolajczyk, R. T., DiSilvesto, A., & Zhang, J. (2008) · Obstetrics & Gynecology · doi:10.1097/aog.0b013e318160f38e · Cited by 36
Reporting of Multivariable Methods in the Medical Literature
Tetrault, J. M., Sauler, M., Wells, C. K., & Concato, J. (2008) · Journal of Investigative Medicine · doi:10.2310/jim.0b013e31818914ff · Cited by 33 · cited 1×
“Poor reporting of variable selection has been described similarly in other studies [65,80].”
ResultMolica, S., Mauro, F. R., Callea, V., Gentile, M., Giannarelli, D., Lopez, M., Lauria, F., Rotoli, B., Montanaro, M., Cortelezzi, A., et al. (2005) · Leukemia & lymphoma/Leukemia and lymphoma · doi:10.1080/10428190400029965 · Cited by 27
Hahn, T., Beneklı, M., Wong, C., Moysich, K. B., Hyland, A. J., Michalek, A. M., Alam, A. R., Baer, M. R., Bambach, B., Czuczman, M. S., et al. (2005) · Bone Marrow Transplantation · doi:10.1038/sj.bmt.1704789 · Cited by 27 · cited 3×
“Ten of these studies had the same number of eligible patients as included patients, suggesting that complete patient data were an unspecified inclusion criterion [24,27,30-32,36,38,48,51,55].”
BackgroundMethodResultPrognostic factors in patients with advanced biliary tract cancer receiving chemotherapy.
Saisho, T., Okusaka, T., Ueno, H., Morizane, C., & Okada, S. (2005) · PubMed · Cited by 26
Tey, S., Butler, J., Durrant, S., Hill, G. R., Morton, J., & Kennedy, G. (2005) · Bone Marrow Transplantation · doi:10.1038/sj.bmt.1705044 · Cited by 20
A prognostic index (bladder prognostic index) for bilharzial-related invasive bladder cancer
Khaled, H. M., Hattab, O. E., Moneim, D. A., Kassem, H. A., Morsi, A., Sherif, G. M., Darwish, T., & Gaafar, R. M. (2005) · Urologic Oncology Seminars and Original Investigations · doi:10.1016/j.urolonc.2005.04.005 · Cited by 20 · cited 2×
“In 21% (10) of articles models were based on prospective cohorts [16,21,22,26,34,43,45,46,56,59] (Table 1), two of which were based on phase II trials [16,45].”
BackgroundMethodSolsona, E., Iborra, I., Dumont, R., Rubio, J., Casanova, J. L., & Almenar, S. M. (2005) · The Journal of Urology · doi:10.1097/01.ju.0000173920.31824.2f · Cited by 19 · cited 1×
“Our search string selected 2,076 articles from 681,530 articles with subject MeSH heading of neoplasm published in 2005, of which 47 met inclusion criteria for this study (Figure 1) [13-59].”
MethodStudy design for prediction models
Steyerberg, E. W. (2008) · Statistics in the health sciences · doi:10.1007/978-0-387-77244-8_3 · Cited by 16
Should post-mastectomy radiation be used for node-negative breast cancer patients?
Woodward, W. A. (2007) · The Women s Oncology Review · doi:10.3109/14733400600577479 · Cited by 16
Meyer, J. S., Alvarez, C., Milikowski, C., Olson, N., Russo, I. H., Russo, J., Glass, A. G., Zehnbauer, B. A., Lister, K., & Parwaresch, R. M. (2005) · Modern Pathology · doi:10.1038/modpathol.3800479 · Cited by 13
Rubio, L. P., Vera-Sempere, F. J., López‐Guerrero, J. A., Padilla, J., & Moreno-Baylach, M. J. (2005) · PubMed · Cited by 12
Lin, Y., Chen, S., Chang, H., Hsueh, S., Tsai, C.-S., Lo, Y., Hwang, T., & Chen, M. (2005) · Japanese Journal of Clinical Oncology · doi:10.1093/jjco/hyi143 · Cited by 10 · cited 2×
“Ten of these studies had the same number of eligible patients as included patients, suggesting that complete patient data were an unspecified inclusion criterion [24,27,30-32,36,38,48,51,55].”
MethodResultPrognostic model of event‐free survival for patients with androgen‐independent prostate carcinoma
Shulman, M. J., & Benaim, E. A. (2005) · Cancer · doi:10.1002/cncr.21054 · Cited by 9 · cited 1×
“Our search string selected 2,076 articles from 681,530 articles with subject MeSH heading of neoplasm published in 2005, of which 47 met inclusion criteria for this study (Figure 1) [13-59].”
MethodA population survival model for breast cancer
Stracci, F., Rosa, F. L., Falsettini, E., Ricci, E., Aristei, C., Bellezza, G., Bolis, G. B., Fenocchio, D., Gori, S., Rulli, A., et al. (2004) · The Breast · doi:10.1016/j.breast.2004.08.011 · Cited by 9 · cited 3×
“Ten of these studies had the same number of eligible patients as included patients, suggesting that complete patient data were an unspecified inclusion criterion [24,27,30-32,36,38,48,51,55].”
BackgroundMethodResultHasebe, T., Iwasaki, M., Akashi‐Tanaka, S., Hojo, T., Shibata, T., Sasajima, Y., Kinoshita, T., & Tsuda, H. (2011) · British Journal of Cancer · doi:10.1038/bjc.2011.279 · Cited by 8
A new staging system for colorectal carcinoma with liver metastasis.
Kato, H., Yoshimatsu, K., Ishibashi, K., Watanabe, K., Shiozawa, S., Tsuchiya, A., Yamada, R., Sakamoto, T., & Haga, S. (2005) · PubMed · Cited by 7
Comparison of tumor- and comorbidity-related predictors of mortality after radical prostatectomy
Froehner, M., Koch, R., Litz, R. J., Haase, M., Klenk, U., Oehlschlaeger, S., Baretton, G. B., & Wirth, M. P. (2005) · Scandinavian Journal of Urology and Nephrology · doi:10.1080/00365590510031174 · Cited by 7 · cited 2×
“In two models researchers retained only significant variables in the final model, but then added to or removed variables from the prognostic score without repeating the model fit, based on investigators’ choice [29, 58 ].”
MethodHasebe, T., Sasaki, S., Imoto, S., Wada, N., Ishii, G., & Ochiai, A. (2005) · British Journal of Cancer · doi:10.1038/sj.bjc.6602353 · Cited by 7
Reporting of Predictive Logistic Models Should Be Based on Evidence-Based Guidelines
Hernández, A. V., Vergouwe, Y., & Steyerberg, E. W. (2003) · CHEST Journal · doi:10.1378/chest.124.5.2034 · Cited by 6
Recurrent epithelial ovarian cancer: Validation and improvement of an established prognostic index
Grønlund, B., Christensen, I. J., Bülow-Lehnsby, A. L., Engelholm, S. A., Hansen, H. H., & Høgdall, C. K. (2005) · European Journal of Obstetrics & Gynecology and Reproductive Biology · doi:10.1016/j.ejogrb.2005.03.011 · Cited by 5 · cited 4×
“In 23% (11) of the studies reasons were given for sample size; with one exception all these studies were RCTs or prospective cohorts [14,16,25,26,38,40,43-45,54,56].”
BackgroundMethodInfluentialA Modified Prognostic Score for Patients with Curatively Resected Gastric Cancer
Mariani, L., Miceli, R., Lusa, L., Bartolomeo, M. D., & Bozzetti, F. (2005) · Tumori Journal · doi:10.1177/030089160509100302 · Cited by 5 · cited 6×
“Patients recruited from RCTs in 11% (5) of the studies [14,38,40,44,47].”
BackgroundMethodInfluentialTime-Varying Predictors for Clinical Surveillance of Small Hepatocellular Carcinoma
Liao, C., Yang, K.-C., Yen, M., & Chen, T. H.-H. (2005) · The Cancer Journal · doi:10.1097/00130404-200505000-00009 · Cited by 4 · cited 2×
“In 21% (10) of articles models were based on prospective cohorts [16, 21 ,22,26,34,43,45,46,56,59] (Table 1), two of which were based on phase II trials [16,45].”
BackgroundMethodWhat do we mean by validating a prognostic model?
(2000) · doi:10.1002/(sici)1097-0258(20000229)19:4<453::aid-sim350>3.0.co;2-5
Self-citationHarrell, F., Jr. (1996) · doi:10.1002/(sici)1097-0258(19960229)15:4<361::aid-sim168>3.0.co;2-4
Cited by
Moons, K. G. M., Altman, D. G., Reitsma, J. B., Ioannidis, J. P. A., Macaskill, P., Steyerberg, E. W., Vickers, A. J., Ransohoff, D. F., & Collins, G. S. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0698 · Cited by 5,147
Self-citationCollins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · BMJ · doi:10.1136/bmj.g7594 · Cited by 4,388
“For example, Mallett and colleagues (26) examined 47 reports published in 2005 presenting new prediction models in cancer.”
Self-citationCollins, G. S., Moons, K. G. M., Dhiman, P., Riley, R. D., Beam, A. L., Van Calster, B., Ghassemi, M., Liu, X., Reitsma, J. B., van Smeden, M., et al. (2024) · BMJ · doi:10.1136/bmj-2023-078378 · Cited by 4,718
Collins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · British Journal of Cancer · doi:10.1038/bjc.2014.639 · Cited by 3,293
“For example, Mallett and colleagues (26) examined 47 reports published in 2005 presenting new prediction models in cancer.”
InfluentialSelf-citationCollins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0697 · Cited by 3,267
“For example, Mallett and colleagues [26] examined 47 reports published in 2005 presenting new prediction models in cancer.”
BackgroundSelf-citation
Metadata
- Authors
- Mallett, S., Royston, P., Dutton, S., Waters, R., & Altman, D. G.
- Affiliations
- University of Oxford (GB)
- MRC Clinical Trials Unit at UCL (GB)
- Corresponding author
- Mallett, Susan
- Venue
- BMC Medicine
- Published
- 2010-03-30
- Volume
- 8
- Issue
- 1
- Pages
- 20-20
- ISSN
- 1741-7015 (ISSN-L)
- PMC
- PMC2856521
- OpenAlex
- W2133103557
- Cited by
- 206
- References
- 88
- Citations by year
- 2026: 6 · 2025: 10 · 2024: 8 · 2023: 14 · 2022: 12 · 2021: 13 · 2020: 11 · 2019: 15 · 2018: 12 · 2017: 16 · 2016: 10 · 2015: 22 · 2014: 19 · 2013: 12 · 2012: 12
- FWCI
- 1.70
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- BMC Medicine (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- UCL Discovery (University College London) (submittedVersion)
- University of Birmingham Research Portal (University of Birmingham) (submittedVersion, other-oa)
- Funders
- Cancer Research UK
- Medical Research Council (MC_EX_G0800814, U.1228.06.001.00002.01)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Inference
- Keywords
- prognostic model, cancer, reporting quality, feature selection, time to event, overfitting, prospective cohort, randomized controlled trials, variable coding
- MeSH terms
- Humans, Neoplasms, Prognosis, Statistics as Topic, Multivariate Analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
