Methodological challenges in the evaluation of prognostic factors in breast cancer.
Abstract
Many studies are carried out in an effort to find factors that help explain the large unexplained variation in prognosis of breast cancer patients. The principles of good study design and analysis are less well appreciated for prognostic factor studies than for therapeutic trials. The oncology literature is full of results from studies of varying quality, often with conflicting findings. As a consequence, despite the large number of studies, there is still uncertainty about the importance of most prognostic factors. Few recently proposed prognostic factors for breast cancer have become widely accepted. This paper reviews the important methodological issues underlying such research. These issues are illustrated with examples from published studies and recent reviews of papers published in cancer journals. Guidelines are proposed for conducting and evaluating prognostic factor studies which should improve the quality of research in this important area.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Poor reporting of sample size rationale has also been observed by others.
“Poor reporting of sample size rationale has also been observed by others [3],[21],[68].”
From Reporting and Methods in Clinical Prediction Research: A Systematic Review (2012) · cites this paper as [3]
Cites
STATISTICAL METHODS FOR ASSESSING AGREEMENT BETWEEN TWO METHODS OF CLINICAL MEASUREMENT
Martin Bland, J., & Altman, D. (1986) · The Lancet · doi:10.1016/s0140-6736(86)90837-8 · Cited by 47,965 · see citing works
Self-citationRegression Models and Life-Tables
Cox, D. R. (1972) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1972.tb00899.x · Cited by 39,793 · see citing works
Regression Models and Life-Tables
Cox, D. R. (1992) · Springer Series in Statistics · doi:10.1007/978-1-4612-4380-9_37 · Cited by 26,297 · see citing works
A Predictive Model for Aggressive Non-Hodgkin's Lymphoma
The International Non-Hodgkin's Lymphoma Prognostic Factors Project (1993) · New England Journal of Medicine · doi:10.1056/nejm199309303291402 · Cited by 4,950 · see citing works
Flexible regression models with cubic splines
Durrleman, S., & Simon, R. M. (1989) · Statistics in Medicine · doi:10.1002/sim.4780080504 · Cited by 2,701 · see citing works
Peduzzi, P., Concato, J., Feinstein, A. R., & Holford, T. R. (1995) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(95)00048-8 · Cited by 2,154 · see citing works
Regression Using Fractional Polynomials of Continuous Covariates: Parsimonious Parametric Modelling
Royston, P. J., & Altman, D. G. (1994) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.2307/2986270 · Cited by 1,843 · see citing works
Self-citationSample Size Tables for Clinical Studies
Machin, D., Campbell, M. J., Tan, S. B., & Tan, S. H. (2008) · Wiley eBooks · doi:10.1002/9781444300710 · Cited by 1,336 · see citing works
Dangers of Using "Optimal" Cutpoints in the Evaluation of Prognostic Factors
Altman, D. G., Lausen, B., Sauerbrei, W., & Schumacher, M. (1994) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/86.11.829 · Cited by 1,143 · see citing works
Self-citationPublication bias: evidence of delayed publication in a cohort study of clinical research projects
Stern, J. M., & Simes, R. J. (1997) · BMJ · doi:10.1136/bmj.315.7109.640 · Cited by 818 · see citing works
A prognostic index in primary breast cancer.
Haybittle, J., Blamey, R., Elston, C., Johnson, J., Doyle, P., Campbell, F., Nicholson, R. I., & Griffiths, K. (1982) · British Journal of Cancer · doi:10.1038/bjc.1982.62 · Cited by 758 · see citing works
A bootstrap resampling procedure for model building: Application to the cox regression model
Sauerbrei, W., & Schumacher, M. (1992) · Statistics in Medicine · doi:10.1002/sim.4780111607 · Cited by 601 · see citing works
Statistical aspects of prognostic factor studies in oncology
Simon, R. M., & Altman, D. (1994) · British Journal of Cancer · doi:10.1038/bjc.1994.192 · Cited by 589 · see citing works
Self-citationRegression models for prognostic prediction: advantages, problems, and suggested solutions.
Fe, H., Kl, L., Matchar, D. B., & Ta, R. (1985) · PubMed · Cited by 559 · see citing works
Some Thoughts on Clinical Trials, Especially Problems of Multiplicity
Tukey, J. W. (1977) · Science · doi:10.1126/science.333584 · Cited by 480 · see citing works
Press, M. F., Hung, G., Godolphin, W. J., & Slamon, D. (1994) · PubMed · Cited by 478 · see citing works
Review of survival analyses published in cancer journals
Altman, D., De Stavola, B., Love, S., & Stepniewska, K. (1995) · British Journal of Cancer · doi:10.1038/bjc.1995.364 · Cited by 438 · see citing works
Self-citationConfidence Intervals for Reporting Results of Clinical Trials
Simon, R. M. (1986) · Annals of Internal Medicine · doi:10.7326/0003-4819-105-3-429 · Cited by 416 · see citing works
Commentary: Prognostic models: clinically useful or quickly forgotten?
Wyatt, J. C., & Altman, D. G. (1995) · BMJ · doi:10.1136/bmj.311.7019.1539 · Cited by 367 · see citing works
Self-citationJänicke, F., Schmitt, M., Pache, L., Ulm, K., Harbeck, N., Höfler, H., & Graeff, H. (1993) · Breast Cancer Research and Treatment · doi:10.1007/bf01833260 · Cited by 354 · see citing works
Bootstrap investigation of the stability of a cox regression model
Altman, D. G., & Andersen, P. K. (1989) · Statistics in Medicine · doi:10.1002/sim.4780080702 · Cited by 311 · see citing works
Self-citationConfirmation of a prognostic index in primary breast cancer
Todd, J. H., Dowle, C. S., Williams, M., Elston, C., Ellis, I. O., Hinton, C. P., Blamey, R. W., & Haybittle, J. L. (1987) · British Journal of Cancer · doi:10.1038/bjc.1987.230 · Cited by 308 · see citing works
Breast Cancer Prognostic Factors: Evaluation Guidelines
McGuire, W. L. (1991) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/83.3.154 · Cited by 298 · see citing works
SINGLE SOLUTION FOR ORAL THERAPY OF DIARRHŒA
Hirschhorn, N. (1975) · The Lancet · doi:10.1016/s0140-6736(75)90351-7 · Cited by 281 · see citing works
Gasparini, G., Pozza, F., & Harris, A. L. (1993) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/85.15.1206 · Cited by 244 · see citing works
Tumor Angiogenesis as a Prognostic Assay for Invasive Ductal Breast Carcinoma
Axelsson, K., Ljung, B., Moore, D. H., Thor, A. D., Chew, K., Edgerton, S. M., Smith, H. S., & Mayall, B. H. (1995) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/87.13.997 · Cited by 218 · see citing works
An overview of prognostic factors in small cell lung cancer
Rawson, N. S. B., & Peto, J. (1990) · British Journal of Cancer · doi:10.1038/bjc.1990.133 · Cited by 204 · see citing works
Chen, C., & George, S. L. (1985) · Statistics in Medicine · doi:10.1002/sim.4780040107 · Cited by 202 · see citing works
The future of prognostic factors in outcome prediction for patients with cancer
FIELDING, L. P., Fenoglio‐Preiser, C. M., & Freedman, L. S. (1992) · Cancer · doi:10.1002/1097-0142(19921101)70:9 · Cited by 166 · see citing works
Fox, S. B., Smith, K., Hollyer, J., Greenall, M. J., Hastrich, D. J., & Harris, A. L. (1994) · Breast Cancer Research and Treatment · doi:10.1007/bf00666180 · Cited by 160 · see citing works
Herrero, R., Brinton, L. A., Reeves, W. C., Brenes, M. M., Tenorio, F., de Britton, R. C., Gaitán, E., García, M., & Rawls, W. E. (1990) · Cancer · doi:10.1002/1097-0142(19900115)65:2 · Cited by 158 · see citing works
Re: Tumor Angiogenesis as a Prognostic Assay for Invasive Ductal Breast Carcinoma
Vermeulen, P., Libura, J., Libura, M. A., Hellemans, P., Marck, E. V., OOSTEROM, A. T. V., & DIRIX, L. Y. (1995) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/87.23.1797 · Cited by 157 · see citing works
Thor, A. D., Benz, C. C., Moore, D. H., Goldman, E., Edgerton, S., Landry, J., Schwartz, L. H., Mayall, B. H., Hickey, E. D., & Weber, L. A. (1991) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/83.3.170 · Cited by 150 · see citing works
Small-cell carcinoma of the lung: derivation of a prognostic staging system.
Sagman, U., Maki, E., Evans, W. K., Warr, D. G., Shepherd, F. A., Sculier, J., Haddad, R., Payne, D. G., Pringle, J. F., & Yeoh, J. L. (1991) · Journal of Clinical Oncology · doi:10.1200/jco.1991.9.9.1639 · Cited by 145 · see citing works
Why do so many prognostic factors fail to pan out?
Hilsenbeck, S. G., Clark, G. M., & McGuire, W. L. (1992) · Breast Cancer Research and Treatment · doi:10.1007/bf01840833 · Cited by 138 · see citing works
UROKINASE-TYPE PLASMINOGEN ACTIVATOR ANTIGEN AND EARLY RELAPSE IN BREAST CANCER
Jänicke, F., Schmitt, M., Ulm, K., Gössner, W., & Graeff, H. (1989) · The Lancet · doi:10.1016/s0140-6736(89)91070-2 · Cited by 130 · see citing works
Patient subsets and variation in therapeutic efficacy.
Simon, R. M. (1982) · British Journal of Clinical Pharmacology · doi:10.1111/j.1365-2125.1982.tb02015.x · Cited by 130 · see citing works
Evaluating black-boxes as medical decision aids: issues arising from a study of neural networks
Hart, A., & Wyatt, J. C. (1990) · Medical Informatics · doi:10.3109/14639239009025270 · Cited by 124 · see citing works
Ferrandina, G., Scambia, G., Bardelli, F., Panici, P. B., Mancuso, S., & Messori, A. (1997) · British Journal of Cancer · doi:10.1038/bjc.1997.442 · Cited by 120 · see citing works
Processed meats and risk of childhood leukemia (California, USA)
Peters, J. M., Preston‐Martin, S., London, S. J., Bowthan, J. D., Buckley, J. D., & Thomas, D. C. (1994) · Cancer Causes & Control · doi:10.1007/bf01830266 · Cited by 107 · see citing works
Knoop, A. S., Andreasen, P. A., Andersen, J. A., Hansen, S., Lænkholm, A., Simonsen, A. H., Andersen, J., Overgaard, J., & Rose, C. (1998) · British Journal of Cancer · doi:10.1038/bjc.1998.154 · Cited by 101 · see citing works
A comparison of prognostic covariates for uveal melanoma.
Gamel, J. W., McCurdy, J. B., & McLean, I. W. (1992) · PubMed · Cited by 92 · see citing works
Variation in baseline risk as an explanation of heterogeneity in meta-analysis
Walter, S. D. (1997) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19971230)16:24 · Cited by 80 · see citing works
Sample size: how many patients are necessary?
Fayers, P. M., & Machin, D. (1995) · British Journal of Cancer · doi:10.1038/bjc.1995.268 · Cited by 69 · see citing works
Molino, A., Micciolo, R., Turazza, M., Bonetti, F., Piubello, Q., Corgnati, A., Sperotto, L., Recaldin, E., Spagnolli, R., Manfrin, E., et al. (1997) · Breast Cancer Research and Treatment · doi:10.1023/a:1005769925670 · Cited by 66 · see citing works
Analysis of clinical trial outcomes: Some comments on subgroup analyses
Buyse, M. E. (1989) · Controlled Clinical Trials · doi:10.1016/0197-2456(89)90057-3 · Cited by 62 · see citing works
A comparison of Cox proportional hazards and artificial neural network models for medical prognosis
Ohno‐Machado, L. (1997) · Computers in Biology and Medicine · doi:10.1016/s0010-4825(96)00036-4 · Cited by 55 · see citing works
Buettner, P. G., Garbe, C., & Guggenmoos-Holzmann, I. (1997) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(97)00155-8 · Cited by 54 · see citing works
CAUSALITY INFERENCE IN OBSERVATIONAL VS. EXPERIMENTAL STUDIES
Gray‐Donald, K., & Kramer, M. S. (1988) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a114891 · Cited by 52 · see citing works
Suboptimal analysis using 'optimal' cutpoints
Altman, D. (1998) · British Journal of Cancer · doi:10.1038/bjc.1998.537 · Cited by 46 · see citing works
Self-citationEffect of Categorizing a Continuous Covariate on the Comparison of Survival Time
Morgan, T. M., & Elashoff, R. M. (1986) · Journal of the American Statistical Association · doi:10.1080/01621459.1986.10478350 · Cited by 42 · see citing works
Estrogen-replacement therapy after breast cancer risks versus benefits
Marchant, D. J. (1993) · Cancer · doi:10.1002/1097-0142(19930315)71:6+ · Cited by 36 · see citing works
Flexible covariate effects in the proportional hazards model
Hastie, T., Sleeper, L. A., & Tibshirani, R. J. (1992) · Breast Cancer Research and Treatment · doi:10.1007/bf01840837 · Cited by 35 · see citing works
Confirmation of a long-term prognostic index in breast cancer
Brown, J., Benson, E. A., & Jones, M. (1993) · The Breast · doi:10.1016/0960-9776(93)90057-m · Cited by 25 · see citing works
Prognostic value of total cathepsin B in invasive ductal carcinoma of the breast
Budihna, M., Skrk, J., Zakotnik, B., Gabrijelčič, D., & Lindtner, J. (1995) · European Journal of Cancer · doi:10.1016/0959-8049(94)00504-x · Cited by 15 · see citing works
Knorr, K. L., Hilsenbeck, S. G., Wenger, C. R., Pounds, G., Oldaker, T. A., Vendely, P., Pandian, M. R., Harrington, D. S., & Clark, G. M. (1992) · Breast Cancer Research and Treatment · doi:10.1007/bf01840838 · Cited by 11 · see citing works
Improved prognostication in small (pT1) breast cancers by image cytometry
Aubele, M. M., Auer, G., Falkmer, S., Voß, A., Rodenacker, K., Rutquist, L., & H�fler, H. (1995) · Breast Cancer Research and Treatment · doi:10.1007/bf00690188 · Cited by 10 · see citing works
Romero, H., & Schneider, J. (1995) · European Journal of Cancer · doi:10.1016/0959-8049(95)00116-6 · Cited by 6 · see citing works
Razors and Refrigerators and Reindeer — Oh My!
Jenks, S., & Volkers, N. (1992) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/84.24.1863 · Cited by 4 · see citing works
- Self-citation
Harrell, F., Jr. (1996) · doi:10.1002/(sici)1097-0258(19960229)15:4
Flagged first, then by citations
1 of the 63 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Reporting Recommendations for Tumor Marker Prognostic Studies (REMARK)
McShane, L. M., Altman, D. G., Sauerbrei, W., Taube, S. E., Gion, M., & Clark, G. M. (2005) · JNCI: Journal of the National Cancer Institute · doi:10.1093/jnci/dji237 · Cited by 2,162 · see citing works
Self-citationMoons, K. G. M., de Groot, J. A. H., Bouwmeester, W., Vergouwe, Y., Mallett, S., Altman, D. G., Reitsma, J. B., & Collins, G. S. (2014) · PLoS Medicine · doi:10.1371/journal.pmed.1001744 · Cited by 2,018 · see citing works
Self-citationCassidy, J. D., Carroll, L., Peloso, P. M. J., Borg, J., von Holst, H., Holm, L. W., Kraus, J. F., & Coronado, V. G. (2004) · Journal of Rehabilitation Medicine · doi:10.1080/16501960410023732 · Cited by 1,612 · see citing works
Evaluation of the Quality of Prognosis Studies in Systematic Reviews
Hayden, J. A., Côté, P., & Bombardier, C. (2006) · Annals of Internal Medicine · doi:10.7326/0003-4819-144-6-200603210-00010 · Cited by 1,452 · see citing works
REporting recommendations for tumour MARKer prognostic studies (REMARK).
for the Statistics Subcommittee of the NCI-EORTC Working Group on Cancer Diagnostics, McShane, L. M., Altman, D. G., Sauerbrei, W., Taube, S. E., Gion, M., & Clark, G. M. (2005) · British Journal of Cancer · doi:10.1038/sj.bjc.6602678 · Cited by 1,394 · see citing works
Self-citationCarroll, L., Cassidy, J. D., Peloso, P., Borg, J., von Holst, H., Holm, L., Paniak, C., & Pépin, M. (2004) · Journal of Rehabilitation Medicine · doi:10.1080/16501960410023859 · Cited by 1,326 · see citing works
Reporting Recommendations for Tumor Marker Prognostic Studies (REMARK): Explanation and Elaboration
Altman, D. G., McShane, L. M., Sauerbrei, W., & Taube, S. E. (2012) · PLoS Medicine · doi:10.1371/journal.pmed.1001216 · Cited by 1,019 · see citing works
Self-citationPanel, N. I. O. H. C. D. (2001) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/93.13.979 · Cited by 892 · see citing works
The Burden and Determinants of Neck Pain in the General Population
Hogg‐Johnson, S., van der Velde, G., Carroll, L., Holm, L. W., Cassidy, J. D., Guzmán, J., Côté, P., Haldeman, S., Ammendolia, C., Carragee, E. J., et al. (2008) · Spine · doi:10.1097/brs.0b013e31816454c8 · Cited by 847 · see citing works
Prognosis Research Strategy (PROGRESS) 2: Prognostic Factor Research
Riley, R. D., Hayden, J. A., Steyerberg, E. W., Moons, K. G. M., Abrams, K., Kyzas, P. A., Malats, N., Briggs, A., Schroter, S., Altman, D. G., et al. (2013) · PLoS Medicine · doi:10.1371/journal.pmed.1001380 · Cited by 823 · see citing works
Self-citationREporting recommendations for tumor MARKer prognostic studies (REMARK)
McShane, L. M., Altman, D. G., Sauerbrei, W., Taube, S. E., Gion, M., & Clark, G. M. (2006) · Breast Cancer Research and Treatment · doi:10.1007/s10549-006-9242-8 · Cited by 810 · see citing works
Self-citationReporting Recommendations for Tumor Marker Prognostic Studies
McShane, L. M., Altman, D. G., Sauerbrei, W., Taube, S. E., Gion, M., & Clark, G. M. (2005) · Journal of Clinical Oncology · doi:10.1200/jco.2004.01.0454 · Cited by 780 · see citing works
Self-citationSystematic reviews of evaluations of prognostic variables
Altman, D. G. (2001) · BMJ · doi:10.1136/bmj.323.7306.224 · Cited by 751 · see citing works
Self-citationMaltoni, M. C., Caraceni, A. T., Brunelli, C., Broeckaert, B., Christakis, N. A., Eychmueller, S., Glare, P. A., Nabal, M., Vigano, A. A. L., Larkin, P. J., et al. (2005) · Journal of Clinical Oncology · doi:10.1200/jco.2005.06.866 · Cited by 666 · see citing works
Course and Prognostic Factors for Neck Pain in Whiplash-Associated Disorders (WAD)
Carroll, L., Holm, L. W., Hogg‐Johnson, S., Côté, P., Cassidy, J. D., Haldeman, S., Nordin, M. C., Hurwitz, E. L., Carragee, E. J., van der Velde, G., et al. (2008) · Spine · doi:10.1097/brs.0b013e3181643eb8 · Cited by 645 · see citing works
Reporting and Methods in Clinical Prediction Research: A Systematic Review
Bouwmeester, W., Zuithoff, N. P. A., Mallett, S., Geerlings, M. I., Vergouwe, Y., Steyerberg, E. W., Altman, D. G., & Moons, K. G. M. (2012) · PLoS Medicine · doi:10.1371/journal.pmed.1001221 · Cited by 600 · see citing works
Self-citationvon Dadelszen, P., Payne, B. A., Li, J., Ansermino, J. M., Pipkin, F. B., Côté, A., Douglas, M. J., Gruslin, A., Hutcheon, J. A., Joseph, K. S., et al. (2010) · The Lancet · doi:10.1016/s0140-6736(10)61351-7 · Cited by 585 · see citing works
REporting recommendations for tumor MARKer prognostic studies (REMARK)
McShane, L. M., Altman, D. G., Sauerbrei, W., Taube, S. E., Gion, M., & Clark, G. M. (2005) · Nature Clinical Practice Oncology · doi:10.1038/ncponc0252 · Cited by 562 · see citing works
Self-citationHuguet, A., Hayden, J. A., Stinson, J., McGrath, P. J., Chambers, C. T., Tougas, M. E., & Wozney, L. (2013) · Systematic Reviews · doi:10.1186/2046-4053-2-71 · Cited by 531 · see citing works
The Burden and Determinants of Neck Pain in Workers
Côté, P., van der Velde, G., Cassidy, J. D., Carroll, L., Hogg‐Johnson, S., Holm, L. W., Carragee, E. J., Haldeman, S., Nordin, M. C., Hurwitz, E. L., et al. (2008) · Spine · doi:10.1097/brs.0b013e3181643ee4 · Cited by 528 · see citing works
Côté, P., Cassidy, J. D., Carroll, L., Frank, J. W., & Bombardier, C. (2001) · Spine · doi:10.1097/00007632-200110010-00020 · Cited by 390 · see citing works
Reporting recommendations for tumor marker prognostic studies (REMARK): explanation and elaboration
Altman, D. G., McShane, L. M., Sauerbrei, W., & Taube, S. E. (2012) · BMC Medicine · doi:10.1186/1741-7015-10-51 · Cited by 374 · see citing works
Self-citationDepression as a risk factor for onset of an episode of troublesome neck and low back pain
Carroll, L. J., Cassidy, D. J., & Côté, P. (2004) · Pain · doi:10.1016/j.pain.2003.10.009 · Cited by 373 · see citing works
Course and Prognostic Factors for Neck Pain in the General Population
Carroll, L., Hogg‐Johnson, S., van der Velde, G., Haldeman, S., Holm, L. W., Carragee, E. J., Hurwitz, E. L., Côté, P., Nordin, M. C., Peloso, P. M. J., et al. (2008) · Spine · doi:10.1097/brs.0b013e31816445be · Cited by 338 · see citing works
Predicting functional outcome and survival after acute ischemic stroke
Weimar, C., Ziegler, A., König, I. R., & Diener, H.-C. (2002) · Journal of Neurology · doi:10.1007/s00415-002-0755-8 · Cited by 320 · see citing works
REporting recommendations for tumour MARKer prognostic studies (REMARK)
McShane, L. M., Altman, D. G., Sauerbrei, W., Taube, S. E., Gion, M., & Clark, G. M. (2005) · European Journal of Cancer · doi:10.1016/j.ejca.2005.03.032 · Cited by 312 · see citing works
Self-citationMusculoskeletal Injuries and Pain in Dancers: A Systematic Review
Hincapié, C. A., Morton, E. J., & Cassidy, J. D. (2008) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2008.02.020 · Cited by 305 · see citing works
P53 abnormalities and outcomes in colorectal cancer: a systematic review
Munro, A. J., Laı́n, S., & Lane, D. P. (2005) · British Journal of Cancer · doi:10.1038/sj.bjc.6602358 · Cited by 287 · see citing works
Hypothalamic-pituitary-adrenal axis dysfunction in chronic fatigue syndrome
Papadopoulos, A. S., & Cleare, A. J. (2012) · Nature Reviews Endocrinology · doi:10.1038/nrendo.2011.153 · Cited by 280 · see citing works
The Burden and Determinants of Neck Pain in Workers
Côté, P., van der Velde, G., Cassidy, J. D., Carroll, L., Hogg‐Johnson, S., Holm, L. W., Carragee, E. J., Haldeman, S., Nordin, M. C., Hurwitz, E. L., et al. (2009) · Journal of Manipulative and Physiological Therapeutics · doi:10.1016/j.jmpt.2008.11.012 · Cited by 279 · see citing works
Assessment of Claims of Improved Prediction Beyond the Framingham Risk Score
Tzoulaki, I. (2009) · JAMA · doi:10.1001/jama.2009.1757 · Cited by 276 · see citing works
Lu, W., Jansen, L., Post, W. J., Bonnema, J., de Velde, J. C. V., & de Bock, G. H. (2008) · Breast Cancer Research and Treatment · doi:10.1007/s10549-008-0023-4 · Cited by 261 · see citing works
Predicting Long-Term Outcome After Acute Ischemic Stroke
König, I. R., Ziegler, A., Bluhmki, E., Hacke, W., Bath, P. M., Sacco, R. L., Diener, H., & Weimar, C. (2008) · Stroke · doi:10.1161/strokeaha.107.505867 · Cited by 258 · see citing works
The Burden and Determinants of Neck Pain in Whiplash-Associated Disorders After Traffic Collisions
Holm, L. W., Carroll, L., Cassidy, J. D., Hogg‐Johnson, S., Côté, P., Guzmán, J., Peloso, P. M. J., Nordin, M. C., Hurwitz, E. L., van der Velde, G., et al. (2008) · Spine · doi:10.1097/brs.0b013e3181643ece · Cited by 248 · see citing works
Course and Prognostic Factors for Neck Pain in Whiplash-Associated Disorders (WAD)
Carroll, L., Holm, L. W., Hogg‐Johnson, S., Côté, P., Cassidy, J. D., Haldeman, S., Nordin, M. C., Hurwitz, E. L., Carragee, E. J., van der Velde, G., et al. (2009) · Journal of Manipulative and Physiological Therapeutics · doi:10.1016/j.jmpt.2008.11.014 · Cited by 245 · see citing works
Hayden, J. A., Chou, R., Hogg‐Johnson, S., & Bombardier, C. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.09.004 · Cited by 243 · see citing works
Sutcliffe, P., Hummel, S., Simpson, E., Young, T., Rees, A., Wilkinson, A., Hamdy, F. C., Clarke, N. W., & Staffurth, J. (2009) · Health Technology Assessment · doi:10.3310/hta13050 · Cited by 237 · see citing works
The Burden and Determinants of Neck Pain in the General Population
Hogg‐Johnson, S., van der Velde, G., Carroll, L., Holm, L. W., Cassidy, J. D., Guzmán, J., Côté, P., Haldeman, S., Ammendolia, C., Carragee, E. J., et al. (2009) · Journal of Manipulative and Physiological Therapeutics · doi:10.1016/j.jmpt.2008.11.010 · Cited by 237 · see citing works
Almost all articles on cancer prognostic markers report statistically significant results
Kyzas, P., Denaxa-Kyza, D., & Ioannidis, J. P. A. (2007) · European Journal of Cancer · doi:10.1016/j.ejca.2007.08.030 · Cited by 231 · see citing works
Selective Reporting Biases in Cancer Prognostic Factor Studies
Kyzas, P., Loizou, K. T., & Ioannidis, J. P. A. (2005) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/dji184 · Cited by 231 · see citing works
Course and Prognostic Factors for Neck Pain in Workers
Carroll, L., Hogg‐Johnson, S., Côté, P., van der Velde, G., Holm, L. W., Carragee, E. J., Hurwitz, E. L., Peloso, P. M. J., Cassidy, J. D., Guzmán, J., et al. (2008) · Spine · doi:10.1097/brs.0b013e31816445d4 · Cited by 221 · see citing works
Burton, A., & Altman, D. G. (2004) · British Journal of Cancer · doi:10.1038/sj.bjc.6601907 · Cited by 219 · see citing works
Self-citationPrognostic Factors in Breast Cancer: Current and New Predictors of Metastasis
Hayes, D. F., Isaacs, C. J., & Stearns, V. (2001) · Journal of Mammary Gland Biology and Neoplasia · doi:10.1023/a:1014778713034 · Cited by 207 · see citing works
A Systematic Review of Molecular and Biological Tumor Markers in Neuroblastoma
Riley, R. D., Heney, D., Jones, D. R., Sutton, A. J., Lambert, P. C., Abrams, K. R., Young, B., Wailoo, A., & Burchill, S. A. (2004) · Clinical Cancer Research · doi:10.1158/1078-0432.ccr-1051-2 · Cited by 201 · see citing works
Course and Prognostic Factors for Neck Pain in the General Population
Carroll, L., Hogg‐Johnson, S., van der Velde, G., Haldeman, S., Holm, L. W., Carragee, E. J., Hurwitz, E. L., Côté, P., Nordin, M. C., Peloso, P. M. J., et al. (2009) · Journal of Manipulative and Physiological Therapeutics · doi:10.1016/j.jmpt.2008.11.013 · Cited by 190 · see citing works
Crawford, F., Cezard, G. I., Chappell, F. M., Murray, G. D., Price, J. F., Sheikh, A., Simpson, C. R., Stansby, G. P., & Young, M. J. (2015) · Health Technology Assessment · doi:10.3310/hta19570 · Cited by 188 · see citing works
Can we predict poor recovery from recent-onset nonspecific low back pain? A systematic review
Kent, P. M., & Keating, J. L. (2007) · Manual Therapy · doi:10.1016/j.math.2007.05.009 · Cited by 185 · see citing works
Publication of Tumor Marker Research Results: The Necessity for Complete and Transparent Reporting
McShane, L. M., & Hayes, D. F. (2012) · Journal of Clinical Oncology · doi:10.1200/jco.2012.42.6858 · Cited by 185 · see citing works
Wong, J. J., Côté, P., Quesnele, J. J., Stern, P. J., & Mior, S. A. (2014) · The Spine Journal · doi:10.1016/j.spinee.2014.02.032 · Cited by 181 · see citing works
Impact of a 21‐gene RT‐PCR assay on treatment decisions in early‐stage breast cancer
Lyman, G. H., Cosler, L. E., Kuderer, N. M., & Hornberger, J. C. (2007) · Cancer · doi:10.1002/cncr.22506 · Cited by 179 · see citing works
Self-citationThe Burden and Determinants of Neck Pain in the General Population
Hogg‐Johnson, S., van der Velde, G., Carroll, L., Holm, L. W., Cassidy, J. D., Guzman, J., Côté, P., Haldeman, S., Ammendolia, C., Carragee, E. J., et al. (2008) · European Spine Journal · doi:10.1007/s00586-008-0624-y · Cited by 172 · see citing works
The Burden and Determinants of Neck Pain in Workers
Côté, P., van der Velde, G., Cassidy, J. D., Carroll, L., Hogg‐Johnson, S., Holm, L. W., Carragee, E. J., Haldeman, S., Nordin, M. C., Hurwitz, E. L., et al. (2008) · European Spine Journal · doi:10.1007/s00586-008-0626-9 · Cited by 168 · see citing works
Riley, R. D., Abrams, K. R., Sutton, A. J., Lambert, P. C., Jones, D. R., Heney, D., & Burchill, S. A. (2003) · British Journal of Cancer · doi:10.1038/sj.bjc.6600886 · Cited by 165 · see citing works
Riley, R. D., Burchill, S. A., Abrams, K. R., Heney, D., Lambert, P. C., Jones, D. R., Sutton, A. J., Young, B., Wailoo, A., & Lewis, I. J. (2003) · Health Technology Assessment · doi:10.3310/hta7050 · Cited by 163 · see citing works
Riley, R. D., Sauerbrei, W., & Altman, D. G. (2009) · British Journal of Cancer · doi:10.1038/sj.bjc.6604999 · Cited by 161 · see citing works
Self-citationMinichromosome Maintenance Protein 2 Is a Strong Independent Prognostic Marker in Breast Cancer
Gonzalez, M., Pinder, S. E., Callagy, G. M., Vowler, S. L., Morris, L. S., Bird, K., Bell, J. A., Laskey, R. A., & Coleman, N. (2003) · Journal of Clinical Oncology · doi:10.1200/jco.2003.04.121 · Cited by 158 · see citing works
Edén, P., Cecilia Ritz, B., Rose, C., Fernö, M., & Peterson, C. (2004) · European Journal of Cancer · doi:10.1016/j.ejca.2004.02.025 · Cited by 154 · see citing works
van der Hulst, M., Vollenbroek-Hutten, M. M. R., & IJzerman, M. J. (2005) · Spine · doi:10.1097/01.brs.0000157414.47713.78 · Cited by 154 · see citing works
Panel, N. I. O. H. C. D. (2001) · JNCI Monographs · doi:10.1093/oxfordjournals.jncimonographs.a003460 · Cited by 153 · see citing works
The Burden and Determinants of Neck Pain in Whiplash-Associated Disorders After Traffic Collisions
Holm, L. W., Carroll, L., Cassidy, J. D., Hogg‐Johnson, S., Côté, P., Guzmán, J., Peloso, P. M. J., Nordin, M. C., Hurwitz, E. L., van der Velde, G., et al. (2009) · Journal of Manipulative and Physiological Therapeutics · doi:10.1016/j.jmpt.2008.11.011 · Cited by 145 · see citing works
Hayden, J. A., Côté, P., Steenstra, I. A., & Bombardier, C. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.08.005 · Cited by 138 · see citing works
Practical experiences on the necessity of external validation
König, I. R., Malley, J. D., Weimar, C., Diener, H., & Ziegler, A. (2007) · Statistics in Medicine · doi:10.1002/sim.3069 · Cited by 136 · see citing works
Modelling prognostic factors in advanced pancreatic cancer
Stocken, D. D., Hassan, A. B., Altman, D. G., Billingham, L. J., Bramhall, S., Johnson, P. J., & Freemantle, N. (2008) · British Journal of Cancer · doi:10.1038/sj.bjc.6604568 · Cited by 134 · see citing works
Self-citationMontgomery, D. A., Krupa, K., & Cooke, T. G. (2007) · British Journal of Cancer · doi:10.1038/sj.bjc.6604065 · Cited by 133 · see citing works
de Bock, G. H., Bonnema, J., van der Hage, J. A., Kievit, J., & van de Velde, C. J. H. (2004) · Journal of Clinical Oncology · doi:10.1200/jco.2004.06.080 · Cited by 132 · see citing works
Prognostic Models: A Methodological Framework and Review of Models for Breast Cancer
Altman, D. G. (2009) · Cancer Investigation · doi:10.1080/07357900802572110 · Cited by 129 · see citing works
Self-citationMallett, S., Timmer, A., Sauerbrei, W., & Altman, D. G. (2010) · British Journal of Cancer · doi:10.1038/sj.bjc.6605462 · Cited by 122 · see citing works
Self-citationTak, L. M., Riese, H., de Bock, G. H., Andiappan, M., Kok, I. C., & Rosmalen, J. G. M. (2009) · Biological Psychology · doi:10.1016/j.biopsycho.2009.05.002 · Cited by 122 · see citing works
Pinto, R. Z., Ferreira, P. H., Kongsted, A., Ferreira, M. L., Maher, C. G., & Kent, P. M. (2014) · European Journal of Pain · doi:10.1002/j.1532-2149.2014.00468.x · Cited by 121 · see citing works
Clinical Features in Early Parkinson Disease and Survival
Lo, R. Y., Tanner, C. M., Albers, K. B., Leimpeter, A. D., Fross, R. D., BERNSTEIN, A. L., McGuire, V. M., Quesenberry, C. P., Nelson, L. M., & Eeden, S. K. V. D. (2009) · Archives of Neurology · doi:10.1001/archneurol.2009.221 · Cited by 122 · see citing works
Systematic Reviews of Evaluations of Prognostic Variables
Altman, D. G. (2001) · Wiley eBooks · doi:10.1002/9780470693926.ch13 · Cited by 123 · see citing works
Self-citationAyari, C., LaRue, H., Hovington, H., Decobert, M., Harel, F., Bergeron, A., Têtu, B., Lacombe, L., & Fradet, Y. (2009) · European Urology · doi:10.1016/j.eururo.2009.01.040 · Cited by 119 · see citing works
Prognostic and Predictive Factors
Scott, M. T., & Hall, P. A. (2004) · Humana Press eBooks · doi:10.1385/1-59259-760-2:001 · Cited by 117 · see citing works
Filipits, M., Pirker, R., Dunant, A., Lantuéjoul, S., Schmid, K., Huynh, A. T., Haddad, V., André, F., Stahel, R. A., Pignon, J., et al. (2007) · Journal of Clinical Oncology · doi:10.1200/jco.2006.08.2867 · Cited by 116 · see citing works
Evaluating a New Marker’s Predictive Contribution
Kattan, M. W. (2004) · Clinical Cancer Research · doi:10.1158/1078-0432.ccr-03-0061 · Cited by 114 · see citing works
Span, P. N., Bussink, J., Manders, P., Beex, L. V. A. M., & Sweep, F. C. G. J. (2003) · British Journal of Cancer · doi:10.1038/sj.bjc.6601122 · Cited by 109 · see citing works
Gross, D. P., Battié, M. C., & Cassidy, J. D. (2004) · Spine · doi:10.1097/00007632-200404150-00019 · Cited by 109 · see citing works
Hartmann, O., Spyratos, F., Harbeck, N., Dietrich, D., Faßbender, A., Schmitt, M., Eppenberger‐Castori, S., Vuaroqueaux, V., Lerebours, F., Welzel, K., et al. (2008) · Clinical Cancer Research · doi:10.1158/1078-0432.ccr-08-0166 · Cited by 106 · see citing works
Oratz, R., Paul, D. P., Cohn, A. L., & SEDLACEK, S. M. (2007) · Journal of Oncology Practice · doi:10.1200/jop.0742001 · Cited by 105 · see citing works
Linton, S. J., Gross, D. P., Schultz, I. Z., Main, C. J., Côté, P., Pransky, G., & Johnson, W. G. (2005) · Journal of Occupational Rehabilitation · doi:10.1007/s10926-005-8028-x · Cited by 104 · see citing works
Defining Clinical Utility of Tumor Biomarker Tests: A Clinician's Viewpoint
Hayes, D. F. (2020) · Journal of Clinical Oncology · doi:10.1200/jco.20.01572 · Cited by 102 · see citing works
Linderoth, J., Jerkeman, M., Cavallin-Ståhl, E. K., Kvaløy, S., & Torlakovic, E. (2003) · PubMed · Cited by 101 · see citing works
A new approach to categorising continuous variables in prediction models: Proposal and validation
Barrio, I., Aróstegui, I., Rodríguez‐Álvarez, M. X., & Quintana, J. M. (2015) · Statistical Methods in Medical Research · doi:10.1177/0962280215601873 · Cited by 99 · see citing works
Clinical trial design for microarray predictive marker discovery and assessment
Pusztai, L., & Hess, K. R. (2004) · Annals of Oncology · doi:10.1093/annonc/mdh466 · Cited by 96 · see citing works
Rodins, K., Cheale, M., Coleman, N., & Fox, S. B. (2002) · PubMed · Cited by 95 · see citing works
Kyzas, P., Denaxa-Kyza, D., & Ioannidis, J. P. A. (2007) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/djk032 · Cited by 94 · see citing works
Cyclin-E is a strong predictor of endocrine therapy failure in human breast cancer
Span, P. N., Tjan‐Heijnen, V. C. G., Manders, P., Beex, L. V. A. M., & Sweep, F. C. G. J. (2003) · Oncogene · doi:10.1038/sj.onc.1206818 · Cited by 92 · see citing works
The Importance of Identifying and Validating Prognostic Factors in Oncology
Halabi, S., & Owzar, K. (2010) · Seminars in Oncology · doi:10.1053/j.seminoncol.2010.04.001 · Cited by 91 · see citing works
Prediction Tools for Unfavourable Outcomes in Clostridium difficile Infection: A Systematic Review
Chakra, C. N. A., Pépin, J., & Valiquette, L. (2012) · PLoS ONE · doi:10.1371/journal.pone.0030258 · Cited by 86 · see citing works
Low Bone Density in Systemic Sclerosis. A Systematic Review
Omair, M. A., Pagnoux, C., McDonald-Blumer, H., & Johnson, S. R. (2013) · The Journal of Rheumatology · doi:10.3899/jrheum.130032 · Cited by 85 · see citing works
Overgaard, J. (2000) · Acta Oncologica · doi:10.1080/028418600750013096 · Cited by 85 · see citing works
Brown, J. M., Thorpe, H. C., Napp, V., Fairlamb, D. J., Gower, N. H., Milroy, R., Parmar, M., Rudd, R., Spiro, S. G., Stephens, R. J., et al. (2005) · Journal of Clinical Oncology · doi:10.1200/jco.2005.09.158 · Cited by 84 · see citing works
Trivella, M., Pezzella, F., Pastorino, U., Harris, A. L., & Altman, D. G. (2007) · The Lancet Oncology · doi:10.1016/s1470-2045(07)70145-6 · Cited by 83 · see citing works
Self-citationAlternative methods of follow up in breast cancer: a systematic review of the literature
Montgomery, D. A., Krupa, K., & Cooke, T. G. (2007) · British Journal of Cancer · doi:10.1038/sj.bjc.6603771 · Cited by 82 · see citing works
Filipits, M., Haddad, V., Schmid, K., Huynh, A. T., Dunant, A., André, F., Brambilla, E. M., Stahel, R. A., Pignon, J., Soria, J., et al. (2007) · Clinical Cancer Research · doi:10.1158/1078-0432.ccr-06-2446 · Cited by 81 · see citing works
Course and Prognostic Factors for Neck Pain in Whiplash-Associated Disorders (WAD)
Carroll, L., Holm, L. W., Hogg‐Johnson, S., Côté, P., Cassidy, J. D., Haldeman, S., Nordin, M. C., Hurwitz, E. L., Carragee, E. J., van der Velde, G., et al. (2008) · European Spine Journal · doi:10.1007/s00586-008-0628-7 · Cited by 79 · see citing works
What is the clinical value of cancer stem cell markers in gliomas?
Dahlrot, R. H., Hermansen, S. K., Hansen, S., & Kristensen, B. W. (2013) · Europe PMC (PubMed Central) · Cited by 78 · see citing works
Predicting the future: a critical appraisal of cancer prognosis studies
Hall, ., & Going, J. J. (1999) · Histopathology · doi:10.1046/j.1365-2559.1999.00862.x · Cited by 77 · see citing works
Primer: an evidence-based approach to prognostic markers
Altman, D. G., & Riley, R. D. (2005) · Nature Clinical Practice Oncology · doi:10.1038/ncponc0287 · Cited by 76 · see citing works
Self-citationPhases of methodological research in biostatistics—Building the evidence base for new methods
Heinze, G., Boulesteix, A., Kammer, M., Morris, T. P., White, I. R., & initiative, T. S. P. O. T. S. (2023) · Biometrical Journal · doi:10.1002/bimj.202200222 · Cited by 75 · see citing works
- Authors
- Altman, D. G., & Lyman, G. H.
- Affiliations
- University of Oxford (GB)
- University of South Florida (US)
- Venue
- Breast Cancer Research and Treatment
- Published
- 1998-11
- Volume
- 52
- Issue
- 1-3
- Pages
- 289-303
- ISSN
- 0167-6806 (ISSN-L)
- 1573-7217
- OpenAlex
- W2092931112
- Cited by
- 271
- References
- 63
- Citations by year
- 2026: 4 · 2025: 4 · 2024: 7 · 2023: 6 · 2022: 6 · 2021: 6 · 2020: 9 · 2019: 2 · 2018: 5 · 2017: 4 · 2016: 2 · 2015: 4 · 2014: 9 · 2013: 11 · 2012: 17
- FWCI
- 0.47
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- Breast Cancer Research and Treatment (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Cancer Research › Breast Cancer Treatment Studies
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- breast cancer, prognostic factors, research design, statistical analysis, research quality, methodological issues, research guidelines
- MeSH terms
- Breast Neoplasms, Female, Humans, Prognosis, Regression Analysis, Research Design, Meta-Analysis as Topic, Sample Size
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review