The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.
Assessing Bias in Studies of Prognostic Factors
Abstract
Previous work has identified 6 important areas to consider when evaluating validity and bias in studies of prognostic factors: participation, attrition, prognostic factor measurement, confounding measurement and account, outcome measurement, and analysis and reporting. This article describes the Quality In Prognosis Studies tool, which includes questions related to these areas that can inform judgments of risk of bias in prognostic research.A working group comprising epidemiologists, statisticians, and clinicians developed the tool as they considered prognosis studies of low back pain. Forty-three groups reviewing studies addressing prognosis in other topic areas used the tool and provided feedback. Most reviewers (74%) reported that reaching consensus on judgments was easy. Median completion time per study was 20 minutes; interrater agreement (κ statistic) reported by 9 review teams varied from 0.56 to 0.82 (median, 0.75). Some reviewers reported challenges making judgments across prompting items, which were addressed by providing comprehensive guidance and examples. The refined Quality In Prognosis Studies tool may be useful to assess the risk of bias in studies of prognostic factors.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Currently there is little information available to guide the development of a well-defined review question however the Quality in Prognosis Studies (QUIPS) tool [34] and the Checklist for critical appraisal and data extraction for systematic reviews of prediction modelling studies (CHARMS Checklist) [38] have been developed to assist in this process (see Table 1).”
From What kind of systematic review should I conduct? A proposed typology and guidance for systematic reviewers in the medical and health sciences (2018) · cites this paper as [34]
In the citing paper
“Prognostic research is of high value as it provides clinicians and patients with information regarding the course of a disease and potential outcomes, in addition to potentially providing useful information to deliver targeted therapy relating to specific prognostic factors [20, 34, 35].”
From What kind of systematic review should I conduct? A proposed typology and guidance for systematic reviewers in the medical and health sciences (2018) · cites this paper as [34]
Citations
Cites
The Cochrane Collaboration's tool for assessing risk of bias in randomised trials
Higgins, J. P. T., Altman, D. G., Gotzsche, P. C., Juni, P., Moher, D., Oxman, A. D., Savovic, J., Schulz, K. F., Weeks, L., Sterne, J. A. C., et al. (2011) · BMJ · doi:10.1136/bmj.d5928 · Cited by 33,817
QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies
Whiting, P. F., Rutjes, A. W. S., Westwood, M. E., Mallett, S., Deeks, J. J., Reitsma, J. B., Leeflang, M. M. G., Sterne, J. A. C., Bossuyt, P. M. M., & the QUADAS-2 Group* (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-155-8-201110180-00009 · Cited by 14,199
Research works printed but not linked: 30
- 10.7326/0003-4819-144-6-200603210-00010 · DOI · Paper page
- 10.1136/bmj.b4184 · DOI
- 10.1016/j.jclinepi.2008.09.004 · DOI
- 10.1038/sj.bjc.6604999 · DOI
- 10.1097/BRS.0b013e3182388202 · DOI
- 10.1371/journal.pmed.1000286 · DOI
- 10.1016/j.jclinepi.2007.08.005 · DOI
- 10.1136/bmj.311.7001.376 · DOI · Paper page
- 10.1002/acr.20492 · DOI
- 10.1016/S0140-6736(08)60691-1 · DOI
- 10.1161/STROKEAHA.110.585554 · DOI
- 10.1001/jama.2011.308 · DOI · Paper page
- 10.1038/ki.2008.29 · DOI
- 10.1111/j.1467-789X.2008.00475.x · DOI
- 10.1016/j.resuscitation.2008.10.034 · DOI
- 10.5271/sjweh.3143 · DOI · Paper page
- 10.1016/j.injury.2010.04.005 · DOI
- 10.1016/j.jpsychores.2007.11.001 · DOI
- 10.1016/j.amepre.2010.10.015 · DOI · Paper page
- 10.1542/peds.2010-0941 · DOI
- 10.1038/ijo.2009.121 · DOI
- 10.1378/chest.09-0608 · DOI
- 10.1002/art.24641 · DOI
- 10.1186/1471-2296-12-33 · DOI
- 10.1097/MPG.0b013e3181e82a28 · DOI
- 10.1016/j.resuscitation.2011.01.028 · DOI
- Johnson SR, 2008, J Rheumatol, 35, 1584
- 10.1007/s10549-006-9242-8 · DOI
- 10.1016/j.jclinepi.2011.10.017 · DOI
- 10.1093/ije/dym018 · DOI · Paper page
Cited by
Sorted by citations
Dementia prevention, intervention, and care
Livingston, G., Sommerlad, A., Orgeta, V., Costafreda, S. G., Huntley, J., Ames, D., Ballard, C., Banerjee, S., Burns, A., Cohen-Mansfield, J., et al. (2017) · The Lancet · doi:10.1016/s0140-6736(17)31363-6 · Cited by 6,072
PROBAST: A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies
Wolff, R. F., Moons, K. G. M., Riley, R. D., Whiting, P. F., Westwood, M., Collins, G. S., Reitsma, J. B., Kleijnen, J., Mallett, S., & for the PROBAST Group† (2019) · Annals of Internal Medicine · doi:10.7326/m18-1376 · Cited by 2,984
Moons, K. G. M., de Groot, J. A. H., Bouwmeester, W., Vergouwe, Y., Mallett, S., Altman, D. G., Reitsma, J. B., & Collins, G. S. (2014) · PLoS Medicine · doi:10.1371/journal.pmed.1001744 · Cited by 2,018
Moons, K. G. M., Wolff, R. F., Riley, R. D., Whiting, P. F., Westwood, M., Collins, G. S., Reitsma, J. B., Kleijnen, J., & Mallett, S. (2019) · Annals of Internal Medicine · doi:10.7326/m18-1377 · Cited by 1,669
Ma, L.-L., Wang, Y.-Y., Yang, Z.-H., Huang, D., Weng, H., & Zeng, X.-T. (2020) · Military Medical Research · doi:10.1186/s40779-020-00238-8 · Cited by 1,648
Insomnia as a predictor of mental disorders: a systematic review and meta-analysis
Hertenstein, E., Feige, B., Gmeiner, T., Kienzler, C., Spiegelhalder, K., Johann, A., Jansson-Fröjmark, M., Palagini, L., Rücker, G., Riemann, D., et al. (2019) · Sleep Medicine Reviews · doi:10.1016/j.smrv.2018.10.006 · Cited by 1,201
Munn, Z., Stern, C., Aromataris, E., Lockwood, C., & Jordan, Z. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-017-0468-4 · Cited by 1,164
Blood pressure variability and cardiovascular disease: systematic review and meta-analysis
Stevens, S. L., Wood, S., Koshiaris, C., Law, K., Glasziou, P., Stevens, R. J., & McManus, R. J. (2016) · BMJ · doi:10.1136/bmj.i4098 · Cited by 871
Tinanoff, N., Baez, R. J., Diaz Guillory, C., Donly, K. J., Feldens, C. A., McGrath, C., Phantumvanit, P., Pitts, N. B., Seow, W. K., Sharkov, N., et al. (2019) · International Journal of Paediatric Dentistry · doi:10.1111/ipd.12484 · Cited by 728
Cupp, M. A., Cariolou, M., Tzoulaki, I., Aune, D., Evangelou, E., & Berlanga-Taylor, A. J. (2020) · BMC Medicine · doi:10.1186/s12916-020-01817-1 · Cited by 697
van Prehn, J., Reigadas, E., Vogelzang, E. H., Bouza, E., Hristea, A., Guery, B., Krutova, M., Norén, T., Allerberger, F., Coia, J. E., et al. (2021) · Clinical Microbiology and Infection · doi:10.1016/j.cmi.2021.09.038 · Cited by 640
Sheldon, E., Simmonds-Buckley, M., Bone, C., Mascarenhas, T., Chan, N., Wincott, M., Gleeson, H., Sow, K., Hind, D., & Barkham, M. (2021) · Journal of Affective Disorders · doi:10.1016/j.jad.2021.03.054 · Cited by 644
Daskalopoulou, C., Stubbs, B., Kralj, C., Koukounari, A., Prince, M., & Prina, A. M. (2017) · Ageing Research Reviews · doi:10.1016/j.arr.2017.06.003 · Cited by 501
Foroutan, F., Guyatt, G., Zuk, V., Vandvik, P. O., Alba, A. C., Mustafa, R., Vernooij, R., Arevalo-Rodriguez, I., Munn, Z., Roshanov, P., et al. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.12.023 · Cited by 465
Mansfield, K. E., Sim, J., Jordan, J. L., & Jordan, K. P. (2016) · Pain · doi:10.1097/j.pain.0000000000000314 · Cited by 402
Rabbitts, J. A., Fisher, E., Rosenbloom, B. N., & Palermo, T. M. (2017) · The Journal of Pain · doi:10.1016/j.jpain.2017.03.007 · Cited by 317
Factors Associated With Post-Stroke Physical Activity: A Systematic Review and Meta-Analysis
Thilarajah, S., Mentiplay, B. F., Bower, K. J., Tan, D., Pua, Y. H., Williams, G., Koh, G., & Clark, R. A. (2018) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2017.09.117 · Cited by 310
Page, M. J., McKenzie, J. E., & Higgins, J. P. T. (2018) · BMJ Open · doi:10.1136/bmjopen-2017-019703 · Cited by 278
Mintjens, S., Menting, M. D., Daams, J. G., van Poppel, M. N. M., Roseboom, T. J., & Gemke, R. J. B. J. (2018) · Sports Medicine · doi:10.1007/s40279-018-0974-5 · Cited by 278
Cavero-Redondo, I., Peleteiro, B., Álvarez-Bueno, C., Rodriguez-Artalejo, F., & Martínez-Vizcaíno, V. (2017) · BMJ Open · doi:10.1136/bmjopen-2017-015949 · Cited by 260
Modifiable lifestyle factors and cognitive reserve: a systematic review of current evidence
Song, S., Stern, Y., & Gu, Y. (2022) · Ageing Research Reviews · doi:10.1016/j.arr.2021.101551 · Cited by 248
Canevelli, M., Grande, G., Lacorte, E., Quarchioni, E., Cesari, M., Mariani, C., Bruno, G., & Vanacore, N. (2016) · Journal of the American Medical Directors Association · doi:10.1016/j.jamda.2016.06.020 · Cited by 226
Notarte, K. I., de Oliveira, M. H. S., Peligro, P. J., Velasco, J. V., Macaranas, I., Ver, A. T., Pangilinan, F. C., Pastrana, A., Goldrich, N., Kavteladze, D., et al. (2022) · Journal of Clinical Medicine · doi:10.3390/jcm11247314 · Cited by 209
Giusti, E. M., Lacerenza, M., Manzoni, G. M., & Castelnuovo, G. (2021) · Pain · doi:10.1097/j.pain.0000000000001999 · Cited by 188
Psychological correlates of acute postsurgical pain: a systematic review and meta-analysis
Sobol‐Kwapinska, M., Bąbel, P., Plotek, W., & Stelcer, B. (2016) · European Journal of Pain · doi:10.1002/ejp.886 · Cited by 166
Strasser, L., Downes, M., Kung, J., Cross, J. H., & De Haan, M. (2018) · Developmental Medicine & Child Neurology · doi:10.1111/dmcn.13598 · Cited by 148
Martinez-Calderon, J., Flores-Cortes, M., Morales-Asencio, J. M., & Luque-Suarez, A. (2019) · The Journal of Pain · doi:10.1016/j.jpain.2019.04.009 · Cited by 141
Petersen, K. K., Vaegter, H. B., Stubhaug, A., Wolff, A., Scammell, B. E., Arendt-Nielsen, L., & Larsen, D. B. (2021) · Pain · doi:10.1097/j.pain.0000000000002019 · Cited by 140
Brownrigg, J. R. W., Hinchliffe, R. J., Apelqvist, J., Boyko, E. J., Fitridge, R., Mills, J. L., Reekers, J., Shearman, C. P., Zierler, R. E., Schaper, N. C., et al. (2016) · Diabetes/Metabolism Research and Reviews · doi:10.1002/dmrr.2704 · Cited by 130
Marsden, J., White, M., Annand, F., Burkinshaw, P., Carville, S., Eastwood, B., Kelleher, M., Knight, J., O'Connor, R., Tran, A., et al. (2019) · The Lancet Psychiatry · doi:10.1016/s2215-0366(19)30331-1 · Cited by 106
Smith, D. J., Combellick, J., Jordan, A. E., & Hagan, H. (2015) · International Journal of Drug Policy · doi:10.1016/j.drugpo.2015.07.004 · Cited by 104
Rhodes, E., Wilson, M., Robinson, A., Hayden, J. A., & Asbridge, M. (2019) · BMC Health Services Research · doi:10.1186/s12913-019-4642-8 · Cited by 103
Self-citationO’Caoimh, R., Cornally, N., Weathers, E., O'Sullivan, R., Fitzgerald, C., Orfila, F., Clarnette, R., Paúl, C., & Molloy, D. W. (2015) · Maturitas · doi:10.1016/j.maturitas.2015.03.009 · Cited by 89
“Prediction models usually combine multiple, weighted predictors, and valid instruments should adequately report patient characteristics, attrition, outcomes and confounders [9,10].”
BackgroundMethodInfluentialBours, M. J. L., Linden, B. W. A., Winkels, R. M., Duijnhoven, F. J., Mols, F., Roekel, E. H., Kampman, E., Beijer, S., & Weijenberg, M. P. (2016) · The Oncologist · doi:10.1634/theoncologist.2015-0258 · Cited by 85
Predictors of gastrointestinal bleeding in adult ICU patients: a systematic review and meta-analysis
the GUIDE Group, Granholm, A., Zeng, L., Dionne, J. C., Perner, A., Marker, S., Krag, M., MacLaren, R., Ye, Z., Møller, M. H., et al. (2019) · Intensive Care Medicine · doi:10.1007/s00134-019-05751-6 · Cited by 83
van Reijen, N. S., Ponchant, K., Ubbink, D. T., & Koelemay, M. J. W. (2019) · European Journal of Vascular and Endovascular Surgery · doi:10.1016/j.ejvs.2019.03.040 · Cited by 86
Cnossen, M. C., Scholten, A. C., Lingsma, H. F., Synnot, A., Haagsma, J., Steyerberg, P. E. W., & Polinder, S. (2017) · The Journal of Neuropsychiatry and Clinical Neurosciences · doi:10.1176/appi.neuropsych.16090165 · Cited by 79
Stokke, J., Sung, L., Gupta, A., Lindberg, A., & Rosenberg, A. R. (2015) · Pediatric Blood & Cancer · doi:10.1002/pbc.25514 · Cited by 77
Medeiros, G. C., Gould, T. D., Prueitt, W. L., Nanavati, J., Grunebaum, M. F., Farber, N. B., Singh, B., Selvaraj, S., Machado-Vieira, R., Achtyes, E. D., et al. (2022) · Molecular Psychiatry · doi:10.1038/s41380-022-01652-1 · Cited by 75
Witiw, C. D., Mansouri, A., Mathieu, F., Nassiri, F., Badhiwala, J. H., & Fessler, R. G. (2018) · Neurosurgical Review · doi:10.1007/s10143-016-0720-0 · Cited by 77
Jarvis, H., O'Keefe, H., Craig, D., Stow, D., Hanratty, B., & Anstee, Q. M. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-049767 · Cited by 71
Shen, X., Jiang, H., Xu, H., Ye, J., Lv, C., Lu, Z., & Gan, Y. (2020) · BMC Family Practice · doi:10.1186/s12875-020-01309-4 · Cited by 65
The psychological, social and educational impact of prominent ears: a systematic review
Jones, E. S., Gibson, J. A. G., Dobbs, T. D., & Whitaker, I. S. (2020) · Journal of Plastic, Reconstructive & Aesthetic Surgery · doi:10.1016/j.bjps.2020.05.075 · Cited by 65
Castrillon, J., Rele, S., Sumithran, P., Hii, M., Thuraisingam, S., Spelman, T., Phan, T., Choong, P., Dowsey, M., & Shadbolt, C. (2025) · Anaesthesia · doi:10.1111/anae.16601 · Cited by 65
Classification of foot ulcers in people with diabetes: a systematic review.
Monteiro‐Soares, M., Hamilton, E. J., Russell, D. A., Srisawasdi, G., Boyko, E. J., Mills, J. L., Jeffcoate, W., & Game, F. (2024) · Diabetes/Metabolism Research and Reviews · doi:10.1002/dmrr.3645 · Cited by 59
Varatharajan, S., Ferguson, B., Chrobak, K., Shergill, Y., Côté, P., Wong, J. J., Yu, H., Shearer, H. M., Southerst, D., Sutton, D., et al. (2016) · European Spine Journal · doi:10.1007/s00586-016-4376-9 · Cited by 59
Self-citationYe, Z., Ai, X., Liao, Z., You, C., & Cheng, Y. (2019) · Medicine · doi:10.1097/md.0000000000016371 · Cited by 57
Peters, S., Johnston, V., Hines, S., Ross, M., & Coppieters, M. (2016) · JBI Database of Systematic Reviews and Implementation Reports · doi:10.11124/jbisrir-2016-003099 · Cited by 49
Büttner, F., Winters, M., Delahunt, E., Elbers, R., Lura, C. B., Khan, K. M., Weir, A., & Ardern, C. L. (2020) · British Journal of Sports Medicine · doi:10.1136/bjsports-2019-100806 · Cited by 47
Seidel, D. H., Ditchen, D. M., Hoehne-Hückstädt, U. M., Rieger, M. A., & Steinhilber, B. (2019) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph16010130 · Cited by 43
Sarter, L., Heider, J., Kirchner, L., Schenkel, S., Witthöft, M., Rief, W., & Kleinstäuber, M. (2021) · Journal of Psychosomatic Research · doi:10.1016/j.jpsychores.2021.110486 · Cited by 38
Maternal overnutrition elevates offspring’s blood pressure-A systematic review and meta-analysis
Eitmann, S., Mátrai, P., Németh, D., Hegyi, P., Lukács, A., Bérczi, B., Czumbel, L. M., Kiss, I., Gyöngyi, Z., Varga, G., et al. (2022) · Paediatric and Perinatal Epidemiology · doi:10.1111/ppe.12859 · Cited by 35
Menta, R., Randhawa, K., Côté, P., Wong, J. J., Yu, H., Sutton, D., Varatharajan, S., Southerst, D., D'Angelo, K., Cox, J., et al. (2015) · Journal of Manipulative and Physiological Therapeutics · doi:10.1016/j.jmpt.2015.06.002 · Cited by 33
Self-citationCorazzari, C., Matteucci, M., Kołodziejczak, M., Kowalewski, M., Formenti, A. M., Giustina, A., Beghi, C., Barili, F., & Lorusso, R. (2022) · The Journal of Thoracic and Cardiovascular Surgery · doi:10.1016/j.jtcvs.2021.05.035 · Cited by 36
Tulipani, L. J., Meyer, B., Allen, D., Solomon, A. J., & McGinnis, R. S. (2022) · Gait & Posture · doi:10.1016/j.gaitpost.2022.02.016 · Cited by 32
De Baets, L., Matheve, T., & Timmermans, A. (2020) · American Journal of Physical Medicine & Rehabilitation · doi:10.1097/phm.0000000000001455 · Cited by 30
Janse van Mantgem, M. R., van Rheenen, W., Hackeng, A. V., van Es, M. A., Veldink, J. H., van den Berg, L. H., & van Eijk, R. P. A. (2023) · Neurology · doi:10.1212/wnl.0000000000201657 · Cited by 28
Sarter, L., Heider, J., Witthöft, M., Rief, W., & Kleinstäuber, M. (2022) · General Hospital Psychiatry · doi:10.1016/j.genhosppsych.2022.03.001 · Cited by 17
Methodological quality is underrated in systematic reviews and meta-analyses in health psychology
Oliveras, I., Losilla, J.-M., & Vives, J. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.05.002 · Cited by 15
Karagiannis, T., Advani, A., Davies, M. J., Del Prato, S., Dinneen, S. F., Kuss, O., Lorenzoni, V., Mathieu, C., Mauricio, D., Tankova, T., et al. (2025) · Diabetologia · doi:10.1007/s00125-025-06370-1 · Cited by 14
(2021) · The Journal of Spinal Cord Medicine · doi:10.1080/10790268.2021.1863738 · Cited by 9
Pedersen, P. B., Hrobjartsson, A., Nielsen, D. L., Henriksen, D. P., Brabrand, M., & Lassen, A. T. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0622-4 · Cited by 6
Metadata
- Authors
- Hayden, J. A., van der Windt, D. A., Cartwright, J. L., Côté, P., & Bombardier, C.
- Affiliations
- Dalhousie University (CA)
- From Dalhousie University, Halifax, Nova Scotia, Canada; Arthritis Research UK Primary Care Centre, Primary Care and Health Sciences, Keele University, Staffordshire, United Kingdom; University of Ontario Institute of Technology, Oshawa, Ontario, Canada; and University of Toronto and Institute for Work & Health, Toronto, Ontario, Canada.
- Venue
- Annals of Internal Medicine
- Publisher
- American College of Physicians
- Published
- 2013-02-19
- Volume
- 158
- Issue
- 4
- Pages
- 280-286
- ISSN
- 0003-4819 (ISSN-L)
- 1539-3704
- OpenAlex
- W2082977560
- Cited by
- 3,287
- References
- 2
- Citations by year
- 2026: 408 · 2025: 325 · 2024: 330 · 2023: 342 · 2022: 381 · 2021: 418 · 2020: 335 · 2019: 212 · 2018: 171 · 2017: 145 · 2016: 106 · 2015: 73 · 2014: 33 · 2013: 6
- FWCI
- 80.55
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Annals of Internal Medicine (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Keywords
- risk of bias, prognostic factors, bias assessment, interobserver agreement, research validity, attrition, confounding, outcome measurement, reporting quality
- MeSH terms
- Humans, Prognosis, Research Design, Review Literature as Topic, Bias, Low Back Pain
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review