i-CONTENT tool for assessing therapeutic quality of exercise programs employed in randomised clinical trials
Open accessAbstract
Objective: When appraising the quality of randomised clinical trial (RCTs) on the merits of exercise therapy, we typically limit our assessment to the quality of the methods. However, heterogeneity across studies can also be caused by differences in the quality of the exercise interventions (ie, ‘the potential effectiveness of a specific intervention given the potential target group of patients’)—a challenging concept to assess. We propose an internationally developed, consensus-based tool that aims to assess the quality of exercise therapy programmes studied in RCTs: the international Consensus on Therapeutic Exercise aNd Training (i-CONTENT) tool. Methods: Forty-nine experts (from 12 different countries) in the field of physical and exercise therapy participated in a four-stage Delphi approach to develop the i-CONTENT tool: (1) item generation (Delphi round 1), (2) item selection (Delphi rounds 2 and 3), (3) item specification (focus group discussion) and (4) tool development and refinement (working group discussion and piloting). Results: Out of the 61 items generated in the first Delphi round, consensus was reached on 17 items, resulting in seven final items that form the i-CONTENT tool: (1) patient selection; (2) qualified supervisor; (3) type and timing of outcome assessment; (4) dosage parameters (frequency, intensity, time); (5) type of exercise; (6) safety of the exercise programme and (7) adherence to the exercise programme. Conclusion: The i-CONTENT-tool is a step towards transparent assessment of the quality of exercise therapy programmes studied in RCTs, and ultimately, towards the development of future, higher quality, exercise interventions.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Lack of large effect sizes can be attributed to low training dosage (mode of exercise, frequency, intensity, duration and adherence to training).
“Lack of large effect sizes can be attributed to low training dosage (mode of exercise, frequency, intensity, duration and adherence to training) [76].”
From International urogynecology consultation chapter 3 committee 2; conservative treatment of patient with pelvic organ prolapse: Pelvic floor muscle training (2022) · cites this paper as [76]
The Concluding remarks state that exercise dosage and type affect effectiveness through a dose–response relationship. The Results checklist also treats frequency, intensity, time, and adherence as relevant to whether an exercise programme achieves its intended effect. But the paper does not establish that low training dosage caused the particular lack of large effect sizes asserted in the citing sentence. (LLM, conf=0.84)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The Cochrane Collaboration's tool for assessing risk of bias in randomised trials
Higgins, J. P. T., Altman, D. G., Gotzsche, P. C., Juni, P., Moher, D., Oxman, A. D., Savovic, J., Schulz, K. F., Weeks, L., Sterne, J. A. C., et al. (2011) · BMJ · doi:10.1136/bmj.d5928 · Cited by 33,817 · see citing works
Hoffmann, T. C., Glasziou, P. P., Boutron, I., Milne, R., Perera, R., Moher, D., Altman, D. G., Barbour, V., Macdonald, H., Johnston, M., et al. (2014) · BMJ · doi:10.1136/bmj.g1687 · Cited by 10,548 · see citing works
CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials
Schulz, K. F., Altman, D. G., Moher, D., & for the CONSORT Group (2010) · BMJ · doi:10.1136/bmj.c332 · Cited by 9,329 · see citing works
Schulz (1995) · doi:10.1001/jama.1995.03520290060030 · Cited by 5,634 · see citing works
Mokkink, L. B., Terwee, C. B., Patrick, D. L., Alonso, J., Stratford, P. W., Knol, D. L., Bouter, L. M., & de Vet, H. C. W. (2010) · Quality of Life Research · doi:10.1007/s11136-010-9606-8 · Cited by 4,260 · see citing works
Exercise as medicine - evidence for prescribing exercise as therapy in 26 different chronic diseases
Pedersen, B. K., & Saltin, B. (2015) · Scandinavian Journal of Medicine & Science in Sports · doi:10.1111/sms.12581 · Cited by 3,514 · see citing works
What is an adequate sample size? Operationalising data saturation for theory-based interview studies
Francis, J. J., Johnston, M., Robertson, C., Glidewell, L., Entwistle, V., Eccles, M. P., & Grimshaw, J. M. (2010) · Psychology & Health · doi:10.1080/08870440903194015 · Cited by 3,429 · see citing works
Huber, M., Knottnerus, J. A., Green, L., van der Horst, H. E., Jadad, A. R., Kromhout, D., Leonard, B. E., Lorig, K. R., Loureiro, I., Meer, J. W. M. V. D., et al. (2011) · BMJ · doi:10.1136/bmj.d4163 · Cited by 2,644 · see citing works
The Delphi technique: myths and realities
Powell, C. (2003) · Journal of Advanced Nursing · doi:10.1046/j.1365-2648.2003.02537.x · Cited by 2,611 · see citing works
Boutron, I., Moher, D., Altman, D. G., Schulz, K. F., Ravaud, P., & for the CONSORT Group* (2008) · Annals of Internal Medicine · doi:10.7326/0003-4819-148-4-200802190-00008 · Cited by 2,292 · see citing works
Adverse Events Associated with Testosterone Administration
Basaria, S., Coviello, A. D., Travison, T. G., Storer, T. W., Farwell, W. R., Jette, A. M., Eder, R., Tennstedt, S., Ulloor, J., Zhang, A., et al. (2010) · New England Journal of Medicine · doi:10.1056/nejmoa1000485 · Cited by 1,494 · see citing works
Qualitative research methods in health technology assessment: a review of the literature.
Murphy, Dingwall, Greatbatch, Parker, & Watson (1998) · Health Technology Assessment · doi:10.3310/hta2160 · Cited by 1,107 · see citing works
Blinding in randomised trials: hiding who got what
Schulz, K. F., & Grimes, D. A. (2002) · The Lancet · doi:10.1016/s0140-6736(02)07816-9 · Cited by 952 · see citing works
Hayden, J. A., van Tulder, M. W., & TOMLINSON, G. A. (2005) · Annals of Internal Medicine · doi:10.7326/0003-4819-142-9-200505030-00014 · Cited by 848 · see citing works
Bias in clinical intervention research.
Gluud, L. L. (2006) · American Journal of Epidemiology · doi:10.1093/aje/kwj069 · Cited by 636 · see citing works
Naci, H., & Ioannidis, J. P. A. (2013) · BMJ · doi:10.1136/bmj.f5577 · Cited by 630 · see citing works
Consensus on Exercise Reporting Template (CERT): Modified Delphi Study
Slade, S. C., Dionne, C. E., Underwood, M., Buchbinder, R., Beck, B., Bennell, K., Brosseau, L., Costa, L., Cramp, F., Cup, E., et al. (2016) · Physical Therapy · doi:10.2522/ptj.20150668 · Cited by 478 · see citing works
Self-citationExpert Practice in Physical Therapy
Jensen, G. M., Gwyer, J., Shepard, K. F., & Hack, L. M. (2000) · Physical Therapy · doi:10.1093/ptj/80.1.28 · Cited by 474 · see citing works
Campbell, R. M., Evans, M., Tucker, M., Quilty, B., Dieppe, P. A., & Donovan, J. (2001) · Journal of Epidemiology & Community Health · doi:10.1136/jech.55.2.132 · Cited by 414 · see citing works
Adverse metabolic response to regular exercise: is it a rare or common occurrence?
Bouchard, C., Blair, S. N., Church, T. S., Earnest, C. P., Hagberg, J. M., Häkkinen, K., Jenkins, N. T., Karavirta, L., Kraus, W. E., Leon, A. S., et al. (2012) · PLoS ONE · doi:10.1371/journal.pone.0037887 · Cited by 389 · see citing works
SPIRIT 2013: new guidance for content of clinical trial protocols
Chan, A.-W., Tetzlaff, J. M., Altman, D. G., Dickersin, K., & Moher, D. (2013) · The Lancet · doi:10.1016/s0140-6736(12)62160-6 · Cited by 298 · see citing works
Effects of exercise dose and type during breast cancer chemotherapy: multicenter randomized trial
Courneya, K. S., McKenzie, D. C., Mackey, J. R., Gelmon, K., Friedenreich, C. M., Yasui, Y., Reid, R. D., Cook, D., Jespersen, D., Proulx, C., et al. (2013) · JNCI: Journal of the National Cancer Institute · doi:10.1093/jnci/djt297 · Cited by 297 · see citing works
The Landscape and Lexicon of Blinding in Randomized Trials
Schulz, K. F., Chalmers, I., & Altman, D. G. (2002) · Annals of Internal Medicine · doi:10.7326/0003-4819-136-3-200202050-00022 · Cited by 242 · see citing works
Hawley-Hague, H., Boulton, E. R., Hall, A., Pfeiffer, K., & Todd, C. (2014) · Int. J. Medical Informatics · doi:10.1016/j.ijmedinf.2014.03.002 · Cited by 213 · see citing works
Using Clinical Outcomes to Explore the Theory of Expert Practice in Physical Therapy
Resnik, L. J., & Jensen, G. M. (2003) · Physical Therapy · doi:10.1093/ptj/83.12.1090 · Cited by 187 · see citing works
Hoogeboom, T. J., Oosting, E., Vriezekolk, J. E., Veenhof, C., Siemonsma, P. C., de Bie, R. A., van den Ende, C. H. M., & van Meeteren, N. L. U. (2012) · PLoS ONE · doi:10.1371/journal.pone.0038031 · Cited by 174 · see citing works
Self-citationAnalysis of quality of interventions in systematic reviews
Herbert, R., & B�, K. (2005) · BMJ · doi:10.1136/bmj.331.7515.507 · Cited by 164 · see citing works
Schulz, K. F., Grimes, D. A., Altman, D. G., & Hayes, R. J. (1996) · BMJ · doi:10.1136/bmj.312.7033.742 · Cited by 163 · see citing works
A scale for rating the quality of psychological trials for pain
Yates, S. L., Morley, S., Eccleston, C., & de C. Williams, A. C. (2005) · Pain · doi:10.1016/j.pain.2005.06.018 · Cited by 150 · see citing works
Home Exercise Training Improves Exercise Capacity in Cirrhosis Patients: Role of Exercise Adherence
Kruger, C., McNeely, M. L., Bailey, R. J., Yavari, M., Abraldeṣ, J. G., Carbonneau, M., Newnham, K., Heyer, V. D., Ma, M. M., Thompson, R. B., et al. (2018) · Scientific Reports · doi:10.1038/s41598-017-18320-y · Cited by 131 · see citing works
Neil-Sztramko, S. E., Winters-Stone, K. M., Bland, K. A., & Campbell, K. L. (2019) · British Journal of Sports Medicine · doi:10.1136/bjsports-2017-098389 · Cited by 120 · see citing works
Holden, S., Rathleff, M. S., Jensen, M. B., & Barton, C. J. (2017) · British Journal of Sports Medicine · doi:10.1136/bjsports-2017-097547 · Cited by 88 · see citing works
Dagfinrud, H. S., Halvorsen, S., Vøllestad, N. K., et al. (2010) · Arthritis Care & Research · doi:10.1002/acr.20415 · Cited by 77 · see citing works
Agreement on Disease-Specific Criteria for Do-Not-Resuscitate Orders in Acute Stroke
Alexandrov, A. V., Pullicino, P. M., Meslin, E. M., & Norris, J. W. (1996) · Stroke · doi:10.1161/01.str.27.2.232 · Cited by 72 · see citing works
Osho, O., Owoeye, O. B. A., & Armijo-Olivo, S. L. (2017) · Journal of Aging and Physical Activity · doi:10.1123/japa.2016-0326 · Cited by 72 · see citing works
Exercise Dose, Exercise Adherence, and Associated Health Outcomes in the TIGER Study
Miller, F. L., O’Connor, D. P., Herring, M. P., Sailors, M. H., Jackson, A. S., Dishman, R. K., & Bray, M. S. (2013) · Medicine & Science in Sports & Exercise · doi:10.1249/mss.0b013e3182a038b9 · Cited by 64 · see citing works
Slade, S. C., Finnegan, S., Dionne, C. E., Underwood, M. R., & Buchbinder, R. (2018) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2018.07.009 · Cited by 56 · see citing works
Predictors of Response to Physical Therapy Intervention in Patients With Primary Hip Osteoarthritis
Wright, A. A., Cook, C. E., Flynn, T. W., Baxter, D. G., & Abbott, J. H. (2011) · Physical Therapy · doi:10.2522/ptj.20100171 · Cited by 56 · see citing works
Neil-Sztramko, S. E., Medysky, M. E., Campbell, K. L., Bland, K. A., & Winters-Stone, K. M. (2019) · BMC Cancer · doi:10.1186/s12885-019-5520-9 · Cited by 51 · see citing works
Tew, G. A., Brabyn, S., Cook, L., & Peckham, E. J. (2016) · PLoS ONE · doi:10.1371/journal.pone.0150869 · Cited by 49 · see citing works
Manual therapy in osteoarthritis of the hip: outcome in subgroups of patients
Hoeksma, H. L. (2005) · British journal of rheumatology · doi:10.1093/rheumatology/keh482 · Cited by 44 · see citing works
Movement for movement: exercise as everybody's business?
Gates, A. B., Kerry, R., Moffatt, F., Ritchie, I. K., Meakins, A., Thornton, J. S., Rosenbaum, S., & Taylor, A. (2016) · British Journal of Sports Medicine · doi:10.1136/bjsports-2016-096857 · Cited by 36 · see citing works
Considerations in the Application of a Modified Scree Test for Delphi Survey Data
Altschuld, J. W., & Thomas, P. M. (1991) · Evaluation Review · doi:10.1177/0193841x9101500201 · Cited by 31 · see citing works
Which patientswith osteoarthritis of hip and/or knee benefit most from behavioral graded activity?
Veenhof, C., van den Ende, C. H., Dekker, J., Köke, A. J. A., Oostendorp, R. A. B., & Bijlsma, J. W. J. (2007) · International Journal of Behavioral Medicine · doi:10.1007/bf03004173 · Cited by 26 · see citing works
Replicability of Physical Exercise Interventions in Lung Transplant Recipients; A Systematic Review
Knols, R. H., Fischer, N., Kohlbrenner, D., Manettas, A. I., & de Bruin, E. D. (2018) · Frontiers in Physiology · doi:10.3389/fphys.2018.00946 · Cited by 17 · see citing works
Jo, D., Bel, M. J. D., McEwen, D., et al. (2018) · Clinical Rehabilitation · doi:10.1177/0269215518815931 · Cited by 17 · see citing works
Gallois, M., Davergne, T., Ledinot, P., Ravaud, P., & Régnaux, J.-P. (2017) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2017.03.030 · Cited by 7 · see citing works
Flagged first, then by citations
Research works printed but not linked: 6
- [5]ACSM. ACSM’s Guidelines for Exercise Testing and Prescription. Lippincott Williams And Wilkins, 2013: 480.
- [12]Abell B, Glasziou P, Hoffmann T. Reporting and replicating trials of exercise-based cardiac rehabilitation: do we know what the researchers actually did? Circ Cardiovasc Qual Outcomes 2015; 8: 187– 94. 25737485
- [22]Therapeutic validity of exercise therapy in RCTs. Puijo symposium; 2014; Kuopio.
- [24]Verhagen AP, de Vet HC, de Bie RA, et al. The Delphi list: a criteria list for quality assessment of randomized clinical trials for conducting systematic reviews developed by Delphi consensus. J Clin Epidemiol 1998; 51: 1235– 41. 10086815
- [40]McGinn TG, Guyatt GH, Wyer PC, et al. Users' guides to the medical literature: XXII: how to use articles about clinical decision rules. JAMA 2000; 284: 79– 84. 10872017
- [54]Campbell SM, Hann M, Roland MO, et al. The effect of panel membership and feedback on ratings in a two-round Delphi survey: results of a randomized controlled trial. Med Care 1999; 37: 964– 8. 10493474
Cited by
Sorted by citations
Bø, K., Anglès-Acedo, S., Batra, A., Brækken, I. H., Chan, Y. L., Jorge, C. H., Kruger, J., Yadav, M., & Dumoulin, C. (2022) · International Urogynecology Journal · doi:10.1007/s00192-022-05324-0 · Cited by 64 · see citing works
“Lack of large effect sizes can be attributed to low training dosage (mode of exercise, frequency, intensity, duration and adherence to training) [76].”
BackgroundSelf-citationGiagio, S., Innocenti, T., Salvioli, S., Lami, A., Meriggiola, M. C., Pillastrini, P., & Gava, G. (2021) · Neurourology and Urodynamics · doi:10.1002/nau.24712 · Cited by 19 · see citing works
- Authors
- Hoogeboom, T. J., Kousemaker, M. C., van Meeteren, N. L., Howe, T., Bo, K., Tugwell, P., Ferreira, M., de Bie, R. A., van den Ende, C. H., & Stevens-Lapsley, J. E.
- Affiliations
- Radboud University Medical Center (NL)
- Radboud University Nijmegen (NL)
- Cochrane Canada (CA)
- University of Ottawa (CA)
- The University of Sydney (AU)
- Corresponding author
- Hoogeboom, Thomas J
- Venue
- British Journal of Sports Medicine
- Publisher
- BMJ
- Published
- 2020-11-03
- Volume
- 55
- Issue
- 20
- Pages
- 1153-1160
- ISSN
- 0306-3674 (ISSN-L)
- 1473-0480
- PMC
- PMC8479742
- OpenAlex
- W3096840688
- Cited by
- 76
- References
- 47
- Citations by year
- 2026: 8 · 2025: 18 · 2024: 13 · 2023: 18 · 2022: 13 · 2021: 6
- FWCI
- 12.37
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- British Journal of Sports Medicine (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- Research Publications (Maastricht University) (publishedVersion, cc-by-nc)
- EUR Research Repository (Erasmus University Rotterdam) (publishedVersion, cc-by-nc)
- EUR Research Repository (Erasmus University Rotterdam) (submittedVersion, other-oa)
- RePub (Erasmus University Rotterdam) (submittedVersion, other-oa)
- NORA - Norwegian Open Research Archives (submittedVersion, cc-by-nc)
- Radboud Repository (Radboud University) (submittedVersion)
- Radboud Repository (Radboud University) (submittedVersion, other-oa)
- Research Publications (Maastricht University) (submittedVersion, cc-by-nc)
- Topics
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Medicine › Physiology › Physical Activity and Health
- Keywords
- exercise therapy, randomized controlled trials, exercise safety, exercise adherence, patient selection, supervised exercise, Delphi consensus
- MeSH terms
- Delphi Technique, Exercise Therapy, Humans, Randomized Controlled Trials as Topic, Consensus
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
