Empirical Evidence of an Association Between Internal Validity and Effect Size in Randomized Controlled Trials of Low-Back Pain
Open accessThe DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Abstract
STUDY DESIGN: We conducted a methodologic study. OBJECTIVE: The objective of this study was to assess the validity of the criteria list recommended by the Cochrane Back Review Group Editorial Board by evaluating whether individual items and a total score are associated with effect sizes in randomized controlled trials of back-pain interventions. SUMMARY OF BACKGROUND DATA: There is concern that studies of low methodologic quality may exaggerate the effectiveness of treatments for low back pain. We performed this study to examine the association between a common measure of internal validity and the reported magnitude of treatment effects. METHODS: We assessed the relationship between the 11 items contained in the Cochrane Back Review Group Internal Validity checklist and effect size in randomized trials of interventions for back pain. Of 267 trials in 15 Cochrane reviews that were eligible for inclusion, 51 were excluded, leaving 216 trials included in the analysis. The scores on the 11 items for each trial were taken from the original review. We extracted effect sizes from each low back pain trial. RESULTS: We found that trials that fulfilled a specific item had smaller effect sizes compared with trials that did not fulfill that item for 10 of the 11 items, and for 6 of the criteria, the absolute difference in effect sizes was 0.10 or greater. The 95% confidence interval of the difference in effect sizes crossed the null value in each case. The number of items fulfilled showed that trials with higher scores consistently reported smaller effect sizes than trials with lower scores. At the thresholds of 5 or 6 items fulfilled, the difference in effect sizes was 0.20 in each case (95% confidence intervals 0.05-0.35 and 0.06-0.34, respectively). Stratified analyses did not support confounding by intervention. CONCLUSION: We conclude that the 11-item Internal Validity Checklist is associated with effect size in randomized trials of interventions for back pain, and that our data support the use of a sum score of the number of fulfilled items in this list.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Moher, D., Pham, B., Jones, A., Cook, D. J., Jadad, A. R., Moher, M., Tugwell, P., & Klassen, T. P. (1998) · The Lancet · doi:10.1016/s0140-6736(98)01085-x · Cited by 3,560
Assessing the quality of controlled clinical trials
Jüni, P., Altman, D. G., & Egger, M. (2001) · BMJ · doi:10.1136/bmj.323.7303.42 · Cited by 3,309 · cited 2×
“Juni et al reported a meta-analysis of 3 items—generation of the allocation sequence, concealment of treatment allocation, and double-blinding—that were assessed in 4 methodologic studies that gauged their relative importance in a large number of clinical trials, while avoiding confounding by disease or by interventio…”
BackgroundMethodUpdated Method Guidelines for Systematic Reviews in the Cochrane Collaboration Back Review Group
van Tulder, M., Furlan, A., Bombardier, C., & Bouter, L. (2003) · Spine · doi:10.1097/01.brs.0000065484.95996.af · Cited by 2,180 · cited 2×
“They recommended that 11 criteria be used as the standard measure to assess the methodologic quality, or internal validity, of RCTs.(11) This list was based on the criteria in Jadad et al’s(4) and Verhagen et al’s lists(6) and was a modification of the list used by Koes et al.”
MethodSelf-citationThe Hazards of Scoring the Quality of Clinical Trials for Meta-analysis
Jüni, P. (1999) · JAMA · doi:10.1001/jama.282.11.1054 · Cited by 1,981
Imputing missing standard deviations in meta-analyses can provide accurate results
Furukawa, T. A., Barbui, C., Cipriani, A., Brambilla, P., & Watanabe, N. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2005.06.006 · Cited by 1,568 · cited 1×
“standard deviation by taking the mean standard deviation, weighted by the relevant treatment group’s sample size, across all other trials that reported standard deviations for that outcome.(17) Effect sizes were assessed because they convert different scales into the same metric, so that they can be compared.”
MethodMultidisciplinary rehabilitation for chronic low back pain: systematic review
Guzmán, J., Esmail, R., Karjalainen, K., Malmivaara, A., Irvin, E., & Bombardier, C. (2001) · BMJ · doi:10.1136/bmj.322.7301.1511 · Cited by 941
Exercise therapy for treatment of non-specific low back pain
Hayden, J., van Tulder, M. W., Malmivaara, A., & Koes, B. W. (2005) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000335.pub2 · Cited by 879
Behavioural treatment for chronic low-back pain
Henschke, N., Ostelo, R. W., van Tulder, M. W., Vlaeyen, J. W., Morley, S., Assendelft, W. J., & Main, C. J. (2010) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd002014.pub3 · Cited by 718
Balk, E. M. (2002) · JAMA · doi:10.1001/jama.287.22.2973 · Cited by 478 · cited 2×
“Balk et al examined 24 quality criteria, assessing 276 trials in 4 clinical conditions, and found no criterion that was associated with a statistically significantly difference in treatment effects.(3) Eight of these 24 criteria were identical or related to criteria in the CBRG internal validity checklist.”
Background
Cited by
2009 updated method guidelines for systematic reviews in the Cochrane Back Review Group
Furlan, A. D., Pennick, V., Bombardier, C., & van Tulder, M. (2009) · Spine · doi:10.1097/brs.0b013e3181b1c99f · Cited by 1,663
“” There is empirical evidence from a methodologic study conducted with data from the CBRG that a compliance threshold of less than 50% of the criteria is associated with bias.(34) The results of the assessment, including the rationale for the decision, should be presented in the “Risk of Bias” table, which is included…”
BackgroundKamper, S. J., Apeldoorn, A. T., Chiarotto, A., Smeets, R. J. E. M., Ostelo, R. W. J. G., Guzman, J., & van Tulder, M. W. (2015) · BMJ · doi:10.1136/bmj.h444 · Cited by 1,092
Behavioural treatment for chronic low-back pain
Henschke, N., Ostelo, R. W., van Tulder, M. W., Vlaeyen, J. W., Morley, S., Assendelft, W. J., & Main, C. J. (2010) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd002014.pub3 · Cited by 718
Physical activity and low back pain: a systematic review of recent literature
Heneweer, H., Staes, F., Aufdemkampe, G., van Rijn, M., & Vanhees, L. (2011) · European Spine Journal · doi:10.1007/s00586-010-1680-7 · Cited by 395
Interventions for preventing and treating pelvic and back pain in pregnancy
Pennick, V., & Liddle, S. D. (2013) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd001139.pub3 · Cited by 364
Metadata
- Authors
- van Tulder, M. W., Suttorp, M., Morton, S., Bouter, L. M., & Shekelle, P.
- Venue
- Spine
- Published
- 2009-07
- Volume
- 34
- Issue
- 16
- Pages
- 1685-1692
- ISSN
- 0362-2436 (ISSN-L)
- OpenAlex
- W2080109974
- Cited by
- 132
- References
- 9
- FWCI
- 20.30
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Empirical Evidence of an Association Between Internal Validity and Effect Size in Randomized Controlled Trials of Low-Back Pain. https://app.cochat.ai/papers/10.1097/brs.0b013e3181ab6a78
- The paper
- van Tulder, M. W., Suttorp, M., Morton, S., Bouter, L. M., & Shekelle, P. (2009). Empirical Evidence of an Association Between Internal Validity and Effect Size in Randomized Controlled Trials of Low-Back Pain. Spine. https://doi.org/10.1097/brs.0b013e3181ab6a78
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.