A meta-regression analysis shows no impact of design characteristics on outcome in trials on tension-type headaches
Abstract
Objectives In the conduct of a systematic review or meta-analysis, many possible sources of bias exist, such as bias caused by design characteristics. We studied the influence of the methodological study characteristics of randomized clinical trials (RCTs) on the outcome in a systematic review on conservative treatments in patients with tension-type headache (TTH).Study design and setting Included were RCTs from a systematic review on TTH, which were a control group receiving a placebo or no treatment and presented data on recovery or headache severity, intensity, or frequency. Design characteristics were assessed using the Delphi list. Regression analysis is performed on separate design characteristics on size of treatment effect.Results Out of the original data set of 146 trials, 61 trials fulfilled our selection criteria. The number of trials presenting only dichotomous data was larger than trials presenting only continuous data. All study characteristics show a nonsignificant relation with the effect estimate. Whether outcome is presented dichotomous or continuous appears to have a significant impact on treatment effect estimates.Conclusion In this study, sample design characteristics do not show to have an impact on treatment effect estimates, but the way the treatment effect is measured has a significant impact.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The use of a dichotomous analysis metric compared with a continuous metric reduces statistical power.
“The use of a dichotomous analysis metric compared with a continuous metric reduces statistical power268 269 270”
From SPIRIT 2025 explanation and elaboration: updated guideline for protocols of randomised trials (2025) · cites this paper as [269]
Cites
Statistical Power Analysis for the Behavioral Sciences
Muller, K. E., & Cohen, J. (1989) · Technometrics · doi:10.2307/1270020 · Cited by 99,588 · see citing works
Wasserman, S. S., Hedges, L. V., & Olkin, I. (1988) · Journal of Educational Statistics · doi:10.2307/1164953 · Cited by 10,347 · see citing works
Schulz (1995) · doi:10.1001/jama.1995.03520290060030 · Cited by 5,634 · see citing works
Moher, D., Pham, B., Jones, A., Cook, D. J., Jadad, A. R., Moher, M., Tugwell, P., & Klassen, T. P. (1998) · The Lancet · doi:10.1016/s0140-6736(98)01085-x · Cited by 3,560 · see citing works
How should meta‐regression analyses be undertaken and interpreted?
Thompson, S. G., & Higgins, J. P. T. (2002) · Statistics in Medicine · doi:10.1002/sim.1187 · Cited by 3,026 · see citing works
The cost of dichotomising continuous variables
Altman, D. G., & Royston, P. (2006) · BMJ · doi:10.1136/bmj.332.7549.1080 · Cited by 2,688 · see citing works
Statistical Methods for Meta-Analysis
(1985) · Elsevier eBooks · doi:10.1016/c2009-0-03396-0 · Cited by 2,549 · see citing works
Verhagen, A., de Vet, H. D., de Bie, R. D., Kessels, A., Boers, M., Bouter, L., & Knipschild, P. (1998) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(98)00131-0 · Cited by 2,473 · see citing works
Self-citationAdvanced methods in meta‐analysis: multivariate approach and meta‐regression
van Houwelingen, H. C., Arends, L. R., & Stijnen, T. (2002) · Statistics in Medicine · doi:10.1002/sim.1040 · Cited by 1,574 · see citing works
Self-citationControlling the risk of spurious findings from meta-regression
Higgins, J. P. T., & Thompson, S. G. (2004) · Statistics in Medicine · doi:10.1002/sim.1752 · Cited by 1,267 · cited 1× · see citing works
“5 indicates substantial heterogeneity [24].”
Language bias in randomised controlled trials published in English and German
Egger, M., Zellweger-Zähner, T., Schneider, M., Junker, C., Lengeler, C., & Antes, G. (1997) · The Lancet · doi:10.1016/s0140-6736(97)02419-7 · Cited by 1,028 · see citing works
Direction and impact of language bias in meta-analyses of controlled trials: empirical study
Jüni, P., Holenstein, F., Sterne, J., Bartlett, C., & Egger, M. (2002) · International Journal of Epidemiology · doi:10.1093/ije/31.1.115 · Cited by 892 · see citing works
Kunz, R., & Oxman, A. D. (1998) · BMJ · doi:10.1136/bmj.317.7167.1185 · Cited by 770 · see citing works
Effect-Size Indices for Dichotomized Outcomes in Meta-Analysis.
Sánchez‐Meca, J., Marín‐Martínez, F., & Moscoso, S. C. (2003) · Psychological Methods · doi:10.1037/1082-989x.8.4.448 · Cited by 635 · see citing works
Balk, E. M. (2002) · JAMA · doi:10.1001/jama.287.22.2973 · Cited by 478 · see citing works
Assessing the Quality of Randomized Controlled Trials:<i>Current Issues and Future Directions</i>
Moher, D., Jadad, A. R., & Tugwell, P. (1996) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462300009570 · Cited by 436 · see citing works
Sterne, J. A. C., Jüni, P., Schulz, K. F., Altman, D. G., Bartlett, C., & Egger, M. (2002) · Statistics in Medicine · doi:10.1002/sim.1184 · Cited by 382 · see citing works
Schmid, C. H., Stark, P. C., Berlin, J. A., Landais, P., & Lau, J. (2004) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2003.12.001 · Cited by 245 · see citing works
The art of quality assessment of RCTs included in systematic reviews
Verhagen, A. P., de Vet, H. C. W., de Bie, R. A., Boers, M., & van den Brandt, P. A. (2001) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(00)00360-7 · Cited by 237 · see citing works
Self-citationSchulz, K. F., Grimes, D. A., Altman, D. G., & Hayes, R. J. (1996) · BMJ · doi:10.1136/bmj.312.7033.742 · Cited by 163 · see citing works
Lenssinck, M.-L. B., Damen, L., Verhagen, A. P., Berger, M. Y., Passchier, J., & Koes, B. W. (2004) · Pain · doi:10.1016/j.pain.2004.09.026 · Cited by 111 · see citing works
Self-citationPublication bias in the medical literature: A review by a Canadian research ethics board
Hall, R., de Antueno, C., & Webber, A. P. (2007) · Canadian Journal of Anesthesia/Journal canadien d anesthésie · doi:10.1007/bf03022661 · Cited by 52 · see citing works
Is any one analgesic superior for episodic tension-type headache?
Verhagen, A. P., Damen, L., Berger, M. Y., Passchier, J., Merlijn, V. P. B. M., & Koes, B. W. (2006) · PubMed · Cited by 42 · see citing works
Self-citationJackson, D. (2007) · Biometrics · doi:10.1111/j.1541-0420.2006.00663.x · Cited by 23 · see citing works
Verhagen, A. P., de Vet, H. C. W., Vermeer, F., Widdershoven, J. W. M. G., de Bie, R. A., Kessels, A. G. H., Boers, M., & van den Brandt, P. A. (2002) · PubMed · Cited by 20 · see citing works
Self-citationConservative treatments of children with episodic tension–type headache
Verhagen, A. P., Damen, L., Berger, M. Y., Passchier, J., Merlijn, V. P. B. M., & Koes, B. W. (2005) · Journal of Neurology · doi:10.1007/s00415-005-0940-7 · Cited by 18 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials
Chan, A.-W., Tetzlaff, J. M., Gotzsche, P. C., Altman, D. G., Mann, H., Berlin, J. A., Dickersin, K., Hrobjartsson, A., Schulz, K. F., Parulekar, W. R., et al. (2013) · BMJ · doi:10.1136/bmj.e7586 · Cited by 6,251 · see citing works
Randomisation to protect against selection bias in healthcare trials
Odgaard-Jensen, J., Vist, G. E., Timmer, A., Kunz, R., Akl, E. A., Schünemann, H., Briel, M., Nordmann, A. J., Pregno, S., & Oxman, A. D. (2011) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000012.pub3 · Cited by 439 · see citing works
Interventions for enhancing adherence with physiotherapy: A systematic review
McLean, S. M., Burton, M., Bradley, L., & Littlewood, C. (2010) · Manual Therapy · doi:10.1016/j.math.2010.05.012 · Cited by 204 · see citing works
SPIRIT 2025 explanation and elaboration: updated guideline for protocols of randomised trials
Hróbjartsson, A., Boutron, I., Hopewell, S., Moher, D., Schulz, K. F., Collins, G. S., Tunn, R., Aggarwal, R., Berkwits, M., Berlin, J. A., et al. (2025) · BMJ · doi:10.1136/bmj-2024-081660 · Cited by 171 · see citing works
Armijo-Olivo, S. L., Saltaji, H., da Costa, B. R., Fuentes, J., Ha, C., & Cummings, G. G. (2015) · BMJ Open · doi:10.1136/bmjopen-2015-008562 · Cited by 87 · see citing works
Chronic tension-type headache: what is new?
Fernández‐de‐las‐Peñas, C., & Schoenen, J. E. (2009) · Current Opinion in Neurology · doi:10.1097/wco.0b013e32832973ce · Cited by 66 · see citing works
Fernández‐de‐las‐Peñas, C., Cleland, J. A., Palomeque-del-Cerro, L., Caminero, A. B., Guillem-Mesado, A., & Jiménez‐García, R. (2010) · Headache The Journal of Head and Face Pain · doi:10.1111/j.1526-4610.2010.01789.x · Cited by 40 · see citing works
Mousavi, S. A., Mirbod, S. M., & Khorvash, F. (2011) · PubMed Central · Cited by 28 · see citing works
Pharmacotherapy of tension-type headache (TTH)
Lenaerts, M. E. P. (2009) · Expert Opinion on Pharmacotherapy · doi:10.1517/14656560902942046 · Cited by 19 · see citing works
Placebo effect was influenced by publication year in three-armed acupuncture trials
We, S. R., Koog, Y. H., Park, M. S., & Min, B.-I. (2011) · Complementary Therapies in Medicine · doi:10.1016/j.ctim.2011.10.003 · Cited by 13 · see citing works
Problems Caused by Heterogeneity in Meta-Analysis: A Case Study of Acupuncture Trials
Prady, S. L., Burch, J., Crouch, S., & MacPherson, H. (2013) · Acupuncture in Medicine · doi:10.1136/acupmed-2013-010364 · Cited by 9 · see citing works
No evidence for intervention-dependent influence of methodological features on treatment effect
Jacobs, W. C. H., Kruyt, M. C., Moojen, W. A., Verbout, A. J., & Öner, F. C. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2013.06.007 · Cited by 7 · see citing works
Yogi’s Headache: Chronic Tension-Type Headache
Ready, D. M., Dai, W., Keyser, L. K., & Cabret-Aymat, C. (2018) · Springer eBooks · doi:10.1007/978-3-319-91491-6_5 · Cited by 3 · see citing works
(2009) · Current Opinion in Neurology · doi:10.1097/wco.0b013e32832cf9cb
- Authors
- Verhagen, A. P., de Vet, H. C. W., Willemsen, S., & Stijnen, T.
- Affiliations
- Erasmus MC (NL)
- Leiden University Medical Center (NL)
- Corresponding author
- Verhagen, A.P.
- Venue
- Journal of Clinical Epidemiology
- Publisher
- Elsevier BV
- Published
- 2008-08
- Volume
- 61
- Issue
- 8
- Pages
- 813-818
- ISSN
- 0895-4356 (ISSN-L)
- OpenAlex
- W2038134238
- Cited by
- 14
- References
- 26
- Citations by year
- 2025: 1 · 2018: 1 · 2015: 1 · 2013: 3
- FWCI
- 3.03
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Clinical Epidemiology (publishedVersion)
- PubMed (publishedVersion)
- Pure Amsterdam UMC (publishedVersion)
- EUR Research Repository (Erasmus University Rotterdam) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Medicine › Psychiatry and Mental health › Migraine and Headache Studies
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Keywords
- tension-type headache, randomized controlled trials, design features, treatment effect size, conservative treatment, outcome measurement, meta-analysis bias, placebo comparison
- MeSH terms
- Data Interpretation, Statistical, Delphi Technique, Humans, Regression Analysis, Research Design, Review Literature as Topic, Meta-Analysis as Topic, Randomized Controlled Trials as Topic, Treatment Outcome, Tension-Type Headache
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review