The inclusion of reports of randomised trials published in languages other than English in systematic reviews
Open accessThe DOI matches its Crossref record; 235 of the 236 references it cites were found in a registry or an index.
Abstract
OBJECTIVE: To assemble a large dataset of language restricted and language inclusive systematic reviews, including both conventional medicinal (CM) and complementary and alternative medicine (CAM) interventions. To then assess the quality of these reports by considering and comparing different types of systematic reviews and their associated RCTs; CM and CAM interventions; the effect of language restrictions compared with language inclusions, and whether these results are influenced by other issues, including statistical heterogeneity and publication bias, in the systematic review process. DATA SOURCES: MEDLINE, EMBASE, the Cochrane Database of Systematic Reviews and the Centralised Information Service for Complementary Medicine. REVIEW METHODS: Three types of systematic reviews were included: language restricted; language inclusive/English language (EL) reviews that searched RCTs in languages other than English (LOE) but did not find any and, hence, could not include any, in the quantitative data synthesis; and systematic reviews that searched for RCTs in LOE and included them in the quantitative data synthesis. Fisher's exact test was applied to compare the three different types of systematic reviews with respect to their reporting characteristics and the systematic review quality assessment tool. The odds ratio of LOE trials versus EL trials was computed for each review and this information was pooled across the reviews to examine the influence that language of publication and type of intervention (CM, CAM) have on the estimates of intervention effect. Several sensitivity analyses were performed. RESULTS: The LOE RCTs were predominantly in French and German. Language inclusive/LOE systematic reviews were of the highest quality compared with the other types of reviews. The CAM reviews were of higher quality compared with the CM reviews. There were only minor differences in the quality of reports of EL RCTs compared with the eight other languages considered. However, there are inconsistent differences in the quality of LOE reports depending on the intervention type. The results, and those reported previously, suggest that excluding reports of RCTs in LOE from the analytical part of a systematic review is reasonable. Because the present research and previous efforts have not included every type of CM RCT and the resulting possibility of the uncertainty as to when bias will be present by excluding LOE, it is always prudent to perform a comprehensive search for all evidence. This result only applies to reviews investigating the benefits of CM interventions. This does not imply that systematic reviewers should neglect reports in LOE. We recommend that systematic reviewers search for reports regardless of the language. There may be merit in including them in some aspects of the review process although this decision is likely to depend on several factors, including fiscal and other resources being available. Language restrictions significantly shift the estimates of an intervention's effectiveness when the intervention is CAM. Here, excluding trials reported in LOE, compared with their inclusion, resulted in a reduced intervention effect. The present results do not appear to be influenced by statistical heterogeneity and publication bias. CONCLUSIONS: With the exception of CAM systematic reviews, the quality of recently published systematic reviews is less than optimal. Language inclusive/LOE systematic reviews appear to be a marker for a better quality systematic review. Language restrictions do not appear to bias the estimates of a conventional intervention's effectiveness. However, there is substantial bias in the results of a CAM systematic review if LOE reports are excluded from it.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“There is mixed evidence for the potential impact of not including non-English-language trials: one study found that non-English trials were more likely to produce significant results at p <0.05 (Jüni et al., 2002), whereas an earlier review concluded that excluding non-English trials did not significantly affect the results of meta-analyses (Moher et al., 2003).”
From Self-help interventions to reduce self-stigma in people with mental health problems: A systematic literature review (2020) · cites this paper as Moher et al., 2003
In the citing paper
“In addition, it has been reported that language-restricted meta-analyses only minimally overestimate treatment effects (ϳ2% on average) compared with language-inclusive meta-analyses.112”
From Effectiveness of Manual Therapy and Therapeutic Exercise for Temporomandibular Disorders: Systematic Review and Meta-Analysis (2016) · cites this paper as [112]
In the citing paper
“Therefore, language-restricted metaanalyses do not appear to lead to biased estimates of intervention effectiveness.112,113”
From Effectiveness of Manual Therapy and Therapeutic Exercise for Temporomandibular Disorders: Systematic Review and Meta-Analysis (2016) · cites this paper as [112]
In the citing paper
“Over the last two decades, empirical evidence has accumulated from studies investigating different sources of bias related to specific SR tasks.”
From How to conduct systematic reviews more expeditiously? (2015) · cites this paper as [31]
In the citing paper
“Some authors suggested that the impact of excluding non-English language studies may depend on the topic of the review and the quality of non-English language studies [29, 31, 32].”
From How to conduct systematic reviews more expeditiously? (2015) · cites this paper as [31]
2 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
3 retracted
Daya, S., Maheshwari, A., Siristatidis, C. S., Bhattacharya, S., & Gibreel, A. (2000) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd001299 · Cited by 114
RetractedRice-based oral rehydration solution for treating diarrhoea
Fontaine, O., Gore, S., & Pierce, N. (1998) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd001264 · Cited by 79
RetractedAndrogens versus placebo or no treatment for idiopathic oligo/asthenospermia
Vandekerckhove, P., Lilford, R. J., Vail, A., & Hughes, E. G. (1996) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000150 · Cited by 30
RetractedThe Measurement of Observer Agreement for Categorical Data
Landis, J. R., & Koch, G. G. (1977) · Biometrics · doi:10.2307/2529310 · Cited by 81,148
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722
Meta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552
Quantifying heterogeneity in a meta‐analysis
Higgins, J. P. T., & Thompson, S. G. (2002) · Statistics in Medicine · doi:10.1002/sim.1186 · Cited by 36,763
Intraclass correlations: Uses in assessing rater reliability.
Shrout, P. E., & Fleiss, J. L. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.2.420 · Cited by 24,532
Assessing the quality of reports of randomized clinical trials: Is blinding necessary?
Jadad, A. R., Moore, R. A., Carroll, D., Jenkinson, C., Reynolds, D. J. M., Gavaghan, D. J., & McQuay, H. J. (1996) · Controlled Clinical Trials · doi:10.1016/0197-2456(95)00134-4 · Cited by 17,582
Eisenberg, D. M., Davis, R. B., Ettner, S. L., Appel, S., Wilkey, S., Van Rompay, M., & Kessler, R. C. (1998) · JAMA · doi:10.1001/jama.280.18.1569 · Cited by 6,658
Cited by
Systematic reviews: CRD's guidance for undertaking reviews in health care
Tacconelli, E. (2010) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(10)70065-7 · Cited by 3,245
2009 updated method guidelines for systematic reviews in the Cochrane Back Review Group
Furlan, A. D., Pennick, V., Bombardier, C., & van Tulder, M. (2009) · Spine · doi:10.1097/brs.0b013e3181b1c99f · Cited by 1,663
Morrison, A., Polisena, J., Husereau, D., Moulton, K., Clark, M., Fiander, M., Mierzwinski-Urban, M., Clifford, T., Hutton, B., & Rabb, D. (2012) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462312000086 · Cited by 1,257
“Two reports did not search EMBASE (11;15).”
BackgroundInfluentialEvidence summaries: the evolution of a rapid review approach
Khangura, S., Konnyu, K., Cushman, R., Grimshaw, J., & Moher, D. (2012) · Systematic Reviews · doi:10.1186/2046-4053-1-10 · Cited by 1,232
“Although exclusion of languages other than English is undesirable, it is considered to be a reasonable practice given limited time and resources and there is some evidence to suggest that it may not markedly bias review findings [9].”
BackgroundSelf-citationDissemination and publication of research findings: an updated review of related biases
Song, F., Parekh, Hooper, L., Loke, Y., Ryder, J., Sutton, A., Hing, C., Kwok, C., Pang, C., & Harvey, I. (2010) · Health Technology Assessment · doi:10.3310/hta14080 · Cited by 1,039
“The funnel plot asymmetry is not statistically significant for inception, regulatory, abstract cohort studies and manuscript cohort studies (see Figure 6). However, there are reasons to suspect the existence of publication and reporting bias in studies of meeting abstracts. A large number of reports of full publicatio…”
BackgroundChapter 10: Addressing reporting biases
Egger, M., & Moher, D. (2011) · Cited by 997
Self-citationExpediting systematic reviews: methods and implications of rapid reviews
Ganann, R., Ciliska, D., & Thomas, H. (2010) · Implementation Science · doi:10.1186/1748-5908-5-56 · Cited by 752
“found that language restricted meta-analyses did not differ significantly in intervention effectiveness estimates when compared to language inclusive meta-analyses [38,39].”
ResultO’Connor, S., Hanlon, P., O’Donnell, C. A., Garcia, S., Glanville, J., & Mair, F. S. (2016) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-016-0359-3 · Cited by 636
2015 Updated Method Guideline for Systematic Reviews in the Cochrane Back and Neck Group
Furlan, A. D., Malmivaara, A., Chou, R., Maher, C. G., Deyo, R. A., Schoene, M., Bronfort, G., & van Tulder, M. W. (2015) · Spine · doi:10.1097/brs.0000000000001061 · Cited by 704
Does Continuous Peripheral Nerve Block Provide Superior Pain Control to Opioids? A Meta-Analysis
Richman, J. M., Liu, S. S., Courpas, G., Wong, R., Rowlingson, A. J., McGready, J., Cohen, S. R., & Wu, C. L. (2006) · Anesthesia & Analgesia · doi:10.1213/01.ane.0000181289.09675.7d · Cited by 687
“This may introduce publication bias if only positive findings are published primarily in English language journals (25).”
BackgroundHobart, J., & Cano, S. (2009) · Health Technology Assessment · doi:10.3310/hta13120 · Cited by 621
Cooper, C., Booth, A., Varley-Campbell, J., Britten, N., & Garside, R. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0545-3 · Cited by 561
Searching for and selecting studies
Lefebvre, C., Glanville, J., Briscoe, S., Littlewood, A., Marshall, C., Metzendorf, M., Noel‐Storr, A., Rader, T., Shokraneh, F., Thomas, J., et al. (2019) · Cochrane Handbook for Systematic Reviews of Interventions · doi:10.1002/9781119536604.ch4 · Cited by 647
Early Prediction of Outcome of Activities of Daily Living After Stroke: A Systematic Review
Veerbeek, J. M., Kwakkel, G., van Wegen, E. E. H., Ket, J. C. F., & Heymans, M. W. (2011) · Stroke · doi:10.1161/strokeaha.110.604090 · Cited by 537
Nussbaumer-Streit, B., Klerings, I., Dobrescu, A. I., Persad, E., Stevens, A., Garritty, C., Kamel, C., Affengruber, L., King, V. J., & Gartlehner, G. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.10.011 · Cited by 440
Effectiveness of manual therapies: the UK evidence report
Bronfort, G., Haas, M., Evans, R., Leininger, B., & Triano, J. (2010) · Chiropractic & Osteopathy · doi:10.1186/1746-1340-18-3 · Cited by 434
“Including only English language reviews and trials may be considered another limitation of this report leading to language bias; however, the impact of excluding non-English trials from metaanalyses and systematic reviews is conflicting [319,320], and the incidence of randomized trials published in non-English journal…”
BackgroundArmijo-Olivo, S., Pitance, L., Singh, V., Neto, F., Thie, N., & Michelotti, A. (2016) · Physical Therapy · doi:10.2522/ptj.20140548 · Cited by 526
Dagenais, S., Tricco, A. C., & Haldeman, S. (2010) · The Spine Journal · doi:10.1016/j.spinee.2010.03.032 · Cited by 404
The efficacy and safety of exogenous melatonin for primary sleep disorders a meta-analysis
Buscemi, N., Vandermeer, B., Hooton, N., Pandya, R., Tjosvold, L., Hartling, L., Baker, G., Klassen, T. P., & Vohra, S. (2005) · Journal of General Internal Medicine · doi:10.1111/j.1525-1497.2005.0243.x · Cited by 399
“There is evidence of ‘‘reverse’’ publication bias, in that negative CAM studies are overrepresented in English-language journals and positive CAM studies are overrepresented in nonEnglish language journals.(45) We searched a number of electronic databases for nonEnglish language literature on melatonin and sleep disor…”
BackgroundSelf-citationNon-steroidal anti-inflammatory drugs for chronic low back pain
Enthoven, W. T., Roelofs, P. D., Deyo, R. A., van Tulder, M. W., & Koes, B. W. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012087 · Cited by 399
Diagnosing Acute Heart Failure in the Emergency Department: A Systematic Review and Meta‐analysis
Martindale, J. L., Wakai, A., Collins, S. P., Levy, P. D., Diercks, D., Hiestand, B. C., Fermann, G. J., deSouza, I., & Sinert, R. (2016) · Academic Emergency Medicine · doi:10.1111/acem.12878 · Cited by 368
Shah, P. S., & Shah, J. (2010) · Journal of Women's Health · doi:10.1089/jwh.2010.2051 · Cited by 327
“69 We did not include gray literature, abstracts, and proceedings, as the quality of such studies, particularly for observational association without peer review, could not be assessed adequately.”
Hartling, L., Featherstone, R., Nuspl, M., Shave, K., Dryden, D. M., & Vandermeer, B. (2017) · BMC Medical Research Methodology · doi:10.1186/s12874-017-0347-z · Cited by 324
“Language bias is also a concern for systematic reviewers, [15] although empiric evidence of the existence and direction of a bias is inconsistent when non-English language publications are excluded [6].”
BackgroundAlternative models of cardiac rehabilitation: A systematic review
Clark, R. A., Conway, A., Poulsen, V., Keech, W., Tirimacco, R., & Tideman, P. (2015) · European Journal of Preventive Cardiology · doi:10.1177/2047487313501093 · Cited by 298
Self-monitoring of blood glucose in type 2 diabetes: systematic review
Clar, C., Barnard, K., Cummins, E., Royle, P., & Waugh, N. (2010) · Health Technology Assessment · doi:10.3310/hta14120 · Cited by 274
Dobrescu, A., Nussbaumer-Streit, B., Klerings, I., Wagner, G., Persad, E., Sommer, I., Herkner, H., & Gartlehner, G. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.04.012 · Cited by 265
Lee, I., Agarwal, R. K., Lee, B. Y., Fishman, N. O., & Umscheid, C. A. (2010) · Infection Control & Hospital Epidemiology · doi:10.1086/657134 · Cited by 252
Decision Making in Health and Medicine: Integrating Evidence and Values
Hunink, M. G. M., Weinstein, M. C., Wittenberg, E., Drummond, M. F., Pliskin, J. S., Wong, J. B., & Glasziou, P. P. (2014) · doi:10.1017/cbo9781139506779 · Cited by 425
Ismail, K., Maissi, E., Thomas, S., Chalder, T., Schmidt, U., Bartlett, J., Patel, A., Dickens, C., Creed, F., & Treasure, J. (2010) · Health Technology Assessment · doi:10.3310/hta14220 · Cited by 245
Efficacy and safety of insulin analogues for the management of diabetes mellitus: a meta-analysis
Singh, S. R., Ahmad, F., Lal, A., Yu, C., Bai, Z., & Bennett, H. (2009) · Canadian Medical Association Journal · doi:10.1503/cmaj.081041 · Cited by 243
McDaid, C., Griffin, S., Weatherly, H., Durée, K., van der Burgt, M., van Hout, S., Akers, J., Davies, R., Sculpher, M., & Westwood, M. (2009) · Health Technology Assessment · doi:10.3310/hta13040 · Cited by 241
Christina, S., Magson, N. R., Kakar, V., & Rapee, R. M. (2021) · Clinical Psychology Review · doi:10.1016/j.cpr.2021.101979 · Cited by 241
Diriba, K., Awulachew, E., & Getu, E. (2020) · European Journal of Medical Research · doi:10.1186/s40001-020-00439-w · Cited by 237
Fuentes, J. P., Armijo Olivo, S., Magee, D. J., & Gross, D. P. (2010) · Physical Therapy · doi:10.2522/ptj.20090335 · Cited by 229
Sonnery-Cottet, B., Saithna, A., Quelard, B., Daggett, M., Borade, A., Ouanezar, H., Thaunat, M., & Blakeney, W. G. (2019) · British Journal of Sports Medicine · doi:10.1136/bjsports-2017-098401 · Cited by 228
“38 39 We excluded isolated case reports and articles which included patients with chronic conditions (eg, osteoarthritis) or major trauma (fracture, multiligament injury, neurovascular injury).”
BackgroundLourenco, T., Armstrong, N., N'Dow, J., Nabi, G., Deverill, M., Pickard, R., Vale, L., MacLennan, G., Fraser, C., McClinton, S., et al. (2008) · Health Technology Assessment · doi:10.3310/hta12350 · Cited by 221
Screening for Hepatocellular Carcinoma in Chronic Liver Disease: A Systematic Review
Kansagara, D., Papak, J., Pasha, A. S., O’Neil, M., Freeman, M., Relevo, R., Quiñones, A., Motu’apuaka, M., & Jou, J. H. (2014) · Annals of Internal Medicine · doi:10.7326/m14-0558 · Cited by 217
Conservative Management of Lumbar Disc Herniation With Associated Radiculopathy: A Systematic Review
Hahne, A. J., Ford, J. J., & McMeeken, J. M. (2010) · Spine · doi:10.1097/brs.0b013e3181cc3f56 · Cited by 216
Jolly, K., Taylor, R., Lip, G., Greenfield, S., Raftery, J., Mant, J., Lane, D., Jones, M., Lee, K., & Stevens, A. (2007) · Health Technology Assessment · doi:10.3310/hta11350 · Cited by 216
Pham, B., Klassen, T. P., Lawson, M. L., & Moher, D. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2004.08.021 · Cited by 206
“In the present series of three companion articles, we report that systematic reviews evaluating a CAM intervention included more RCTs than those evaluating a CM intervention [7,19].”
BackgroundResultInfluentialSelf-citationPrematunge, C., MacDougall, C., Johnstone, J., Adomako, K., Lam, F., Robertson, J., & Garber, G. (2016) · Infection Control & Hospital Epidemiology · doi:10.1017/ice.2015.228 · Cited by 212
“We limited our review to English language articles.(12)”
Neimann Rasmussen, L., & Montgomery, P. (2018) · Systematic Reviews · doi:10.1186/s13643-018-0786-6 · Cited by 204
Enestvedt, B. K., Tofani, C., Laine, L. A., Tierney, A., & Fennerty, M. B. (2012) · Clinical Gastroenterology and Hepatology · doi:10.1016/j.cgh.2012.08.029 · Cited by 200
Manchikanti, L. (2009) · Pain Physician · doi:10.36076/ppj.2009/12/35 · Cited by 190
Criteria for return to running after anterior cruciate ligament reconstruction: a scoping review
Rambaud, A. J. M., Ardern, C. L., Thoreux, P., Regnaux, J.-P., & Edouard, P. (2018) · British Journal of Sports Medicine · doi:10.1136/bjsports-2017-098602 · Cited by 185
Shaw, L., Rodgers, Price, C., van Wijck, F., Shackley, P., Steen, N., Barnes, M., Ford, G., & Graham, L. (2010) · Health Technology Assessment · doi:10.3310/hta14260 · Cited by 187
Mowatt, G., Zhu, S., Kilonzo, M., Boachie, C., Fraser, C., Griffiths, T., N'Dow, J., Nabi, G., Cook, J., & Vale, L. (2010) · Health Technology Assessment · doi:10.3310/hta14040 · Cited by 313
Rathod, S., Livergant, J., Klein, J., Witterick, I., & Ringash, J. (2015) · Oral Oncology · doi:10.1016/j.oraloncology.2015.07.002 · Cited by 186
“Restriction of our search to articles published in English is unlikely to significantly impact our findings [67,68].”
BackgroundSong, F., Parekh-Bhurke, S., Hooper, L., Loke, Y. K., Ryder, J. J., Sutton, A. J., Hing, C. B., & Harvey, I. (2009) · BMC Medical Research Methodology · doi:10.1186/1471-2288-9-79 · Cited by 175
Bussières, A. E., Stewart, G., Al-Zoubi, F., Decina, P., Descarreaux, M., Hayden, J., Hendrickson, B., Hincapié, C., Pagé, I., Passmore, S., et al. (2016) · Journal of Manipulative and Physiological Therapeutics · doi:10.1016/j.jmpt.2016.08.007 · Cited by 183
Manchikanti, L. (2009) · Pain Physician · doi:10.36076/ppj.2009/12/819 · Cited by 168
The International Caries Detection and Assessment System – ICDAS: A Systematic Review
Ekstrand, K. R., Gimenez, T., Ferreira, F. R., Mendes, F. M., & Braga, M. M. (2018) · Caries Research · doi:10.1159/000486429 · Cited by 179
“On the other hand, systematic reviews seem not to have been impacted when non-English studies were included [Juni et al., 2002; Moher et al., 2003].”
BackgroundBlood glucose self-monitoring in type 2 diabetes: a randomised controlled trial
Farmer, A., Wade, A., French, D., Simon, J., Yudkin, P., Gray, A., Craven, A., Goyder, L., Holman, R., Mant, D., et al. (2009) · Health Technology Assessment · doi:10.3310/hta13150 · Cited by 180
Kuratani, N., & Oi, Y. (2008) · Anesthesiology · doi:10.1097/aln.0b013e31817f5c18 · Cited by 179
Wang, D., Cummins, C., Bayliss, S., Sandercock, J., & Burls, A. (2008) · Health Technology Assessment · doi:10.3310/hta12360 · Cited by 175
Morrell, C., Warner, R., Slade, P., Dixon, S., Walters, S., Paley, G., & Brugha, T. (2009) · Health Technology Assessment · doi:10.3310/hta13300 · Cited by 176
Mamluk, L., Edwards, H. B., Savović, J., Leach, V., Jones, T., Moore, T. H. M., Ijaz, S., Lewis, S. J., Donovan, J. L., Lawlor, D., et al. (2017) · BMJ Open · doi:10.1136/bmjopen-2016-015410 · Cited by 176
McLean, G., Band, R., Saunderson, K., Hanlon, P., Murray, E., Little, P., McManus, R. J., Yardley, L., & Mair, F. S. (2016) · Journal of Hypertension · doi:10.1097/hjh.0000000000000859 · Cited by 168
“…blood pressure (SBP or included as they present the strongest level of evidence; we only considered studies published in Digital interventions in adults with hypertension Journal of Hypertension English as evidence suggests that limiting studies in this way does not introduce significant bias [21].”
BackgroundSchiftan, G. S., Ross, L. A., & Hahne, A. J. (2015) · Journal of Science and Medicine in Sport · doi:10.1016/j.jsams.2014.04.005 · Cited by 165
Hyde, J., Hankins, M., Deale, A., & Marteau, T. M. (2008) · Journal of Health Psychology · doi:10.1177/1359105308090933 · Cited by 164
Alebel, A., Tesema, C., Temesgen, B., Gebrie, A., Petrucka, P., & Kibret, G. D. (2018) · PLOS ONE · doi:10.1371/journal.pone.0199684 · Cited by 164
Pharmacotherapy for methamphetamine/amphetamine use disorder—a systematic review and meta‐analysis
Chan, B., Freeman, M., Kondo, K., Ayers, C., Montgomery, J., Paynter, R., & Kansagara, D. (2019) · Addiction · doi:10.1111/add.14755 · Cited by 160
“Our search was limited to English-language studies; however, the likelihood is low that the exclusion of non-English language studies would alter conclusions [44].”
Abubakar, I. I., Irvine, L., Aldus, C. F., Wyatt, G., Fordham, R. J., Schelenz, S., Shepstone, L., Howe, A., Peck, M. W., & Hunter, P. R. (2007) · Health Technology Assessment · doi:10.3310/hta11360 · Cited by 160
Bussières, A., Stewart, G., Zoubi, F. A., Decina, P., Descarreaux, M., Haskett, D., Hincapié, C. A., Pagé, I., Passmore, S. R., Srbely, J. Z., et al. (2018) · Journal of Manipulative and Physiological Therapeutics · doi:10.1016/j.jmpt.2017.12.004 · Cited by 178
Pharmacotherapy for Cocaine Use Disorder—a Systematic Review and Meta-analysis
Chan, B., Kondo, K., Freeman, M., Ayers, C., Montgomery, J., & Kansagara, D. (2019) · Journal of General Internal Medicine · doi:10.1007/s11606-019-05074-8 · Cited by 150
Shah, P. S. (2010) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2009.08.026 · Cited by 148
Chen, Y., Jowett, S. M., Barton, P. M., Malottki, K., Hyde, C., Gibbs, J. S. R., Pepke‐Żaba, J., Fry-Smith, A., Roberts, J., & Moore, D. (2009) · Health Technology Assessment · doi:10.3310/hta13490 · Cited by 153
Niebuur, J., van Lente, L., Liefbroer, A. C., Steverink, N., & Smidt, N. (2018) · BMC Public Health · doi:10.1186/s12889-018-6077-2 · Cited by 158
Andronis, L., Barton, P. M., & Bryan, S. (2009) · Health Technology Assessment · doi:10.3310/hta13290 · Cited by 175
Little, P., Turner, S., Rumsby, K., Warner, G., Moore, M., Lowes, J., Smith, H., Hawke, C., Turner, D., Leydon, G., et al. (2009) · Health Technology Assessment · doi:10.3310/hta13190 · Cited by 137
Rostom, A., Goldkind, L., & Laine, L. (2005) · Clinical Gastroenterology and Hepatology · doi:10.1016/s1542-3565(04)00777-3 · Cited by 134
Prevalence of depressive symptoms among medical students: overview of systematic reviews
Tam, W., Lo, K. K.-H., & Pacheco, J. P. G. (2018) · Medical Education · doi:10.1111/medu.13770 · Cited by 158
McIntosh, B., Cameron, C. G., Singh, S. R., Yu, C., Ahuja, T. K., Welton, N. J., & Dahl, M. (2011) · PubMed Central · Cited by 155
Husk, K., Lovell, R., Cooper, C., Stahl-Timmins, W., & Garside, R. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd010351.pub2 · Cited by 128
Walpole, S. C. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.03.004 · Cited by 147
How to conduct systematic reviews more expeditiously?
Tsertsvadze, A., Chen, Y.-F., Moher, D., Sutcliffe, P., & McCarthy, N. (2015) · Systematic Reviews · doi:10.1186/s13643-015-0147-7 · Cited by 104
“For example, Moher and colleagues found that in SRs of conventional interventions, language restriction did not alter the review results, whereas such restrictions resulted in a substantial change in the review results of complementary and alternative medicine interventions [31].”
BackgroundSelf-citationAmphetamines for attention deficit hyperactivity disorder (ADHD) in children and adolescents
Punja, S., Shamseer, L., Hartling, L., Urichuk, L., Vandermeer, B., Nikles, J., & Vohra, S. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd009996.pub2 · Cited by 122
Dorn, S. D., Kaptchuk, T. J., Park, J. B., Nguyen, L. T., Canenguez, K., Nam, B. H., Woods, K. B., Conboy, L. A., Stason, W. B., & Lembo, A. J. (2007) · Neurogastroenterology & Motility · doi:10.1111/j.1365-2982.2007.00937.x · Cited by 119
Blake-James, B. T., Rashidian, A., Ikeda, Y., & Emberton, M. (2006) · British Journal of Urology · doi:10.1111/j.1464-410x.2006.06574.x · Cited by 124
Yapp, E. J., & Quayle, E. (2018) · Aggression and Violent Behavior · doi:10.1016/j.avb.2018.05.002 · Cited by 119
Phillips, R. S., Friend, A. J., Gibson, F., Houghton, E., Gopaul, S., Craig, J. V., & Pizer, B. L. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd007786.pub3 · Cited by 135
Vaknin, Z., Kurzweil, Y., & Sherman, D. (2010) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2010.04.038 · Cited by 117
Developing efficient search strategies to identify reports of adverse effects in medline and embase
Golder, S., McIntosh, H. M., Duffy, S., & Glanville, J. (2006) · Health Information & Libraries Journal · doi:10.1111/j.1471-1842.2006.00634.x · Cited by 124
Wong, J. J., Shearer, H. M., Mior, S. A., Jacobs, C. L., Côté, P., Randhawa, K. A., Yu, H., Southerst, D., Varatharajan, S., Sutton, D. A., et al. (2015) · The Spine Journal · doi:10.1016/j.spinee.2015.08.024 · Cited by 146
Management of Bartholin Duct Cysts and Abscesses
Wechter, M. E., Wu, J. M., Marzano, D., & Haefner, H. K. (2009) · Obstetrical & Gynecological Survey · doi:10.1097/ogx.0b013e31819f9c76 · Cited by 115
Jacobs, M., Macefield, R. C., Elbers, R. G., Sitnikova, K., Korfage, I. J., Smets, E. M. A., Henselmans, I., Henegouwen, M. I. V. B., de Haes, J. C. J. M., Blazeby, J. M., et al. (2013) · Quality of Life Research · doi:10.1007/s11136-013-0545-z · Cited by 123
“Third, we omitted non-English studies, because we believe the potential impact of this selection criterion to be limited [68].”
MethodAbdulla, S., Southerst, D., Côté, P., Shearer, H. M., Sutton, D. A., Randhawa, K. A., Varatharajan, S., Wong, J. J., Yu, H., Marchand, A.-A., et al. (2015) · Manual Therapy · doi:10.1016/j.math.2015.03.013 · Cited by 138
A systematic review of presumed consent systems for deceased organ donation
Rithalia, A. D., McDaid, C., Suekarran, S., Norman, G., Myers, L., & Sowden, A. J. (2009) · Health Technology Assessment · doi:10.3310/hta13260 · Cited by 112
Gowers, S., Clark, A., Roberts, C., Byford, S., Barrett, B. M., Griffiths, A., Edwards, V., Bryan, C., Smethurst, N., Rowlands, L., et al. (2010) · Health Technology Assessment · doi:10.3310/hta14150 · Cited by 126
Local anaesthetic for post‐tonsillectomy pain: a systematic review and meta‐analysis
Grainger, J., & Saravanappa, N. (2008) · Clinical Otolaryngology · doi:10.1111/j.1749-4486.2008.01815.x · Cited by 114
Performance of screening tests for child physical abuse in accident and emergency departments
Woodman, J., Pitt, M., Wentz, R., Taylor, B., Hodes, D., & Gilbert, R. E. (2008) · Health Technology Assessment · doi:10.3310/hta12330 · Cited by 115
Ibrahim, A., Ahmed, M., Conway, R. P., & CAREY, J. J. (2018) · Journal of Clinical Medicine · doi:10.3390/jcm8010015 · Cited by 106
Gagnier, J. J., Mullins, M. A., Huang, H., Marinac‐Dabic, D., Ghambaryan, A., Eloff, B. C., Mirza, F. M., & Bayona, M. (2017) · The Journal of Arthroplasty · doi:10.1016/j.arth.2016.12.052 · Cited by 106
VenUS II: a randomised controlled trial of larval therapy in the management of leg ulcers
Dumville, J. C., Worthy, G., Soares, M. O., Bland, J., Cullum, N. A., Dowson, C. G., Iglesias, C., McCaughan, D., Mitchell, J., Nelson, E. A., et al. (2009) · Health Technology Assessment · doi:10.3310/hta13550 · Cited by 105
Corso, E., Hind, D., Beever, D., Fuller, G. W., Wilson, M., Wrench, I. J., & Chambers, D. (2017) · BMC Pregnancy and Childbirth · doi:10.1186/s12884-017-1265-0 · Cited by 119
“The exclusion of non-English language papers from systematic reviews is not thought to bias estimates of effectiveness for most types of intervention [90].”
BackgroundAlebel, A., Kibret, G. D., Wagnew, F., Leshargie, C. T., Ferede, A., Petrucka, P. M., Bobo, F. T., Birhanu, M. Y., Tadesse, A. A., & Eshetie, S. (2018) · Reproductive Health · doi:10.1186/s12978-018-0637-x · Cited by 101
“All searches were limited to articles written in English which does not alter the outcome of the systematic reviews and meta-analyses [23].”
BackgroundZoubi, F. A., Menon, A., Mayo, N. E., & Bussières, A. (2018) · BMC Health Services Research · doi:10.1186/s12913-018-3253-0 · Cited by 108
Mowatt, G., Houston, J. G., Hernández, R. A., de Verteuil, R., Fraser, C. M., Cuthbertson, B. H., & Vale, L. D. (2009) · Health Technology Assessment · doi:10.3310/hta13070 · Cited by 94
Lefebvre, C., Eisinga, A., McDonald, S., & Paul, N. (2008) · Emerging Themes in Epidemiology · doi:10.1186/1742-7622-5-13 · Cited by 93
“Although there is evidence that exclusion of studies in languages other than English from reviews might make no significant difference to the overall estimates of the effects of treatments [5-7], some subject areas (for example, complementary and alternative medicine) have been shown to require a more comprehensive se…”
Prime, H., Andrews, K., Markwell, A., González, A., Janus, M., Tricco, A. C., Bennett, T. A., & Atkinson, L. R. (2023) · Clinical Child and Family Psychology Review · doi:10.1007/s10567-022-00423-2 · Cited by 157
Metadata
- Authors
- Moher, D., Pham, B., Lawson, M., & Klassen, T.
- Affiliations
- University of Ottawa (CA)
- Venue
- Health Technology Assessment
- Published
- 2003-12
- Volume
- 7
- Issue
- 41
- Pages
- 1-90
- ISSN
- 1366-5278 (ISSN-L)
- 2046-4924
- OpenAlex
- W2119404638
- Cited by
- 500
- References
- 236
- Citations by year
- 2026: 9 · 2025: 11 · 2024: 18 · 2023: 13 · 2022: 33 · 2021: 27 · 2020: 26 · 2019: 26 · 2018: 27 · 2017: 18 · 2016: 22 · 2015: 44 · 2014: 19 · 2013: 24 · 2012: 16
- FWCI
- 2.71
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Health Technology Assessment (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Health Technology Assessment programme (96/52/99)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Keywords
- randomized controlled trials, systematic review quality, complementary and alternative medicine, publication bias, statistical heterogeneity, language of publication, bias in systematic reviews, Cochrane Collaboration
- MeSH terms
- Complementary Therapies, United Kingdom, Humans, Language, Publishing, Randomized Controlled Trials as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.