THE EFFECT OF ENGLISH-LANGUAGE RESTRICTION ON SYSTEMATIC REVIEW-BASED META-ANALYSES: A SYSTEMATIC REVIEW OF EMPIRICAL STUDIES
The DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Abstract
Objectives: The English language is generally perceived to be the universal language of science. However, the exclusive reliance on English-language studies may not represent all of the evidence. Excluding languages other than English (LOE) may introduce a language bias and lead to erroneous conclusions. Study Design and Setting: We conducted a comprehensive literature search using bibliographic databases and grey literature sources. Studies were eligible for inclusion if they measured the effect of excluding randomized controlled trials (RCTs) reported in LOE from systematic review-based meta-analyses (SR/MA) for one or more outcomes. Results: None of the included studies found major differences between summary treatment effects in English-language restricted meta-analyses and LOE-inclusive meta-analyses. Findings differed about the methodological and reporting quality of trials reported in LOE. The precision of pooled estimates improved with the inclusion of LOE trials. Conclusions: Overall, we found no evidence of a systematic bias from the use of language restrictions in systematic review-based meta-analyses in conventional medicine. Further research is needed to determine the impact of language restriction on systematic reviews in particular fields of medicine.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Exclusion of non-English literature does not cause systematic bias ( Morrison et al., 2012 ) and the proper and reliable translation from other languages into English may be problematic ( Jackson and Kuriyama, 2019 ).”
From The effect of contextual interference on transfer in motor learning - a systematic review and meta-analysis (2024) · cites this paper as Morrison et al., 2012
In the citing paper
“we excluded studies published in languages other than English, although there is some evidence that this is unlikely to be a major limitation [211].”
From Using Normalization Process Theory in feasibility studies and process evaluations of complex healthcare interventions: a systematic review (2018) · cites this paper as [211]
In the citing paper
“However, the evidence whether or not the exclusion of non-English language study publications of conventional healthcare interventions introduces bias has been inconsistent, some authors showing meta-analyses of only English language studies yielding more conservative estimates [29], and others not demonstrating the presence of any difference [30, 32, 47].”
From How to conduct systematic reviews more expeditiously? (2015) · cites this paper as [47]
In the citing paper
“However, studies have shown that the effect may be small in general but may be difficult to measure for an individual systematic review [183,184].”
From Reliability and Validity of Commercially Available Wearable Devices for Measuring Steps, Energy Expenditure, and Heart Rate: Systematic Review (2020) · cites this paper as [183]
In the citing paper
“There is some potential for bias when including only English language studies in systematic reviews.”
From Reliability and Validity of Commercially Available Wearable Devices for Measuring Steps, Energy Expenditure, and Heart Rate: Systematic Review (2020) · cites this paper as [183]
Citations
Cites
Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048 · cited 6×
“Comprehensive searches to identify all relevant studies and minimize biases are essential for systematic reviews (1;8).”
BackgroundMethodInfluentialAssessing the quality of reports of randomized clinical trials: Is blinding necessary?
Jadad, A. R., Moore, R. A., Carroll, D., Jenkinson, C., Reynolds, D. J. M., Gavaghan, D. J., & McQuay, H. J. (1996) · Controlled Clinical Trials · doi:10.1016/0197-2456(95)00134-4 · Cited by 17,582
Downs, S. H., & Black, N. (1998) · Journal of Epidemiology and Community Health · doi:10.1136/jech.52.6.377 · Cited by 8,252 · cited 1×
“Quality Assessment A checklist (3) validated for human analytic studies was adapted for this review and applied by two reviewers (A.”
MethodCochrane Handbook for Systematic Reviews of Interventions
Tarsilla, M. (2010) · Journal of MultiDisciplinary Evaluation · doi:10.56645/jmde.v6i14.284 · Cited by 5,178
Moher, D., Pham, B., Jones, A., Cook, D. J., Jadad, A. R., Moher, M., Tugwell, P., & Klassen, T. P. (1998) · The Lancet · doi:10.1016/s0140-6736(98)01085-x · Cited by 3,560
Egger, M., Jüni, P., Bartlett, C., Holenstein, F., & Sterne, J. (2003) · Health Technology Assessment · doi:10.3310/hta7010 · Cited by 1,181
Language bias in randomised controlled trials published in English and German
Egger, M., Zellweger-Zähner, T., Schneider, M., Junker, C., Lengeler, C., & Antes, G. (1997) · The Lancet · doi:10.1016/s0140-6736(97)02419-7 · Cited by 1,028 · cited 2×
“The potential for this type of bias in English-language only study selection is called a “Tower of Babel” bias (7) or “English-language” bias (5).”
BackgroundDirection and impact of language bias in meta-analyses of controlled trials: empirical study
Jüni, P., Holenstein, F., Sterne, J., Bartlett, C., & Egger, M. (2002) · International Journal of Epidemiology · doi:10.1093/ije/31.1.115 · Cited by 892 · cited 12×
“Systematic reviews in four of the reports considered RCTs published in German, French, Italian, and Spanish (4;11;12;15).”
BackgroundResultInfluentialMoher, D., Fortin, P., Jadad, A. R., Jüni, P., Klassen, T., Le Lorier, J., Liberati, A., Penna, A., & Linde, K. (1996) · The Lancet · doi:10.1016/s0140-6736(96)90538-3 · Cited by 537 · cited 1×
“(13;14) detected no differences in the reporting of randomization, double-blinding, dropouts, withdrawals, and allocation concealment.”
BackgroundGrégoire, G., Derderian, F., & Le Lorier, J. (1995) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(94)00098-b · Cited by 526 · cited 5×
“Bias may lead to an over- or underestimation of an intervention’s effectiveness, and ultimately, to inappropriate health policy decisions or patient care (7).”
BackgroundInfluential
Research works printed but not linked: 5
- 2007, Conventional medicine
- Morrison, 2009, English-language restriction when conducting systematic review-based metaanalyses: Systematic review of published studies
- 2009, Systematic reviews: CRD's guidance for undertaking reviews in health care
- Deeks, 2011, Cochrane handbook for systematic reviews of interventions Version 5.1.0
- 2005, Sci Watch, 16
Cited by
Siddaway, A. P., Wood, A. M., & Hedges, L. V. (2019) · Annual Review of Psychology · doi:10.1146/annurev-psych-010418-102803 · Cited by 2,550
Jeong, W., Keighley, C., Wolfe, R., Lee, W. L., Slavin, M. A., Kong, D. C. M., & Chen, S. C.-A. (2019) · Clinical Microbiology and Infection · doi:10.1016/j.cmi.2018.07.011 · Cited by 1,082
Rueger, S. Y., Malecki, C. K., Pyun, Y., Aycock, C., & Coyle, S. (2016) · Psychological Bulletin · doi:10.1037/bul0000058 · Cited by 946
Chrvala, C. A., Sherr, D., & Lipman, R. D. (2016) · Patient Education and Counseling · doi:10.1016/j.pec.2015.11.003 · Cited by 914
Increasing value and reducing waste: addressing inaccessible research
Chan, A.-W., Song, F., Vickers, A., Jefferson, T., Dickersin, K., Gøtzsche, P. C., Krumholz, H. M., Ghersi, D., & van der Worp, H. B. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62296-5 · Cited by 887
Metadata
- Authors
- Morrison, A., Polisena, J., Husereau, D., Moulton, K., Clark, M., Fiander, M., Mierzwinski-Urban, M., Clifford, T., Hutton, B., & Rabb, D.
- Affiliations
- Canada's Drug Agency (CA)
- University of Ottawa (CA)
- Cochrane (BR)
- Ottawa Hospital Research Institute (CA)
- ORCID
- Venue
- International Journal of Technology Assessment in Health Care
- Publisher
- Cambridge University Press (CUP)
- Published
- 2012-04
- Volume
- 28
- Issue
- 2
- Pages
- 138-144
- ISSN
- 0266-4623 (ISSN-L)
- 1471-6348
- OpenAlex
- W2098995101
- Cited by
- 1,257
- References
- 16
- Citations by year
- 2026: 114 · 2025: 140 · 2024: 128 · 2023: 117 · 2022: 136 · 2021: 163 · 2020: 115 · 2019: 104 · 2018: 53 · 2017: 57 · 2016: 51 · 2015: 45 · 2014: 22 · 2013: 9 · 2012: 2
- FWCI
- 6.87
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- International Journal of Technology Assessment in Health Care (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Keywords
- language bias, meta-analysis, randomized controlled trials, methodological quality of trials, trial reporting quality
- MeSH terms
- Data Collection, Humans, Language, Publishing, Review Literature as Topic, Meta-Analysis as Topic, Databases, Bibliographic, Databases, Factual, Publication Bias, Information Dissemination
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: THE EFFECT OF ENGLISH-LANGUAGE RESTRICTION ON SYSTEMATIC REVIEW-BASED META-ANALYSES: A SYSTEMATIC REVIEW OF EMPIRICAL STUDIES. https://app.cochat.ai/papers/10.1017/s0266462312000086
- The paper
- Morrison, A., Polisena, J., Husereau, D., Moulton, K., Clark, M., Fiander, M., Mierzwinski-Urban, M., Clifford, T., Hutton, B., & Rabb, D. (2012). THE EFFECT OF ENGLISH-LANGUAGE RESTRICTION ON SYSTEMATIC REVIEW-BASED META-ANALYSES: A SYSTEMATIC REVIEW OF EMPIRICAL STUDIES. International Journal of Technology Assessment in Health Care. https://doi.org/10.1017/s0266462312000086
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.