The DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
The prevalence of and factors associated with inclusion of non-English language studies in Campbell systematic reviews: a survey and meta-epidemiological study
Open accessAbstract
BACKGROUND: Studies published in languages other than English are often neglected when research teams conduct systematic reviews. Literature on how to deal with non-English studies when conducting reviews have focused on the importance of including such studies, while less attention has been paid to the practical challenges of locating and assessing relevant non-English studies. We investigated the factors which might predict the inclusion of non-English studies in systematic reviews in the social sciences, to better understand how, when and why these are included/excluded. METHODS: We appraised all Campbell Collaboration systematic reviews (n = 123) published to July 2016, categorising each by its language inclusiveness. We sought additional information from review authors via a questionnaire and received responses concerning 47 reviews. Data were obtained for 17 factors and we explored correlations with the number of non-English studies in the reviews via statistical regression models. Additionally, we asked authors to identify factors that support or hinder the inclusion of non-English studies. RESULTS: Of 123 reviews, 108 did not explicitly exclude, and of these, 17 included non-English language studies. One factor correlated with the number of included non-English studies across all models: the number of countries in which the members of the review team work (B-value = 0.56; SE B = 0.24; 95% CI = 0.07-1.03; p = 0.02). This indicates that reviews which included non-English studies were more likely to be produced by international review teams. Our survey showed a dominance of researchers from English-speaking countries (52.9%) and review teams consisting only of team members from these countries (65.9%). The most frequently mentioned challenge to including non-English studies was a lack of resources (funding and time) followed by a lack of language resources (e.g. professional translators). CONCLUSION: Our findings may indicate a connection between the limited inclusion of non-English studies and a lack of resources, which forces review teams to rely on their limited language skills rather than the support of professional translators. If unaddressed, review teams risk ignoring key data and introduce bias in otherwise high-quality reviews. However, the validity and interpretation of our findings should be further assessed if we are to tackle the challenges of dealing with non-English studies.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Egger, M., Jüni, P., Bartlett, C., Holenstein, F., & Sterne, J. (2003) · Health Technology Assessment · doi:10.3310/hta7010 · Cited by 1,181
Dissemination and publication of research findings: an updated review of related biases
Song, F., Parekh, Hooper, L., Loke, Y., Ryder, J., Sutton, A., Hing, C., Kwok, C., Pang, C., & Harvey, I. (2010) · Health Technology Assessment · doi:10.3310/hta14080 · Cited by 1,039
Language bias in randomised controlled trials published in English and German
Egger, M., Zellweger-Zähner, T., Schneider, M., Junker, C., Lengeler, C., & Antes, G. (1997) · The Lancet · doi:10.1016/s0140-6736(97)02419-7 · Cited by 1,028
Direction and impact of language bias in meta-analyses of controlled trials: empirical study
Jüni, P., Holenstein, F., Sterne, J., Bartlett, C., & Egger, M. (2002) · International Journal of Epidemiology · doi:10.1093/ije/31.1.115 · Cited by 892
PRISMA for Abstracts: Reporting Systematic Reviews in Journal and Conference Abstracts
Beller, E. M., Glasziou, P. P., Altman, D. G., Hopewell, S., Bastian, H., Chalmers, I., Gøtzsche, P. C., Lasserson, T., Tovey, D., & for the PRISMA for Abstracts Group (2013) · PLoS Medicine · doi:10.1371/journal.pmed.1001419 · Cited by 706
Moher, D., Pham, B., Lawson, M., & Klassen, T. (2003) · Health Technology Assessment · doi:10.3310/hta7410 · Cited by 500
What contributions do languages other than English make on the results of meta-analyses?
Moher, D., Pham, Klassen, T. P., Schulz, K. F., Berlin, J. A., Jadad, A. R., & Liberati, A. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(00)00188-8 · Cited by 453
Searching for studies: a guide to information retrieval for Campbell systematic reviews
Kugley, S., Wade, A., Thomas, J., Mahood, Q., Jørgensen, A. K., Hammerstrøm, K., & Sathe, N. (2017) · Campbell Systematic Reviews · doi:10.4073/cmg.2016.1 · Cited by 371
Should we search Chinese biomedical databases when performing systematic reviews?
Cohen, J. F., Korevaar, D. A., Wang, J., Spijker, R., & Bossuyt, P. M. (2015) · Systematic Reviews · doi:10.1186/s13643-015-0017-3 · Cited by 76
Are randomized trials conducted in China or India biased? A comparative empirical analysis
Zhang, D., Freemantle, N., & Cheng, K. K. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.02.010 · Cited by 51
Cited by
Corazon, S. S., Sidenius, U., Poulsen, D. V., Gramkow, M. C., & Stigsdotter, U. K. (2019) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph16101711 · Cited by 269
The challenge of biased evidence in conservation
Christie, A. P., Amano, T., Martin, P. A., Petrovan, S. O., Shackelford, G. E., Simmons, B. I., Smith, R. K., Williams, D. R., Wordley, C. F. R., & Sutherland, W. J. (2021) · Conservation Biology · doi:10.1111/cobi.13577 · Cited by 202
Wittkowski, A., Vatter, S., Muhinyi, A., Garrett, C., & Henderson, M. (2020) · Clinical Psychology Review · doi:10.1016/j.cpr.2020.101906 · Cited by 121
Hennessy, E. A., Johnson, B. T., & Keenan, C. (2019) · Applied Psychology: Health and Well-Being · doi:10.1111/aphw.12169 · Cited by 91
“A uthor M anuscript A uthor M anuscript A uthor M anuscript A uthor M anuscript as it may increase bias and reduce precision of findings through the exclusion of useful and relevant data (Neimann Rasmussen & Montgomery, 2018).”
BackgroundLanguage models for data extraction and risk of bias assessment in complementary medicine
Lai, H., Liu, J., Bai, C., Liu, H., Pan, B., Luo, X., Hou, L., Zhao, W., Xia, D., Tian, J., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01457-w · Cited by 59
Metadata
- Authors
- Neimann Rasmussen, L., & Montgomery, P.
- Venue
- Systematic Reviews
- Published
- 2018-08-23
- Volume
- 7
- Issue
- 1
- Pages
- 129-129
- ISSN
- 2046-4053 (ISSN-L)
- OpenAlex
- W2888123127
- Cited by
- 204
- References
- 10
- FWCI
- 2.22
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The prevalence of and factors associated with inclusion of non-English language studies in Campbell systematic reviews: a survey and meta-epidemiological study. https://app.cochat.ai/papers/10.1186/s13643-018-0786-6
- The paper
- Neimann Rasmussen, L., & Montgomery, P. (2018). The prevalence of and factors associated with inclusion of non-English language studies in Campbell systematic reviews: a survey and meta-epidemiological study. Systematic Reviews. https://doi.org/10.1186/s13643-018-0786-6
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.