The methodological and reporting quality of systematic reviews from China and the USA are similar
Open accessAbstract
OBJECTIVES: To compare the methodological and reporting quality of systematic reviews by authors from China and those from the United States (USA). STUDY DESIGN AND SETTING: From systematic reviews of randomized trials published in 2014 in English, we randomly selected 100 from China and 100 from the USA. The methodological quality was assessed using the Assessing the Methodological Quality of Systematic Reviews (AMSTAR) tool, and reporting quality assessed using the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) tool. RESULTS: Compared with systematic reviews from the USA, those from China were more likely to be a meta-analysis, published in low-impact journals, and a non-Cochrane review. The mean summary Assessing the Methodological Quality of Systematic Reviews score was 6.7 (95% confidence interval: 6.5, 7.0) for reviews from China and 6.6 (6.1, 7.1) for reviews from the USA, and the mean summary Preferred Reporting Items for Systematic Reviews and Meta-analyses score was 21.2 (20.7, 21.6) for reviews from China and 20.6 (19.9, 21.3) for reviews from the USA. The differences in summary quality scores between China and the USA were statistically nonsignificant after adjusting for multiple review factors. CONCLUSION: The overall methodological and reporting quality of systematic reviews by authors from China are similar to those from the USA, although the quality of systematic reviews from both countries could be further improved.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Two researchers independently searched the database.
“Two researchers independently searched the database at the Center for Evidence-Based Medicine using the following specific strategies (Tian et al., 2017):”
From Characteristics and trends in acceptance and commitment therapy research: A bibliometric analysis (2022) · cites this paper as Tian et al., 2017
Section 2.1 states that one reviewer (F.S.) searched PubMed, contrary to the claim that two researchers independently searched the database. Section 2.2 describes a second reviewer checking extracted data, not conducting an independent search. The paper does not identify the searched database as one at the Center for Evidence-Based Medicine. (LLM, conf=0.94)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164 · see citing works
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.06.006 · Cited by 10,578 · see citing works
Development of AMSTAR: A measurement tool to assess the methodological quality of systematic reviews
Shea, B. J., Grimshaw, J. M., Wells, G. A., Boers, M., Andersson, N., Hamel, C., Porter, A. C., Tugwell, P., Moher, D., & Bouter, L. M. (2007) · BMC Medical Research Methodology · doi:10.1186/1471-2288-7-10 · Cited by 4,374 · see citing works
Systematic Reviews: Synthesis of Best Evidence for Clinical Decisions
Cook, D. J., Mulrow, C. D., & Haynes, R. B. (1997) · Annals of Internal Medicine · doi:10.7326/0003-4819-126-5-199703010-00006 · Cited by 2,184 · see citing works
Shea, B. J., Hamel, C., Wells, G. A., Bouter, L. M., Kristjansson, E., Grimshaw, J., Henry, D. A., & Boers, M. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.10.009 · Cited by 1,755 · see citing works
How to increase value and reduce waste when research priorities are set
Chalmers, I., Bracken, M. B., Djulbegovic, B., Garattini, S., Grant, J., Gülmezoglu, A. M., Howells, D. W., Ioannidis, J. P. A., & Oliver, S. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62229-1 · Cited by 1,436 · see citing works
Seventy-five trials and eleven systematic reviews a day: how will we ever keep up?
Bastian, H., Glasziou, P., & Chalmers, I. (2010) · PLoS Medicine · doi:10.1371/journal.pmed.1000326 · Cited by 1,188 · see citing works
Epidemiology and reporting characteristics of systematic reviews.
Moher, D., Tetzlaff, J., Tricco, A. C., Sampson, M., & Altman, D. G. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040078 · Cited by 891 · cited 1× · see citing works
“The reporting quality of a systematic review refers to the clarity and transparency of its reporting, and poor reporting reduces the value and usefulness of systematic reviews [22].”
BackgroundPage, M. J., Shamseer, L., Altman, D. G., Tetzlaff, J., Sampson, M., Tricco, A. C., Catalá-López, F., Li, L., Reid, E. K., Sarkis-Onofre, R., et al. (2016) · PLOS Medicine · doi:10.1371/journal.pmed.1002028 · Cited by 768 · see citing works
Do Certain Countries Produce Only Positive Results? A Systematic Review of Controlled Trials
Vickers, A. J., Goyal, N., Harland, R., & Rees, R. W. (1998) · Controlled Clinical Trials · doi:10.1016/s0197-2456(97)00150-5 · Cited by 684 · see citing works
A brief history of research synthesis
Chalmers, I., Hedges, L. V., & Cooper, H. (2002) · Evaluation & the Health Professions · doi:10.1177/0163278702025001003 · Cited by 552 · see citing works
Murad, M. H., Montori, V. M., Ioannidis, J. P. A., Jaeschke, R., Devereaux, P. J., Prasad, K., Neumann, I., Carrasco-Labra, A., Agoritsas, T., Hatala, R., et al. (2014) · JAMA · doi:10.1001/jama.2014.5559 · Cited by 454 · see citing works
The quality of evidence in Chinese meta-analyses needs to be improved
Yao, L., Sun, R., Chen, Y., Wang, Q., Wei, D., Wang, X., & Yang, K. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2016.01.003 · Cited by 181 · see citing works
Self-citationSystematic review found AMSTAR, but not R(evised)-AMSTAR, to have good measurement properties
Pieper, D., Büchter, R. B., Li, L., Prediger, B., & Eikermann, M. (2014) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2014.12.009 · Cited by 160 · see citing works
Mallett, S. V. (2003) · BMJ Evidence-Based Medicine · doi:10.1136/ebm.8.4.100 · Cited by 71 · see citing works
Helfer, B., Prosser, A., Samara, M. T., Geddes, J. R., Cipriani, A., Davis, J. M., Mavridis, D., Salanti, G., & Leucht, S. (2015) · BMC Medicine · doi:10.1186/s12916-015-0317-4 · Cited by 63 · see citing works
The Geometric Increase in Meta-Analyses from China in the Genomic Era
Ioannidis, J. P. A., Chang, C. Q., Lam, T. K., Schully, S. D., & Khoury, M. J. (2013) · PLoS ONE · doi:10.1371/journal.pone.0065602 · Cited by 62 · see citing works
A Case Study of Discordant Overlapping Meta-Analyses: Vitamin D Supplements and Fracture
Bolland, M. J., & Grey, A. B. (2014) · PLoS ONE · doi:10.1371/journal.pone.0115934 · Cited by 60 · see citing works
Petticrew, M. P. (2002) · BMJ · doi:10.1136/bmj.324.7336.545/a · Cited by 59 · see citing works
Riaz, I. B., Khan, M. S., Riaz, H., & Goldberg, R. J. (2015) · The American Journal of Medicine · doi:10.1016/j.amjmed.2015.10.009 · Cited by 59 · see citing works
Ma, B., Guo, J., Qi, G., Li, H., Peng, J., Zhang, Y., Ding, Y., & Yang, K. (2011) · PLoS ONE · doi:10.1371/journal.pone.0020185 · Cited by 56 · see citing works
Self-citationAre randomized trials conducted in China or India biased? A comparative empirical analysis
Zhang, D., Freemantle, N., & Cheng, K. K. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.02.010 · Cited by 51 · see citing works
Susan Mallett, D., & Mike Clarke, D. (2003) · ACP Journal Club · doi:10.7326/acpjc-2003-139-1-a11 · Cited by 37 · see citing works
Quality of reporting of systematic reviews published in “evidence-based” Chinese journals
Li, J.-L., Ge, L., Ma, J., Zeng, Q.-L., Lu, Y., An, N., Ding, J.-X., Gan, Y., & Tian, J. (2014) · Systematic Reviews · doi:10.1186/2046-4053-3-58 · Cited by 33 · see citing works
Self-citationMa, B., Qi, G.-Q., Lin, X., Wang, T., Chen, Z., & Yang, K. (2012) · The Journal of Alternative and Complementary Medicine · doi:10.1089/acm.2011.0274 · Cited by 24 · see citing works
Self-citationFan, H., & Song, F. (2015) · Scientific Reports · doi:10.1038/srep13221 · Cited by 24 · see citing works
Self-citationWhat is the future of biomedical research?
Tebala, G. D. (2015) · Medical Hypotheses · doi:10.1016/j.mehy.2015.07.003 · Cited by 22 · see citing works
Cochrane Collaboration and the evolution of evidence
Summerskill, W. S. M. (2005) · The Lancet · doi:10.1016/s0140-6736(05)67710-0 · Cited by 19 · see citing works
Jin, Y., Ma, E., Gao, W., Wei, H., & Dou, H. (2013) · International Journal of Nursing Practice · doi:10.1111/ijn.12123 · Cited by 19 · see citing works
Channeling the flood of meta-analyses
Fuhr, U., & Hellmich, M. (2015) · European Journal of Clinical Pharmacology · doi:10.1007/s00228-015-1838-7 · Cited by 10 · see citing works
Cochrane Collaboration in China 1996–2013
Zhang, M., & Li, Y. (2014) · Journal of Evidence-Based Medicine · doi:10.1111/jebm.12087 · Cited by 3 · see citing works
Editorial Commentary: Meta‐analyses Are on the Rise, but Are We Learning More?
D’Agostino, R. B. (2016) · Arthroscopy The Journal of Arthroscopic and Related Surgery · doi:10.1016/j.arthro.2015.12.036 · Cited by 3 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Page, M. J., & Moher, D. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0663-8 · Cited by 769 · see citing works
Hoffmann, F., Allers, K., Rombey, T., Helbach, J., Hoffmann, A., Mathes, T., & Pieper, D. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.05.022 · Cited by 278 · see citing works
Yan, P., Yao, L., Li, H., Zhang, M., Xun, Y., Li, M., Cai, H., Lu, C., Hu, L., Guo, T., et al. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2018.12.013 · Cited by 107 · see citing works
Wang, L., Wang, X., Zhang, G., Ma, Y., Zhang, Q., Li, Z., Ran, J., Hou, X., Geng, Y., Yang, Z., et al. (2021) · Radiation Oncology · doi:10.1186/s13014-021-01899-y · Cited by 48 · see citing works
Characteristics and trends in acceptance and commitment therapy research: A bibliometric analysis
Li, Z., Shang, W., Wang, C., Yang, K., & Guo, J. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.980848 · Cited by 36 · see citing works
“Two researchers independently searched the database at the Center for Evidence-Based Medicine using the following specific strategies (Tian et al., 2017): Topic: (“Acceptance and Commitment therap*” OR “ACT*” OR “Commitment and acceptance method*” OR “Commitment Acceptance therap*”), spanning the period 1980 to Januar…”
MethodSelf-citationQuality of abstract of systematic reviews and meta-analyses in paediatric dentistry journals
Pulikkotil, S. J., Jayaraman, J., & Nagendrababu, V. (2019) · European Archives of Paediatric Dentistry · doi:10.1007/s40368-019-00432-w · Cited by 21 · see citing works
Predictors of Higher Quality of Systematic Reviews Addressing Nutrition and Cancer Prevention
Storman, D., Koperny, M., Zając, J., Polak, M., Weglarz, P., Bochenek-Cibor, J., Swierz, M. J., Staskiewicz, W., Gorecka, M., Skuza, A., et al. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19010506 · Cited by 6 · see citing works
- Authors
- Tian, J., Zhang, J., Ge, L., Yang, K., & Song, F.
- Affiliations
- Lanzhou University (CN)
- Gansu University of Traditional Chinese Medicine (CN)
- University of East Anglia (GB)
- Corresponding author
- Yang, Kehu
- Song, Fujian
- Venue
- Journal of Clinical Epidemiology
- Publisher
- Elsevier BV
- Published
- 2017-05
- Volume
- 85
- Pages
- 50-58
- ISSN
- 0895-4356 (ISSN-L)
- OpenAlex
- W2568362468
- Cited by
- 248
- References
- 32
- Citations by year
- 2026: 6 · 2025: 9 · 2024: 16 · 2023: 18 · 2022: 25 · 2021: 51 · 2020: 50 · 2019: 58 · 2018: 10 · 2017: 5
- FWCI
- 8.28
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Journal of Clinical Epidemiology (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- UEA Digital Repository (University of East Anglia) (acceptedVersion)
- Lanzhou University Institutional Repository (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- systematic reviews, methodological quality, reporting quality, AMSTAR, PRISMA, cross-national comparison, China, United States, meta-analysis, journal impact
- MeSH terms
- China, Humans, Review Literature as Topic, United States, Epidemiologic Research Design, Randomized Controlled Trials as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
