The DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Assessing the Quality of Studies in Meta-Research: Review/Guidelines on the Most Important Quality Assessment Tools
Abstract
Summary: Systematic reviews and meta‐analyses pool data from individual studies to generate a higher level of evidence to be evaluated by guidelines. These reviews ultimately guide clinicians and stakeholders in health‐related decisions. However, the informativeness and quality of evidence synthesis inherently depend on the quality of what has been pooled into meta‐research projects. Moreover, beyond the quality of included individual studies, only a methodologically correct process, in relation to systematic reviews and meta‐analyses themselves, can produce a reliable and valid evidence synthesis. Hence, quality of meta‐research projects also affects evidence synthesis reliability. In this overview, the authors provide a synthesis of advantages and disadvantages and main characteristics of some of the most frequently used tools to assess quality of individual studies, systematic reviews, and meta‐analyses. Specifically, the tools considered in this work are the Newcastle‐Ottawa scale (NOS) and the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) for observational studies, the Consolidated Standards of Reporting Trials (CONSORT), the Jadad scale, the Cochrane risk of bias tool 2 (RoB2) for randomized controlled trials, the Preferred Reporting Items for Systematic Reviews and Meta‐analysis (PRISMA) and the Assessment of Multiple Systematic Reviews 2 (AMSTAR2), and AMSTAR‐PLUS for meta‐analyses. What is already known?: The informativeness and quality of evidence synthesis inherently depend on the quality of what has been pooled into meta‐research projects. Beyond the quality of included individual studies, only a methodologically correct process, in relation to systematic reviews and meta‐analyses themselves, can produce a reliable and valid evidence synthesis. What is new?: In this overview, the authors provide a synthesis of advantages and disadvantages and main characteristics of some of the most frequently used tools to assess quality of individual studies, systematic reviews, and meta‐analyses. Potential impact: This overview serves as a starting point and a brief guide to identify and understand the main and most frequently used tools for assessing the quality of studies included in meta‐research. The authors here share their experience in publishing several meta‐research‐related articles covering different areas of medical sciences.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Several long-term imaging studies have demonstrated structural changes in the brain in response to concussion, including altered neurite density in sleep-related white matter tracts [48,60].”
From Concussion Disrupts Sleep in Adolescents: A Systematic Review and Meta-Analysis (2025) · cites this paper as [60]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
RoB 2: a revised tool for assessing risk of bias in randomised trials
Sterne, J. A. C., Savović, J., Page, M. J., Elbers, R. G., Blencowe, N. S., Boutron, I., Cates, C. J., Cheng, H.-Y., Corbett, M. S., Eldridge, S. M., et al. (2019) · BMJ · doi:10.1136/bmj.l4898 · Cited by 34,787
The Cochrane Collaboration's tool for assessing risk of bias in randomised trials
Higgins, J. P. T., Altman, D. G., Gotzsche, P. C., Juni, P., Moher, D., Oxman, A. D., Savovic, J., Schulz, K. F., Weeks, L., Sterne, J. A. C., et al. (2011) · BMJ · doi:10.1136/bmj.d5928 · Cited by 33,817
Stang, A. (2010) · European Journal of Epidemiology · doi:10.1007/s10654-010-9491-z · Cited by 17,396
Shea, B. J., Reeves, B. C., Wells, G., Thuku, M., Hamel, C., Moran, J., Moher, D., Tugwell, P., Welch, V., Kristjansson, E., et al. (2017) · BMJ · doi:10.1136/bmj.j4008 · Cited by 10,636
Vandenbroucke, J. P., von Elm, E., Altman, D. G., Gøtzsche, P. C., Mulrow, C. D., Pocock, S. J., Poole, C., Schlesselman, J. J., Egger, M., & Initiative, F. T. S. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040297 · Cited by 8,613
Moher, D., Schulz, K. F., & Altman, D. G. (2001) · The Lancet · doi:10.1016/s0140-6736(00)04337-3 · Cited by 5,373
Cuschieri, S. (2019) · Saudi Journal of Anaesthesia · doi:10.4103/sja.sja_543_18 · Cited by 4,331 · cited 1×
“STROBE provides general reporting recommendations for descriptive observational studies and studies that investigate associations between exposures and health outcomes.(17,18) STROBE addresses the three main types of observational studies: cohort, case–control, and cross-sectional studies.”
BackgroundImproving the Quality of Reporting of Randomized Controlled Trials: The CONSORT Statement
Begg, C. (1996) · JAMA · doi:10.1001/jama.1996.03540080059030 · Cited by 3,127
Moher, D., Schulz, K. F., & Altman, D. G. (2001) · BMC Medical Research Methodology · doi:10.1186/1471-2288-1-2 · Cited by 2,755
Newcastle-Ottawa Scale: comparing reviewers’ to authors’ assessments
Lo, C. K.-L., Mertz, D., & Loeb, M. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-45 · Cited by 2,477
Moher, D., Hopewell, S., Schulz, K. F., Montori, V., Gøtzsche, P. C., Devereaux, P. J., Elbourne, D., Egger, M., & Altman, D. G. (2012) · International Journal of Surgery · doi:10.1016/j.ijsu.2011.10.001 · Cited by 2,345
Cook, D. A., & Reed, D. A. (2015) · Academic Medicine · doi:10.1097/acm.0000000000000786 · Cited by 781
Hopewell, S., Clarke, M., Moher, D., Wager, E., Middleton, P., Altman, D. G., Schulz, K. F., & and the CONSORT Group (2008) · PLoS Medicine · doi:10.1371/journal.pmed.0050020 · Cited by 674
External Validation of a Measurement Tool to Assess Systematic Reviews (AMSTAR)
Shea, B. J., Bouter, L. M., Peterson, J., Boers, M., Andersson, N., Ortiz, Z., Ramsay, T., Bai, A., Shukla, V. K., & Grimshaw, J. M. (2007) · PLoS ONE · doi:10.1371/journal.pone.0001350 · Cited by 569
(2016) · doi:10.1002/14651858.cd201601 · Cited by 278
Dimairo, M., Pallmann, P., Wason, J., Todd, S., Jaki, T., Julious, S. A., Mander, A. P., Weir, C. J., Koenig, F., Walton, M. K., et al. (2020) · Trials · doi:10.1186/s13063-020-04334-x · Cited by 58 · cited 1×
“transparency and improving the report of adaptive-design randomized trials, to increase both the interpretability of their results and reproducibility of their methods, results, and inference.(25)”
Method
Research works printed but not linked: 23
- 10.1007/978-3-642-37131-8_13 · DOI
- 10.1097/PRS.0b013e318219c171 · DOI · Paper page
- 10.1016/0197-2456(95)00134-4 · DOI · Paper page
- 10.1001/jama.2014.5559 · DOI · Paper page
- 10.13105/wjma.v5.i4.80 · DOI · Paper page
- 10.1093/annonc/mdv494 · DOI
- 10.1002/jso.24070 · DOI
- 10.1016/j.ejso.2016.02.259 · DOI
- 10.1007/s11605-016-3199-7 · DOI
- 10.1136/jclinpath-2016-203830 · DOI
- 10.1097/MEG.0000000000000520 · DOI
- 10.1038/s41430-020-0581-z · doi:10.1038/s41430-020-0581-z
- 10.1097/AOG.0000000000002689 · DOI
- 10.1016/j.jclinepi.2019.07.019 · DOI
- 10.1016/j.jclinepi.2018.11.006 · DOI
- 10.1136/bmj.m115 · DOI
- 10.1016/j.jclinepi.2016.12.014 · DOI
- Higgins JPT, 2016, A revised tool for assessing risk of bias in randomized trials, Cochrane Database Syst Rev, 10, 29
- 10.1016/0197-2456(86)90046-2 · DOI · Paper page
- 10.1136/bmj.b2535 · DOI · Paper page
- 10.1186/1471-2288-7-10 · DOI · Paper page
- 10.1016/j.jclinepi.2008.10.009 · DOI · Paper page
- 10.1001/jamapsychiatry.2017.0624 · DOI · Paper page
Cited by
Automated Paper Screening for Clinical Reviews Using Large Language Models
Guo, E., Gupta, M., Deng, J., Park, Y.-J., Paget, M., & Naugler, C. (2023) · arXiv · doi:10.48550/arxiv.2305.00844 · Cited by 230
Automated paper screening for clinical reviews using large language models: Data analysis study
Guo, E., Gupta, M., Deng, J., Park, Y.-J., Paget, M., & Naugler, C. (2024) · Journal of Medical Internet Research · doi:10.2196/48996 · Cited by 205
Concussion Disrupts Sleep in Adolescents: A Systematic Review and Meta-Analysis
Noordeen, S., Wang, P., Strazda, A. E., Arany, E. S., Ergisi, M., Yeo, L. J., Popovic, R., Mahendran, A., Khawaja, M., Grover, K., et al. (2025) · Clocks & Sleep · doi:10.3390/clockssleep7030046 · Cited by 2
Metadata
- Authors
- Luchini, C., Veronese, N., Nottegar, A., Shin, J. I., Gentile, G., Granziol, U., Soysal, P., Alexinschi, O., Smith, L., & Solmi, M.
- Affiliations
- Department of Diagnostics and Public Health University and Hospital Trust of Verona Verona Italy
- National Research Council, Neuroscience Institute Padova Italy
- Department of Diagnostics, Section of Pathology San Bortolo Hospital Vicenza Italy
- Department of Pediatrics Yonsei University College of Medicine Seoul Republic of Korea
- Department of Neurosciences University of Padova Padova Italy
- Department of General Psychology University of Padova Padova Italy
- Department of Geriatric Medicine, Faculty of Medicine Bezmialem Vakif University Istanbul Turkey
- Institute of Psychiatry “Socola” Iasi Romania
- The Cambridge Centre for Sport and Exercise Sciences Anglia Ruskin University Cambridge UK
- ORCID
- Smith, Lee
- Venue
- Pharmaceutical Statistics
- Publisher
- Wiley
- Published
- 2020-09-15
- Volume
- 20
- Issue
- 1
- Pages
- 185-195
- ISSN
- 1539-1604
- 1539-1612
- Cited by
- 223
- References
- 16
- Language
- en
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
