The DOI matches its Crossref record; 12 of the 13 references it cites were found in a registry or an index; 4 of the 12 sources its claims cite were checked for support.
SANRA—a scale for the quality assessment of narrative review articles
Open accessAbstract
BACKGROUND: Narrative reviews are the commonest type of articles in the medical literature. However, unlike systematic reviews and randomized controlled trials (RCT) articles, for which formal instruments exist to evaluate quality, there is currently no instrument available to assess the quality of narrative reviews. In response to this gap, we developed SANRA, the Scale for the Assessment of Narrative Review Articles. METHODS: A team of three experienced journal editors modified or deleted items in an earlier SANRA version based on face validity, item-total correlations, and reliability scores from previous tests. We deleted an item which addressed a manuscript's writing and accessibility due to poor inter-rater reliability. The six items which form the revised scale are rated from 0 (low standard) to 2 (high standard) and cover the following topics: explanation of (1) the importance and (2) the aims of the review, (3) literature search and (4) referencing and presentation of (5) evidence level and (6) relevant endpoint data. For all items, we developed anchor definitions and examples to guide users in filling out the form. The revised scale was tested by the same editors (blinded to each other's ratings) in a group of 30 consecutive non-systematic review manuscripts submitted to a general medical journal. RESULTS: Raters confirmed that completing the scale is feasible in everyday editorial work. The mean sum score across all 30 manuscripts was 6.0 out of 12 possible points (SD 2.6, range 1-12). Corrected item-total correlations ranged from 0.33 (item 3) to 0.58 (item 6), and Cronbach's alpha was 0.68 (internal consistency). The intra-class correlation coefficient (average measure) was 0.77 [95% CI 0.57, 0.88] (inter-rater reliability). Raters often disagreed on items 1 and 4. CONCLUSIONS: SANRA's feasibility, inter-rater reliability, homogeneity of items, and internal consistency are sufficient for a scale of six items. Further field testing, particularly of validity, is desirable. We recommend rater training based on the "explanations and instructions" document provided with SANRA. In editorial decision-making, SANRA may complement journal-specific evaluation of manuscripts-pertaining to, e.g., audience, originality or difficulty-and may contribute to improving the standard of non-systematic reviews.
Overview
CoChat Summary
TL;DR
SANRA is a six-item, 0–2 scale (maximum 12) for assessing the quality of narrative review articles, developed and field-tested by three journal editors. In 30 consecutive non-systematic review manuscripts, it showed feasible use, sufficient internal consistency and inter-rater reliability, and a modest association with editorial acceptance.
Key findings
- Across 30 manuscripts (90 ratings), the mean SANRA sum score was 6.0 out of 12 (SD 2.6, range 1–12, median 6).
- Internal consistency was Cronbach’s alpha 0.68; corrected item-total correlations ranged from 0.33 (item 3) to 0.58 (item 6).
- Inter-rater reliability was an intra-class correlation coefficient (average measure) of 0.77 [95% CI 0.57, 0.88]; raters often disagreed on items 1 and 4.
- Average SANRA scores were modestly associated with editorial acceptance (mean 6.6, SD 1.9, n = 17) versus rejection (mean 5.1, SD 2.1, n = 13), point biserial correlation 0.37 (t = 2.09, df 28; two-sided p = 0.046).
Methods
- SANRA was developed between 2010 and 2017 by three experienced editors at Deutsches Ärzteblatt; the revised six-item version deleted an item on writing and accessibility due to poor inter-rater reliability.
- The revised scale was tested on 30 consecutive non-systematic review manuscripts submitted to Deutsches Ärzteblatt in 2015; three editor-raters scored them independently and blinded to each other’s ratings.
- Analyses included descriptive statistics, Cronbach’s alpha, corrected item-total correlations, average-measure intra-class correlation using a two-way random effects model (consistency definition), and a post hoc point biserial correlation between SANRA sum scores and acceptance/rejection.
Limitations
- The paper states the main limitation is sample size: 30 manuscripts was a compromise between representativeness, power, and feasibility, and took about 6 months to sample.
- The authors state that the scale’s authors were also the test-raters, so inter-rater reliability may be lower in groups less familiar with SANRA; the study also did not assess test-retest reliability, and results are study-specific.
- The paper acknowledges restricted content validity because SANRA is limited to six items and is a critical appraisal tool rather than a reporting guideline; criterion validity evidence is post hoc and weak, and no established cut-offs exist for comparing manuscripts across settings.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
3 Valid · 0 Not valid · 5 Not checked · 27 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Bastian et al. found that they constitute the largest share of all text types in medicine and they concluded that they “remain the staple of medical literature” [1].”
- Also stated in: § Background · plain
“Narrative reviews are the commonest type of articles in the medical literature.”
§ SANRA—a scale for the quality assessment of narrative review articles > Background
What the sources say
- Partly supportedcites [1] Bastian H, Glasziou P, Chalmers I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med. 2010;7(9):e1000326. https://doi.org/10.1371/journal.pmed.1000326 . sourceExperimental — accuracy not yet measured
In “How Close Are We to Archie Cochrane’s Goal?” and the Summary Points (pages unavailable), the paper calls non-systematic narrative reviews the staple of medical literature synthesis. It does not say they constitute the largest share of all medical text types; that extends a claim about literature synthesis to medical literature as a whole. (LLM, conf=0.88)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Background · plain
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“non-systematic reviews or narrative reviews have been widely criticized as unreliable [1, 4].”
§ SANRA—a scale for the quality assessment of narrative review articles > Background
What the sources say
- Supportedcites [1] Bastian H, Glasziou P, Chalmers I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med. 2010;7(9):e1000326. https://doi.org/10.1371/journal.pmed.1000326 . source
In “How Close Are We to Archie Cochrane’s Goal?”, the paper describes reviews not conducted systematically as having great potential for error and calls them less reliable forms of information. Box 1 also says that articles and textbooks had highlighted the risk of bias and chance in reviews. Together, these passages support the broad characterization that narrative reviews have been criticized as unreliable; page numbers are not provided in our copy. (LLM, conf=0.89)
- Not checked — no full text of the sourcecites [4] Mulrow CD. The medical review article: state of the science. Ann Intern Med. 1987;106:485–8. source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Cronbach’s alpha was 0.68 (internal consistency).”
- Also stated in: § Results · plain
“The scale’s internal consistency, measured as Cronbach’s alpha, was 0.68.”
§ SANRA—a scale for the quality assessment of narrative review articles > Abstract > Results
plainn = 90- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Results · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Raters often disagreed on items 1 and 4.”
- Also stated in: § Results · plain
“Disagreements most often occurred with regard to items 1 and 4.”
§ SANRA—a scale for the quality assessment of narrative review articles > Abstract > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Results · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The intra-class correlation coefficient (average measure) was 0.77 [95% CI 0.57, 0.88] (inter-rater reliability).”
- Also stated in: § Results · plain
“The intra-class correlation coefficient (average measure) amounted to 0.77 [95% CI 0.57, 0.88; F 4.3; df 29, 58].”
§ SANRA—a scale for the quality assessment of narrative review articles > Abstract > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Results · plain
14 more claims
Create a free account to see all 35 claims and to verify the 3 references still open.
- Probably not a substantial claim. 11
- It may state two separate claims as one. 11
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The review was conducted and reported in accordance with the principles of the Scale for the Assessment of Narrative Review Articles (SANRA), which provides a validated framework for ensuring methodological transparency, scientific rigor, and balanced interpretation in narrative reviews [ 19 ].”
From Paracetamol: A History of Assumptions, Errors, and Evidence Reversals in Pain Management (2026) · cites this paper as [19]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“As a narrative review, the present work does not include an aggregated quantitative assessment and is more exposed to selection subjectivity than a registered systematic review [124,125].”
From Berberine-Drug Interactions: Mechanisms, Clinical Relevance and Risk Stratification-A Narrative Review (2026) · cites this paper as [124]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“This work is designed as a structured narrative review, with transparent reporting of the documentation strategy, guided by the quality criteria of the Scale for the Assessment of Narrative Review Articles (SANRA) [124].”
From Berberine-Drug Interactions: Mechanisms, Clinical Relevance and Risk Stratification-A Narrative Review (2026) · cites this paper as [124]
The cited source says what the claim attributes to it.
In the citing paper
“we used the following:-Cochrane Risk of Bias 2 (RoB 2) [29] and PEDRO scale tools for RCTs.-AMSTAR 2 [30] checklist for systematic review.-SANRA (Scale for the Assessment of Narrative Review Articles) [31] for narrative reviews.-JBI Checklist for Text and Opinion [32] for expert opinion papers.”
From Pelvic Floor Muscle Training Following Surgery for Pelvic Organ Prolapse: Recommendation from Scientific Literature (2026) · cites this paper as [31]
The cited source says what the claim attributes to it.
In the citing paper
“Data analysis employed a narrative and thematic synthesis methodology,adhering to the SANRA guidelines for narrative reviews (32).”
From Ethical issues in multi-agent AI systems for healthcare: a narrative review (2026) · cites this paper as [32]
10 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Shea, B. J., Reeves, B. C., Wells, G., Thuku, M., Hamel, C., Moran, J., Moher, D., Tugwell, P., Welch, V., Kristjansson, E., et al. (2017) · BMJ · doi:10.1136/bmj.j4008 · Cited by 10,636
Streiner, D. L., & Norman, G. R. (2008) · doi:10.1093/acprof:oso/9780199231881.001.0001 · Cited by 2,631
Critical appraisal skills programme
Singh, J. (2013) · Journal of Pharmacology and Pharmacotherapeutics · doi:10.4103/0976-500x.107697 · Cited by 2,556
Guidelines for Reporting Reliability and Agreement Studies (GRRAS) were proposed
Kottner, J., Audigé, L., Brorson, S., Donner, A., Gajewski, B. J., Hróbjartsson, A., Roberts, C., Shoukri, M., & Streiner, D. L. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.03.002 · Cited by 2,174
The Mass Production of Redundant, Misleading, and Conflicted Systematic Reviews and Meta‐analyses
IOANNIDIS, J. P. A. (2016) · The Milbank Quarterly · doi:10.1111/1468-0009.12210 · Cited by 1,483
Seventy-five trials and eleven systematic reviews a day: how will we ever keep up?
Bastian, H., Glasziou, P., & Chalmers, I. (2010) · PLoS Medicine · doi:10.1371/journal.pmed.1000326 · Cited by 1,188
The Medical Review Article: State of the Science
MULROW, C. D. (1987) · Annals of Internal Medicine · doi:10.7326/0003-4819-106-3-485 · Cited by 856
Writing a narrative biomedical review: considerations for authors, peer reviewers, and editors
Gasparyan, A. Y., Ayvazyan, L., Blackmore, H., & Kitas, G. D. (2011) · Rheumatology International · doi:10.1007/s00296-011-1999-3 · Cited by 694
Ten Simple Rules for Writing a Literature Review
Pautasso, M. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003149 · Cited by 272
Improving the peer review of narrative literature reviews
Byrne, J. A. (2016) · Research Integrity and Peer Review · doi:10.1186/s41073-016-0019-2 · Cited by 159
Research works printed but not linked: 3
- Higgins JPT, Green S. (eds.). Cochrane Handbook for Systematic Reviews and Interventions. Version 5.1.0. The Cochrane Collaboration 2011, section 1.2.2., www.handbook.cochrane.org , retrieved on Oct 31, 2018.
- Baethge C, Mertens S, Goldbeck-Wood S. Development of a quality score for the assessment of non-systematic review articles (SANRA). In: Poster, Seventh International Congress on Peer Review and Biomedical Publication. USA: Chicago, Illinois; 2013.
- Baethge C, Goldbeck-Wood S, Mertens S. A scale for the assessment of non-systematic review articles (SANRA). In: Poster, Eighth International Congress on Peer and Biomedical Publication. USA: Chicago, Illinois; 2017.
Cited by
Research co-design in health: a rapid overview of reviews
Slattery, P., Saeri, A. K., & Bragge, P. (2020) · Health Research Policy and Systems · doi:10.1186/s12961-020-0528-9 · Cited by 1,313
Use of Delphi in health sciences research: A narrative review
Shang, Z. (2023) · Medicine · doi:10.1097/md.0000000000032829 · Cited by 657
AlGerafi, M. A. M., Zhou, Y., Oubibi, M., & Wijaya, T. T. (2023) · Electronics · doi:10.3390/electronics12183953 · Cited by 373
A Narrative Review on Sarcopenia in Type 2 Diabetes Mellitus: Prevalence and Associated Factors
Izzo, A., Massimino, E., Riccardi, G., & Della Pepa, G. (2021) · Nutrients · doi:10.3390/nu13010183 · Cited by 418
Alfei, S., Schito, G. C., Schito, A. M., & Zuccari, G. (2024) · International Journal of Molecular Sciences · doi:10.3390/ijms25137182 · Cited by 302
Metadata
- Authors
- Baethge, C., Goldbeck-Wood, S., & Mertens, S.
- Affiliations
- University of Cologne (DE)
- BMJ Group (United Kingdom) (GB)
- Corresponding author
- Baethge, Christopher
- ORCID
- Baethge, Christopher
- Venue
- Research Integrity and Peer Review
- Published
- 2019-03-26
- Volume
- 4
- Issue
- 1
- Pages
- 5-5
- ISSN
- 2058-8615 (ISSN-L)
- PMC
- PMC6434870
- OpenAlex
- W2922577842
- Semantic Scholar
- ea7f35b3069b53791c3b746c5c4b3c72a83a526b
- Cited by
- 3,469
- References
- 13
- Citations by year
- 2026: 1823 · 2025: 551 · 2024: 379 · 2023: 309 · 2022: 218 · 2021: 152 · 2020: 32 · 2019: 4
- FWCI
- 20.57
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0/ (tdm)
- Locations
- Research Integrity and Peer Review (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, other-oa)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Kölner Universitäts PublikationsServer (Universität zu Köln) (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Library and Information Sciences › Academic Writing and Publishing
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Keywords
- quality assessment scale, manuscript review, inter-rater reliability, level of evidence, scale development
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-11-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: SANRA—a scale for the quality assessment of narrative review articles. https://app.cochat.ai/papers/10.1186/s41073-019-0064-8
- The paper
- Baethge, C., Goldbeck-Wood, S., & Mertens, S. (2019). SANRA—a scale for the quality assessment of narrative review articles. Research Integrity and Peer Review. https://doi.org/10.1186/s41073-019-0064-8
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.

