CoChat

ROSES RepOrting standards for Systematic Evidence Syntheses: pro forma, flow-diagram and descriptive summary of the plan and conduct of environmental systematic reviews and systematic maps

Open access

The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index; 6 of the 17 sources its claims cite were checked for support.

Abstract

Reliable synthesis of the various rapidly expanding bodies of evidence is vital for the process of evidence-informed decision-making in environmental policy, practice and research. With the rise of evidence-base medicine and increasing numbers of published systematic reviews, criteria for assessing the quality of reporting have been developed. First QUOROM (Lancet 354:1896–1900, 1999 ) and then PRISMA (Ann Intern Med 151:264, 2009 ) were developed as reporting guidelines and standards to ensure medical meta-analyses and systematic reviews are reported to a high level of detail. PRISMA is now widely used by a range of journals as a pre-submission checklist. However, due to its development for systematic reviews in healthcare, PRISMA has limited applicability for reviews in conservation and environmental management. We highlight 12 key problems with the application of PRISMA to this field, including an overemphasis on meta-analysis and no consideration for other synthesis methods. We introduce ROSES (RepOrting standards for Systematic Evidence Syntheses), a pro forma and flow diagram designed specifically for systematic reviews and systematic maps in the field of conservation and environmental management. We describe how ROSES solves the problems with PRISMA. We outline the key benefits of our approach to designing ROSES, in particular the level of detail and inclusion of rich guidance statements. We also introduce the extraction of meta-data that describe key aspects of the conduct of the review. Collated together, this summary record can help to facilitate rapid review and appraisal of the conduct of a systematic review or map, potentially speeding up the peer-review process. We present the results of initial road testing of ROSES with systematic review experts, and propose a plan for future development of ROSES.

Overview

CoChat Summary

TL;DR

ROSES is a reporting standard, pro forma, flow diagram and meta-data summary tailored to systematic reviews and maps in conservation and environmental management, developed to address 12 identified problems with PRISMA. It aims to improve transparency, detail and peer-review efficiency, with initial road testing by six experts.

Key findings

  • The paper identifies 12 key problems with applying PRISMA to environmental systematic reviews and maps, including overemphasis on meta-analysis, no consideration for other synthesis methods, and limited applicability to systematic maps.
  • ROSES is introduced as a pro forma and flow diagram specifically for systematic reviews and systematic maps in conservation and environmental management, with a checklist plus extracted meta-data to summarise review conduct.
  • ROSES was road tested with six external systematic review experts from environmental evidence synthesis; feedback was collected via semi-structured questionnaire or 30-min interview, and the pro forma were revised accordingly.
  • The paper reports that ROSES is considerably longer than current PRISMA forms and may take authors several hours to complete, but argues this can improve manuscripts and reduce manuscript consideration time.

Methods

  • Two authors critically reviewed current PRISMA versions for review protocols and final review reports, identified shortfalls for conservation and environmental science, and proposed solutions.
  • The suggested form was reviewed by all four authors and revised, then circulated to six external systematic review experts; feedback was collected via semi-structured questionnaire or 30-min interview, subject to expert availability.
  • The pro forma were revised according to these comments; the paper presents ROSES as a reporting standard with checklist, flow diagram and collated meta-data, and proposes future digitisation.

Limitations

  • Stated: ROSES is considerably longer than current PRISMA forms and will take more time for authors to complete, described as a matter of several hours.
  • Stated: ROSES focuses on earlier and middle stages of the review process—searching, screening, data extraction and critical appraisal—with limited detail regarding synthesis; tailored add-on reporting standards for synthesis methods are planned.
  • Evident: initial road testing involved only six external experts and feedback via questionnaire or 30-min interview; no larger-scale validation or formal reliability testing is reported.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1186/s13750-018-0121-7

Cites

38 of 38 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

4 Valid · 0 Not valid · 8 Not checked · 52 Own results and interpretations

11 more claims

Create a free account to see all 64 claims and to verify the 11 references still open.

See all claims — free account
  • It may state two separate claims as one. 20
  • Probably not a substantial claim. 44

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Show all 38 references

Research works printed but not linked: 3

  1. CEE (Collaboration for Environmental Evidence). Guidelines for Systematic Review and Evidence Synthesis in Environmental Management. Version 4.2. 2013. http://environmentalevidence.org/wp-content/uploads/2014/06/Review-guidelines-version-4.2-final.pdf .
  2. The Steering Group of the Campbell Collaboration. Campbell Collaboration Systematic Reviews: policies and guidelines. Campbell Syst Rev. 2015;46:1.
  3. Environmental Evidence. Preparing your manuscript. 2017. https://environmentalevidencejournal.biomedcentral.com/submission-guidelines/preparing-your-manuscript . Accessed 25 July 2017.

Cited by

See all 1,001

Metadata

Authors
Haddaway, N. R., Macura, B., Whaley, P., & Pullin, A. S.
Affiliations
  • Stockholm Environment Institute (SE)
  • Lancaster University (GB)
  • Bangor University (GB)
Corresponding author
Macura, Biljana
Venue
Environmental Evidence
Published
2018-03-19
Volume
7
Issue
1
ISSN
2047-2382 (ISSN-L)
OpenAlex
W2795357522
Cited by
946
References
38
Citations by year
2026: 142 · 2025: 147 · 2024: 201 · 2023: 135 · 2022: 124 · 2021: 96 · 2020: 66 · 2019: 25 · 2018: 9
FWCI
9.84
Type
journal article
Language
en
Open access
Licence
  • cc-by
  • creativecommons.org/licenses/by/4.0/ (tdm)
Locations
  • Environmental Evidence (publishedVersion, cc-by)
  • Bangor University Research Portal (Bangor University) (publishedVersion, other-oa)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Lancaster EPrints (Lancaster University) (submittedVersion)
Funders
  • Stiftelsen för Miljöstrategisk Forskning
  • Svenska Forskningsrådet Formas
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Social Sciences › Sociology and Political Science › Delphi Technique in Research
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
Keywords
roses, reporting guidelines, systematic reviews, systematic map, PRISMA, environmental management, conservation, meta-analysis, flowcharts, peer review, evidence-informed decision-making

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (qwen3.8-omni-flash).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: ROSES RepOrting standards for Systematic Evidence Syntheses: pro forma, flow-diagram and descriptive summary of the plan and conduct of environmental systematic reviews and systematic maps. https://app.cochat.ai/papers/10.1186/s13750-018-0121-7
The paper
Haddaway, N. R., Macura, B., Whaley, P., & Pullin, A. S. (2018). ROSES RepOrting standards for Systematic Evidence Syntheses: pro forma, flow-diagram and descriptive summary of the plan and conduct of environmental systematic reviews and systematic maps. Environmental Evidence. https://doi.org/10.1186/s13750-018-0121-7

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.