Preferred Reporting Items for Overviews of Reviews (PRIOR): a protocol for development of a reporting guideline for overviews of reviews of healthcare interventions
Open accessThe DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
Abstract
Background: Overviews of reviews (i.e., overviews) compile information from multiple systematic reviews to provide a single synthesis of relevant evidence for healthcare decision-making. Despite their increasing popularity, there are currently no systematically developed reporting guidelines for overviews. This is problematic because the reporting of published overviews varies considerably and is often substandard. Our objective is to use explicit, systematic, and transparent methods to develop an evidence-based and agreement-based reporting guideline for overviews of reviews of healthcare interventions (PRIOR, Preferred Reporting Items for Overviews of Reviews). Methods: We will develop the PRIOR reporting guideline in four stages, using established methods for developing reporting guidelines in health research. First, we will establish an international and multidisciplinary expert advisory board that will oversee the conduct of the project and provide methodological support. Second, we will use the results of comprehensive literature reviews to develop a list of prospective checklist items for the reporting guideline. Third, we will use a modified Delphi exercise to achieve a high level of expert agreement on the list of items to be included in the PRIOR reporting guideline. We will identify and recruit a group of up to 100 international experts who will provide input into the guideline in three Delphi rounds: the first two rounds will occur via online survey, and the third round will occur during a smaller (8 to 10 participants) in-person meeting that will use a nominal group technique. Fourth, we will produce and publish the PRIOR reporting guideline. Discussion: A systematically developed reporting guideline for overviews could help to improve the accuracy, completeness, and transparency of overviews. This, in turn, could help maximize the value and impact of overviews by allowing more efficient interpretation and use of their research findings.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“Preferred Reporting Items for Overview of Reviews (Pollock et al., 2019) was used as the guideline for umbrella review reporting.”
From Risk and Protective Factors of Self-harm and Suicidality in Adolescents: An Umbrella Review with Meta-Analysis (2024) · cites this paper as Pollock et al., 2019
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredIn the citing paper
“We registered our intent to develop PRIOR with the EQUATOR Network on 4 January 2017,27 and the protocol to develop PRIOR was published on 23 December 2019.26”
From Reporting guideline for overviews of reviews of healthcare interventions: development of the PRIOR statement. (2022) · cites this paper as [26]
In the citing paper
“The number of Delphi rounds was prespecified, with the potential for additional rounds to be added if agreement could not be reached (this was not required).”
From Reporting guideline for overviews of reviews of healthcare interventions: development of the PRIOR statement. (2022) · cites this paper as [26]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164 · cited 15×
“Bougioukas et al. [13] published a checklist (PRIO, Preferred Reporting Items for Overviews of Systematic Reviews) with specific emphasis on harms reporting that was designed by combining recommendations from PRISMA [11], PRISMA-harms [14], PRISMA-protocols (PRISMA-P) [15], and other relevant documents.”
MethodInfluentialLiberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gotzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · BMJ · doi:10.1136/bmj.b2700 · Cited by 17,648
Cochrane Handbook for Systematic Reviews of Interventions
(2019) · doi:10.1002/9781119536604 · Cited by 14,828
Shamseer, L., Moher, D., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., Stewart, L. A., & the PRISMA-P Group (2015) · BMJ · doi:10.1136/bmj.g7647 · Cited by 13,120
Shea, B. J., Reeves, B. C., Wells, G., Thuku, M., Hamel, C., Moran, J., Moher, D., Tugwell, P., Welch, V., Kristjansson, E., et al. (2017) · BMJ · doi:10.1136/bmj.j4008 · Cited by 10,636
Health Measurement Scales: A Practical Guide to Their Development and Use
Streiner, D. L., Norman, G. R., & Cairney, J. (1989) · Cited by 9,595
Hutton, B., Salanti, G., Caldwell, D. M., Chaimani, A., Schmid, C. H., Cameron, C., Ioannidis, J. P. A., Straus, S., Thorlund, K., Jansen, J. P., et al. (2015) · Annals of Internal Medicine · doi:10.7326/m14-2385 · Cited by 8,269
Research guidelines for the Delphi survey technique
Hasson, F. (2000) · Journal of Advanced Nursing · doi:10.1046/j.1365-2648.2000.01567.x · Cited by 5,951 · cited 2×
“, the core project team but not the other study participants will know the identities and responses of the participants [31]), using a versatile online Delphi platform (e.”
MethodThe Delphi Technique: Making Sense of Consensus
Hsu, C.-C., & Sandford, B. A. (2007) · University of Massachusetts Amherst · doi:10.7275/pdz9-th90 · Cited by 4,500
Development of AMSTAR: A measurement tool to assess the methodological quality of systematic reviews
Shea, B. J., Grimshaw, J. M., Wells, G. A., Boers, M., Andersson, N., Hamel, C., Porter, A. C., Tugwell, P., Moher, D., & Bouter, L. M. (2007) · BMC Medical Research Methodology · doi:10.1186/1471-2288-7-10 · Cited by 4,374
Diamond, I. R., Grant, R. C., Feldman, B. M., Pencharz, P. B., Ling, S. C., Moore, A. M., & Wales, P. W. (2014) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2013.12.002 · Cited by 3,740 · cited 1×
“, score of 1–2) the reporting guideline [38], based on the total number of responses obtained per question.”
BackgroundQualitative Research: Consensus methods for medical and health services research
Jones, J., & Hunter, D. (1995) · BMJ · doi:10.1136/bmj.311.7001.376 · Cited by 3,471
Health Measurement Scales: A practical guide to their development and use
Streiner, D. L., Norman, G. R., & Cairney, J. (2014) · doi:10.1093/med/9780199685219.001.0001 · Cited by 2,564
Consensus development methods, and their use in clinical guideline development.
Murphy, Black, Lamping, McKee, Sanderson, Askham, & Marteau (1998) · Health Technology Assessment · doi:10.3310/hta2030 · Cited by 2,051 · cited 9×
“, ≥ 70% agreement, as previously described) on those checklist items still lacking agreement [33, 34].”
BackgroundInfluentialGreenhalgh, T., & Peacock, R. (2005) · BMJ · doi:10.1136/bmj.38636.593461.68 · Cited by 2,045
Guidance for Developers of Health Research Reporting Guidelines
Moher, D., Schulz, K. F., Simera, I., & Altman, D. G. (2010) · PLoS Medicine · doi:10.1371/journal.pmed.1000217 · Cited by 1,437
Fitzpatrick, Shortall, Sculpher, Murray, Morris, Lodge, Dawson, Carr, Britton, & Briggs (1998) · Health Technology Assessment · doi:10.3310/hta2200 · Cited by 969
Consensus development methods, and their use in clinical guideline development.
Murphy, M. P., Black, N. A., Lamping, D. L., et al. (1998) · PubMed · Cited by 965
Plint, A. C., Moher, D., Morrison, A., Schulz, K., Altman, D. G., Hill, C., & Gaboury, I. (2006) · Medical Journal of Australia · doi:10.5694/j.1326-5377.2006.tb00557.x · Cited by 872
Validation of an index of the quality of review articles
Oxman, A. D., & Guyatt, G. H. (1991) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(91)90160-b · Cited by 786
PRISMA for Abstracts: Reporting Systematic Reviews in Journal and Conference Abstracts
Beller, E. M., Glasziou, P. P., Altman, D. G., Hopewell, S., Bastian, H., Chalmers, I., Gøtzsche, P. C., Lasserson, T., Tovey, D., & for the PRISMA for Abstracts Group (2013) · PLoS Medicine · doi:10.1371/journal.pmed.1001419 · Cited by 706
PRISMA harms checklist: improving harms reporting in systematic reviews
Zorzela, L., Loke, Y. K., Ioannidis, J. P., Golder, S., Santaguida, P., Altman, D. G., Moher, D., Vohra, S., & PRISMA harms group (2016) · BMJ · doi:10.1136/bmj.i157 · Cited by 557
Checking reference lists to find additional studies for systematic reviews
Horsley, T., Dingwall, O., & Sampson, M. (2011) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000026.pub2 · Cited by 443
Igúzquiza, V. D. D. R. (2016) · Redalyc (Universidad Autónoma del Estado de México) · Cited by 365
Principles for Constructing Web Surveys
Dillman, D. A., Tortora, R. D., & Bowker, D. (1998) · Cited by 296
Bougioukas, K. I., Liakos, A., Tsapas, A., Ntzani, E., & Haidich, A.-B. (2018) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.10.002 · Cited by 251
A descriptive analysis of overviews of reviews published between 2000 and 2011.
Hartling, L., Chisholm, A., Thomson, D., & Dryden, D. M. (2012) · PLoS ONE · doi:10.1371/journal.pone.0049667 · Cited by 241
Self-citationThe quality of diagnostic accuracy studies since the STARD statement: Has it improved?
Smidt, N., Rutjes, A. W. S., van der Windt, D. A. W. M., Ostelo, R. W. J. G., Bossuyt, P. M., Reitsma, J. B., Bouter, L. M., & de Vet, H. C. W. (2006) · Neurology · doi:10.1212/01.wnl.0000238386.41398.30 · Cited by 235
Overviews of reviews often have limited rigor: a systematic review
Pieper, D., Buechter, R., Jerinic, P., & Eikermann, M. (2012) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2012.06.015 · Cited by 213
Self-citationPollock, M., Fernandes, R. M., Becker, L. A., Featherstone, R., & Hartling, L. (2016) · Systematic Reviews · doi:10.1186/s13643-016-0367-5 · Cited by 195
Self-citationLunny, C., Brennan, S. E., McDonald, S., & McKenzie, J. E. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0617-1 · Cited by 184
Lunny, C., Brennan, S. E., McDonald, S., & McKenzie, J. E. (2018) · Systematic Reviews · doi:10.1186/s13643-018-0784-8 · Cited by 139
Simera, I., Moher, D., Hoey, J., Schulz, K. F., & Altman, D. G. (2009) · Maturitas · doi:10.1016/j.maturitas.2009.03.011 · Cited by 97
Lunny, C., Brennan, S. E., Reid, J., McDonald, S., & McKenzie, J. E. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.09.025 · Cited by 71
Quality and transparency of overviews of systematic reviews
Li, L., Tian, J., Tian, H., Sun, R., Liu, Y., & Yang, K. (2012) · Journal of Evidence-Based Medicine · doi:10.1111/j.1756-5391.2012.01185.x · Cited by 62
Bougioukas, K. I., Bouras, E., Apostolidou-Kiouti, F., Kokkali, S., Arvanitidou, M., & Haidich, A.-B. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2018.10.005 · Cited by 56
Pieper, D., Pollock, M., Fernandes, R. M., Büchter, R. B., & Hartling, L. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0468-9 · Cited by 25
Self-citationDevelopment of the Metareview Assessment of Reporting Quality (MARQ) Checklist
Singh, J. P. (2012) · Revista de la Facultad de Medicina · Cited by 10
Research works printed but not linked: 4
- 9. Onishi A, Furakawa TA. Chapter 13: State-of-the-art reporting. In: Biondi- Zoccai G, editor. Umbrella reviews. Cham: Springer International Publishing; 2016.
- 10. Becker LA, Oxman AD. Chapter 22: Overviews of reviews. In: Higgins JPT, Green S, editors. Cochrane handbook for systematic reviews of interventions (version 5.1.0). London: The Cochrane Collaboration; 2011.
- 21. Posadzki P. Standards for reporting of overviews of reviews and umbrella reviews (STROVI) statement. Cape Town: Cochrane Colloquium; 2017.
- 37. Dillman DA. Mail and internet surveys: the tailored design method-2007 update with new internet, visual, and mixed-mode guide. New York: Wiley; 2011.
Cited by
Health effects of climate change: an overview of systematic reviews
Rocque, R. J., Beaudoin, C., Ndjaboue, R., Cameron, L., Poirier-Bergeron, L., Poulin-Rheault, R.-A., Fallon, C., Tricco, A. C., & Witteman, H. O. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-046333 · Cited by 932
“A senior analyst resolved any conflict or disagreement.”
BackgroundSelf-citationGates, M., Gates, A., Pieper, D., Fernandes, R. M., Tricco, A. C., Moher, D., Brennan, S. E., Li, T., Pollock, M., Lunny, C., et al. (2022) · BMJ · doi:10.1136/bmj-2022-070849 · Cited by 856
Self-citationBarriers and facilitators to utilizing digital health technologies by healthcare professionals
Borges do Nascimento, I. J., Abdulazeem, H., Vasanthan, L. T., Martinez, E. Z., Zucoloto, M. L., Østengaard, L., Azzopardi-Muscat, N., Zapata, T., & Novillo-Ortiz, D. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00899-4 · Cited by 673
Graphical Representation of Overlap for OVErviews: GROOVE tool
Bracchiglione, J., Meza, N., Bangdiwala, S. I., Niño de Guzmán, E., Urrútia, G., Bonfill, X., & Madrid, E. (2022) · Research Synthesis Methods · doi:10.1002/jrsm.1557 · Cited by 299
“11,16,17 Matrices of evidence and the calculation of the corrected covered area (CCA) are probably among the most exhaustive methods for measuring overlap.”
MethodResearch Trends, Challenges, and Emerging Topics in Digital Forensics: A Review of Reviews
Casino, F., Dasaklis, T. K., Spathoulas, G. P., Anagnostopoulos, M., Ghosal, A., Borocz, I., Solanas, A., Conti, M., & Patsakis, C. (2022) · IEEE Access · doi:10.1109/access.2022.3154059 · Cited by 165
“However, various multidisciplinary teams around the globe work together to develop relevant standardized reporting guidelines that will soon be available [20].”
Background
Metadata
- Authors
- Pollock, M., Fernandes, R. M., Pieper, D., Tricco, A. C., Gates, M., Gates, A., & Hartling, L.
- Affiliations
- Institute of Health Economics (CA)
- University of Lisbon (PT)
- Witten/Herdecke University (DE)
- St. Michael's Hospital (CA)
- University of Alberta (CA)
- Corresponding author
- Hartling, Lisa
- Venue
- Systematic Reviews
- Published
- 2019-12
- Volume
- 8
- Issue
- 1
- Pages
- 335-335
- ISSN
- 2046-4053 (ISSN-L)
- PMC
- PMC6929355
- OpenAlex
- W2995096233
- Cited by
- 228
- References
- 37
- Citations by year
- 2026: 34 · 2025: 49 · 2024: 40 · 2023: 34 · 2022: 31 · 2021: 34 · 2020: 4
- FWCI
- 11.53
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0/ (tdm)
- creativecommons.org/licenses/by/4.0/ (vor, 22 d)
- Locations
- Systematic Reviews (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- TSpace (University of Toronto) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT) (submittedVersion, other-oa)
- Funders
- Canadian Institutes of Health Research (Tier 2 Canada Research Chair in Knowledge Synthesis, Tier 1 Canada Research Chair in Knowledge Synthesis and Translation)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- overviews of reviews, reporting guidelines, prior distributions, health interventions, evidence synthesis, Delphi consensus, checklist development
- MeSH terms
- Humans, Review Literature as Topic, Guidelines as Topic, Biomedical Research, Checklist
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
