Methodological approaches in conducting overviews: current state in HTA agencies
The DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
Abstract
Objectives: Overviews search for reviews rather than for primary studies. They might have the potential to support decision making within a shorter time frame by reducing production time. We aimed to summarize available instructions for authors intending to conduct overviews as well as the currently applied methodology of overviews in international Health Technology Assessment (HTA) agencies. Methods: We identified 127 HTA agencies and scanned their websites for methodological handbooks as well as published overviews as HTA reports. Additionally, we contacted HTA agencies by e‐mail to retrieve possible unidentified handbooks or other related sources. Results: In total, eight HTA agencies providing methodological support were found. Thirteen HTA agencies were found to have produced overviews since 2007, but only six of them published more than four overviews. Overviews were mostly employed in HTA products related to rapid assessment. Additional searches for primary studies published after the last review are often mentioned in order to update results. Conclusions: Although the interest in overviews is rising, little methodological guidance for the conduct of overviews is provided by HTA agencies. Overviews are of special interest in the context of rapid assessments to support policy‐making within a short time frame. Therefore, empirical work on overviews needs to be extended. National strategies and experience should be disclosed and discussed. Copyright © 2013 John Wiley & Sons, Ltd.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Methodology in conducting a systematic review of systematic reviews of healthcare interventions
Smith, V., Devane, D., Begley, C. M., & Clarke, M. (2011) · BMC Medical Research Methodology · doi:10.1186/1471-2288-11-15 · Cited by 1,227
Seventy-five trials and eleven systematic reviews a day: how will we ever keep up?
Bastian, H., Glasziou, P., & Chalmers, I. (2010) · PLoS Medicine · doi:10.1371/journal.pmed.1000326 · Cited by 1,188
How Quickly Do Systematic Reviews Go Out of Date? A Survival Analysis
Shojania, K. G., Sampson, M., Ansari, M. T., Ji, J., Doucette, S., & Moher, D. (2007) · Annals of Internal Medicine · doi:10.7326/0003-4819-147-4-200708210-00179 · Cited by 838 · cited 1×
“Evidence does not tell us when and how to update SRs (Moher et al., 2008), but there is evidence that approximately 15%, 25% and 50% of SRs are out-dated after 1, 2 and 5.5 years, respectively (Shojania et al., 2007).”
BackgroundInfluentialExpediting systematic reviews: methods and implications of rapid reviews
Ganann, R., Ciliska, D., & Thomas, H. (2010) · Implementation Science · doi:10.1186/1748-5908-5-56 · Cited by 752
A descriptive analysis of overviews of reviews published between 2000 and 2011.
Hartling, L., Chisholm, A., Thomson, D., & Dryden, D. M. (2012) · PLoS ONE · doi:10.1371/journal.pone.0049667 · Cited by 241 · cited 1×
“…related to the problems and advantages of overviews (Becker and Oxman, 2008, Cooper and Koenka, 2012, Smith et al., 2011, Thomson et al., 2010, Hartling et al., 2012, Pieper et al., 2012), we developed an analytical framework to guide the presentation of results and subsequent discussion.”
MethodWatt, A., Cameron, A., Sturm, L., Lathlean, T., Babidge, W., Blamey, S., Facey, K., Hailey, D., Norderhaug, I., & Maddern, G. (2008) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462308080185 · Cited by 226 · cited 1×
“In a review of methods and practice in HTAs, rapid reviews were found to be potentially of limited rigor providing less robustness of results than those of traditional SRs (Watt et al., 2008).”
Becker, L. A., & Oxman, A. D. (2008) · Cochrane Handbook for Systematic Reviews of Interventions · doi:10.1002/9780470712184.ch22 · Cited by 218
Overviews of reviews often have limited rigor: a systematic review
Pieper, D., Buechter, R., Jerinic, P., & Eikermann, M. (2012) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2012.06.015 · Cited by 213 · cited 5×
“We searched bibliographic databases some months prior to the hand-search in order to identify overviews in the context of another project (Pieper et al., 2012).”
BackgroundMethodInfluentialSelf-citationCooper, H., & Koenka, A. C. (2012) · American Psychologist · doi:10.1037/a0027119 · Cited by 196
Using Existing Systematic Reviews in Complex Systematic Reviews
Whitlock, E. P., Lin, J. S., Chou, R., Shekelle, P., & Robinson, K. A. (2008) · Annals of Internal Medicine · doi:10.7326/0003-4819-148-10-200805200-00010 · Cited by 144
Cited by
Hunt, H., Pollock, A., Campbell, P., Estcourt, L., & Brunton, G. (2018) · Systematic Reviews · doi:10.1186/s13643-018-0695-8 · Cited by 314
Bougioukas, K. I., Liakos, A., Tsapas, A., Ntzani, E., & Haidich, A.-B. (2018) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.10.002 · Cited by 251
Silver diamine fluoride for managing carious lesions: an umbrella review
Seifo, N., Cassie, H., Radford, J. R., & Innes, N. P. T. (2019) · BMC Oral Health · doi:10.1186/s12903-019-0830-5 · Cited by 217
Lunny, C., Brennan, S. E., McDonald, S., & McKenzie, J. E. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0617-1 · Cited by 184
Selecting and implementing overview methods: implications from five exemplar overviews
Pollock, A., Campbell, P., Brunton, G., Hunt, H., & Estcourt, L. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0534-3 · Cited by 134
Lunny, C., Brennan, S. E., McDonald, S., & McKenzie, J. E. (2018) · Systematic Reviews · doi:10.1186/s13643-018-0784-8 · Cited by 139
“Aromataris 2015 [39, 59]; Becker 2008 [4]; Caird 2015 [1]; CMIMG 2012 [47]; Cooper 2012 [6]; Hartling 2012 [53]; Hartling 2014 [55]; Jadad 1997 [29]; Ioannidis 2009 [57]; Kramer 2009 [61]; Ryan 2009 [25]; Singh 2012 [76]; Thomson 2010 [26] ▪ Report RoB assessments of primary studies from the included SR(s), noting mis…”
InfluentialBallard, M., & Montgomery, P. (2017) · Research Synthesis Methods · doi:10.1002/jrsm.1229 · Cited by 136
Overviews of systematic reviews: great promise, greater challenge
McKenzie, J. E., & Brennan, S. E. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0582-8 · Cited by 134
Bougioukas, K. I., Vounzoulaki, E., Mantsiou, C. D., Papanastasiou, G. D., Savvides, E. D., Ntzani, E. E., & Haidich, A.-B. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.03.019 · Cited by 46
Metadata
- Authors
- Pieper, D., Antoine, S., Morfeld, J., Mathes, T., & Eikermann, M.
- Venue
- Research Synthesis Methods
- Publisher
- Wiley
- Published
- 2013-12-10
- Volume
- 5
- Issue
- 3
- Pages
- 187-199
- ISSN
- 1759-2879 (ISSN-L)
- 1759-2887
- OpenAlex
- W1934286735
- Cited by
- 39
- References
- 10
- FWCI
- 2.15
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.