The DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Evidence and gap maps: a comparison of different approaches
Open accessAbstract
Evidence maps are a relatively new approach to systematically identify and report the range of research activity in broad topic areas or policy domains. Although they go under various names, they have the common feature of adopting a systematic approach to mapping begins with a broad question, theme or issue which defines the scope of the evidence map. For reasons discussed below, we adopt the term Evidence and Gap Map to refer to the specific approach being proposed for Campbell, and evidence map to refer collectively to the range of approaches which have been adopted. As we describe in Part 2, evidence mapping of various forms has been around for at least 15 years. A range of different approaches to evidence mapping and synthesis have been developed to support evidence informed policy making. In 2016, (Miake-Lye, Hempel, Shanman, & Shekelle, 2016) conducted a systematic review of evidence maps in the health sector. The authors identified 145 titles, of these, 34 publications explicitly presented evidence maps and five publications used a mapping methodology without presenting a map. The authors conclude that there is diversity in definition and methods for mapping, so that stakeholders cannot necessarily know what to expect if they commission an evidence map or seek to identify existing maps. In addition, as there is no repository for evidence maps, maps are difficult to locate and authors are less likely to build on existing approaches. One purpose of this paper is to propose some general principles for evidence maps. Maps often provide a foundation for further, more focused research synthesis. They are used to guide users to high quality research, inform research priority setting, and help to define the focus of evidence synthesis such as systematic reviews (Katz, et al., 2003). In the health sector maps were intended to provide clinicians, researchers, clients, and caregivers with tools and guidance to engage in evidence-based decision making. Maps allow for identification of research gaps which is particularly important for interventions that are implemented without insufficient evidence (Katz, et al., 2003). The popularity of mapping has grown in recent years. A map of maps for international development interventions identified 73 maps, with half of all completed maps commissioned in 2016 (Phillips, et al., 2017). This paper reviews the range of mapping methodologies being adopted across different sectors. The motivation for this paper is to develop general principles for maps, and particularly to inform the development of standards for the Campbell Collaboration as a basis for the publication of Evidence and Gap Maps in the Campbell Library. The approaches discussed include systematic map (Bates, Clapton, & Coren, 2007); (Oakley, Gough, Oliver, & Thomas, 2005); (Salina Bates, 2006); (Peersman, 1996) evidence maps (Liu, Parker, Hetrick, & Callahan, 2010), the evidence-based policing matrix (Lum, Koper, & Telep, 2011), Evidence Mapping (Parkhill, et al., 2011); (Bragge, et al., 2011); Scientific uncertainties (Jacobson, 2014), and Evidence Gap Maps (Snilstveit, Vojtkova, Bhavsar, & Gaarder, 2013). Papers mentioned above have discussed and presented overview of different types of evidence mapping, scoping and synthesis. The focus is on the various approaches towards evidence syntheses products whereas the focus of this paper is on the methodological approach of organisations working on evidence maps. The purposes of this paper are to: We present a brief history of evidence mapping before turning to an analysis of the approach to evidence mapping broken down into stages: definition of map, scope, inclusion criteria, critical appraisal, screening and searching, coding, presentation and reporting, use and maintenance. We conclude with some general principles for evidence mapping. The earliest evidence maps we identified are from 2003. That year saw publication of a map by the Yale Prevention Research Centre in collaboration with Complementary and Alternative Medicine (CAM) (Katz, Williams, Girard, & Goodman, 2003); see Table 1. The same year the Evidence for Policy and Practice Information and Coordinating Centre (EPPI-Center) published a review of personal development planning for improving student learning, distinguishing between the systematic map of the research which been undertaken and the systematic synthesis of what the evidence says (Gough, Kiwan, Sutcliffe, Simpson, & Houghton, 2003). Two years later the first visual map was published as the Evidence Based Policing Matrix (Rosenberg & Knox, 2005). In 2005 EPPI-Centre published a mapping methodology which was adapted the following year by the Social Care Institute for Excellence (SCIE) in response to a lack of empirical data to answer specific questions using systematic reviews, and a perceived need for a methodology to describe the literature in a broader field of interest than that typically covered in a single review. In 2010, 3ie produced its
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Scoping studies: towards a methodological framework
Arksey, H., & O'Malley, L. (2005) · International Journal of Social Research Methodology · doi:10.1080/1364557032000119616 · Cited by 36,742
A methodology for systematic mapping in environmental sciences
James, K. L., Randall, N. P., & Haddaway, N. R. (2016) · Environmental Evidence · doi:10.1186/s13750-016-0059-6 · Cited by 769
Miake-Lye, I. M., Hempel, S., Shanman, R., & Shekelle, P. G. (2016) · Systematic Reviews · doi:10.1186/s13643-016-0204-x · Cited by 660
The Global Evidence Mapping Initiative: Scoping research in broad topic areas
Bragge, P., Clavisi, O., Turner, T., Tavender, E., Collie, A., & Gruen, R. L. (2011) · BMC Medical Research Methodology · doi:10.1186/1471-2288-11-92 · Cited by 310
Do immigrant workers depress the wages of native workers?
National Bureau of Economic Research, U., & Peri, G. (2014) · IZA World of Labor · doi:10.15185/izawol.42 · Cited by 151
Searches for evidence mapping: effective, shorter, cheaper
Parkhill, A. F., Clavisi, O., Pattuwage, L., Chau, M., Turner, T., Bragge, P., & Gruen, R. (2011) · Journal of the Medical Library Association : JMLA · doi:10.3163/1536-5050.99.2.008 · Cited by 22
Cited by
Campbell, F., Tricco, A. C., Munn, Z., Pollock, D., Saran, A., Sutton, A., White, H., & Khalil, H. (2023) · Systematic Reviews · doi:10.1186/s13643-023-02178-5 · Cited by 345
Self-citationBender, A. E., McKinney, S. J., Schmidt-Sane, M. M., Cage, J., Holmes, M. R., Berg, K. A., Salley, J., Bodell, M., Miller, E. K., & Voith, L. A. (2022) · Journal of Family Violence · doi:10.1007/s10896-021-00315-z · Cited by 88
Scoping reviews and rapid reviews
Sabiston, C. M., Vani, M., de Jonge, M., & Nesbitt, A. (2022) · International Review of Sport and Exercise Psychology · doi:10.1080/1750984x.2021.1964095 · Cited by 74
“…for data synthesis (e.g. meta-narrative, evidence mapping), and utilize unique reporting methods for data display, including traditional summary tables, as well as figures, conceptual models and timelines (Greenhalgh et al., 2005; Lockwood et al., 2019; Miake-Lye et al., 2016; Saran & White, 2018).”
MethodEvidence synthesis for tackling research waste
Grainger, M. J., Bolam, F. C., Stewart, G. B., & Nilsen, E. B. (2020) · Nature Ecology & Evolution · doi:10.1038/s41559-020-1141-6 · Cited by 59
Advancing the methodology of mapping reviews: A scoping review
Khalil, H., Campbell, F., Danial, K., Pollock, D., Munn, Z., Welsh, V., Saran, A., Hoppe, D., & Tricco, A. C. (2024) · Research Synthesis Methods · doi:10.1002/jrsm.1694 · Cited by 59
Self-citationEvans, K. E., Schmidt-Sane, M. M., Bender, A. E., Berg, K. A., & Holmes, M. R. (2022) · Journal of Family Violence · doi:10.1007/s10896-021-00318-w · Cited by 33
Meta-epidemiology and reporting characteristics of mapping reviews: a scoping review
Li, Y., Ghogomu, E., Khalil, H., Hui, X., E, F., Campbell, F., Li, X., Gaarder, M., Nduku, P. M., White, H., et al. (2025) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2024.111648 · Cited by 2
Self-citation
Metadata
- Authors
- Saran, A., & White, H.
- Venue
- Campbell Systematic Reviews
- Publisher
- Wiley
- Published
- 2018-01
- Volume
- 14
- Issue
- 1
- Pages
- 1-38
- ISSN
- 1891-1803 (ISSN-L)
- OpenAlex
- W2963993928
- Cited by
- 185
- References
- 6
- FWCI
- 9.25
- Type
- journal article
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.