Asking the right questions: Scoping studies in the commissioning of research on the organisation and delivery of health services
Open accessThe DOI matches its Crossref record; 43 of the 44 references it cites were found in a registry or an index; 3 of the 39 sources its claims cite were checked for support.
Abstract
Scoping studies have been used across a range of disciplines for a wide variety of purposes. However, their value is increasingly limited by a lack of definition and clarity of purpose. The UK's Service Delivery and Organisation Research Programme (SDO) has extensive experience of commissioning and using such studies; twenty four have now been completed.This review article has four objectives; to describe the nature of the scoping studies that have been commissioned by the SDO Programme; to consider the impact of and uses made of such studies; to provide definitions for the different elements that may constitute a scoping study; and to describe the lessons learnt by the SDO Programme in commissioning scoping studies.Scoping studies are imprecisely defined but usually consist of one or more discrete components; most commonly they are non-systematic reviews of the literature, but other important elements are literature mapping, conceptual mapping and policy mapping. Some scoping studies also involve consultations with stakeholders including the end users of research.Scoping studies have been used for a wide variety of purposes, although a common feature is to identify questions and topics for future research. The reports of scoping studies often have an impact that extends beyond informing research commissioners about future research areas; some have been published in peer reviewed journals, and others have been published in research summaries aimed at a broader audience of health service managers and policymakers.Key lessons from the SDO experience are the need to relate scoping studies to a particular health service context; the need for scoping teams to be multi-disciplinary and to be given enough time to integrate diverse findings; and the need for the research commissioners to be explicit not only about the aims of scoping studies but also about their intended uses. This necessitates regular contact between researchers and commissioners.Scoping studies are an essential element in the portfolio of approaches to research, particularly as a mechanism for helping research commissioners and policy makers to ask the right questions. Their utility will be further enhanced by greater recognition of the individual components, definitions for which are provided.
Overview
CoChat Summary
## TL;DR
This review argues that “scoping study” is too broad a label unless its components are specified, and shows that the 24 scoping studies commissioned by the SDO Programme were most often combinations of literature mapping, conceptual mapping, policy mapping, and stakeholder consultation. It concludes that scoping studies are useful for defining research questions, identifying gaps, and shaping later research commissioning, but their value depends on context, multidisciplinary teams, enough time, and clear commissioner–researcher agreement.
## Key findings
- The SDO Programme had commissioned 24 scoping studies by 31 August 2007, out of 205 projects commissioned in total.
- The paper says scoping studies most commonly consist of non-systematic literature reviews, but in the SDO experience they also included literature mapping, conceptual mapping, policy mapping, and stakeholder consultations.
- The authors report that scoping studies often had impacts beyond identifying future research: some led to briefing papers, peer-reviewed publications, books, and even subsequent empirical research; for example, the continuity of care scoping study led directly to five large empirical studies.
- The workforce scoping exercises found that research in that area was largely based on small case studies, was rarely underpinned by sound theory, and failed to constitute a cumulative body of knowledge; these studies then formed the basis of the SDO Programme’s later workforce research, with nine empirical studies commissioned.
## Methods
- Review article based on the authors’ experience of commissioning scoping studies for the NIHR Service Delivery and Organisation (SDO) Programme and a review of the existing literature on scoping studies.
- The programme’s research database was examined to identify projects that did not involve empirical research and matched inclusion criteria such as scoping in the title, non-systematic literature reviews, concept reviews, consultation exercises tied to research agendas, and policy document reviews; only projects with final reports accepted by 31 August 2007 were included.
- The paper synthesizes examples from the SDO Programme to define the main components of scoping studies and to describe their uses, impact, and commissioning lessons.
## Limitations
- The paper explicitly states that scoping studies are often commissioned in a “rapid response” mode, but in complex areas they may require “sense-making” across disciplines, so a study of appropriate quality may be no more speedy than a traditional systematic review.
- It also states that scoping studies can suffer from disciplinary bias and that the commissioner must manage trade-offs between rapid appraisal and quality; detailed commissioning briefs and peer review may be needed.
- Evident from the text: the review is descriptive and based on the SDO Programme’s experience, so its conclusions are grounded in one commissioning context rather than a comparative empirical evaluation of scoping studies across settings.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
3 Valid · 19 Not valid · 24 Not checked · 79 Own results and interpretations
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Two of the teams who have carried out scoping studies commissioned by the SDO Programme (those on continuity of care and on services to support carers of people with mental health problems) have described their experience of undertaking scoping studies in the academic literature [81,82].”
§ Asking the right questions: Scoping studies in the commissioning of research on the organisation and delivery of health services > Background > Lessons learned by the SDO Programme in commissioning scoping studies
What the sources say
- Not checked yetcites [81] Ehrich K, Freeman G, Richards S, Robinson I, Shepperd S: How to do a scoping exercise: continuity of care. Research Policy and Planning. 2002, 20 (1): 25-29.
- Supportedcites [82] Arksey H, O'Malley L: Scoping studies: towards a methodological framework. International Journal of Social Research Methodology. 2005, 8: 1-14. 10.1080/1364557032000119616. source
Split into 2 atoms — 2 confirmed, 0 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“twenty four have now been completed.”
- Also stated in: § Background · plain
“The SDO Programme has now commissioned a total of twenty four scoping studies.”
- Also stated in: § Nature of scoping studies commissioned by the SDO Programme · plain
“Of these twenty four were classified as scoping studies.”
§ Asking the right questions: Scoping studies in the commissioning of research on the organisation and delivery of health services > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Background · plain
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These scoping studies formed the basis of the SDO Programme's subsequent programme of research on the healthcare workforce.”
§ Asking the right questions: Scoping studies in the commissioning of research on the organisation and delivery of health services > Background > Workforce scoping studies
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“was rarely underpinned by sound theory”
§ Asking the right questions: Scoping studies in the commissioning of research on the organisation and delivery of health services > Background > Workforce scoping studies
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“They demonstrated that research in this area was largely based on small case studies”
§ Asking the right questions: Scoping studies in the commissioning of research on the organisation and delivery of health services > Background > Workforce scoping studies
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
26 more claims
Create a free account to see all 125 claims and to verify the 36 references still open.
- Probably not a substantial claim. 87
- It may state two separate claims as one. 33
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“The general purpose for conducting scoping reviews is to identify and map the available evidence [5,22].”
From Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach (2018) · cites this paper as [22]
In the citing paper
“However, no universal scoping study definition or purpose exists (Table 1) [1,2].”
From Scoping studies: advancing the methodology (2010) · cites this paper as [2]
In the citing paper
“Anderson et al. [2] offer criteria for assessing the value and utility of a commissioned scoping study in health policy contexts, but these criteria are not necessarily applicable to scoping studies in other areas of health research.”
From Scoping studies: advancing the methodology (2010) · cites this paper as [2]
Citations
Cites
Scoping studies: towards a methodological framework
Arksey, H., & O'Malley, L. (2005) · International Journal of Social Research Methodology · doi:10.1080/1364557032000119616 · Cited by 36,742 · cited 1×
“Two of the teams who have carried out scoping studies commissioned by the SDO Programme (those on continuity of care and on services to support carers of people with mental health problems) have described their experience of undertaking scoping studies in the academic literature [81,82].”
Systematic Reviews in Health Care : Meta-Analysis in Context
Egger, M., Smith, G. D., & Altman, D. G. (2000) · Cited by 2,401
Systematic Reviews in Health Care: Meta‐Analysis in Context
(2001) · doi:10.1002/9780470693926 · Cited by 1,693
Doing a literature review: releasing the social science research imagination.
Hart, C. (1998) · Sage eBooks · Cited by 1,378
Mays, N., Pope, C., & Popay, J. (2005) · Journal of Health Services Research & Policy · doi:10.1258/1355819054308576 · Cited by 1,375
Using evidenceHow research can inform public services
Nutley, S. M., Walter, I., & Davies, H. T. O. (2007) · Policy Press eBooks · doi:10.1332/policypress/9781861346650.001.0001 · Cited by 1,335
Using Evidence: How Research Can Inform Public Services
Nutley, S., Walter, I., & Davies, H. T. O. (2007) · Bristol University Press eBooks · doi:10.46692/9781847422323 · Cited by 1,299
Using evidence: How research can inform public services
Nutley, S. M., Walter, I., & Davies, H. T. O. (2007) · doi:10.2307/j.ctt9qgwt1 · Cited by 808
The utilisation of health research in policy-making: concepts, examples and methods of assessment
Hanney, S. R., Gonzalez-Block, M. A., Buxton, M. J., & Kogan, M. (2003) · Health Research Policy and Systems · doi:10.1186/1478-4505-1-2 · Cited by 678
Consensus Development Methods: A Review of Best Practice in Creating Clinical Guidelines
Black, N., Murphy, M., Lamping, D., McKee, M., Sanderson, C., Askham, J., & Marteau, T. (1999) · Journal of Health Services Research & Policy · doi:10.1177/135581969900400410 · Cited by 451 · cited 1×
“There is now a substantial literature on establishing research priorities and the techniques available to assist with this, such as consensus development, a technique that has been used in several areas including critical care [20], on the interface between primary and secondary care [21], and in guideline development…”
Research works printed but not linked: 41
- Harper RJ, Hatton TJ, Crombie DS, Dawes WR, Abbott LK, Challen RP, House C: Phase farming with trees: a scoping study of its potential for salinity control and soil quality. 2000, Kingston, ACT, Australia: Rural Industries Research and Development Corporation
- Turrall A, Mulhall A, Rees D, Okwadi J, Emerot J, Omadi R: Understanding the communication context in Teso and Lango farming systems: The agricultural information scoping study. 2000, London: Department for International Development
- Grayson L, Gomersall A: A difficult business: finding the evidence for social science reviews. Working Paper 19. 2003, London: ESRC Centre for Evidence Based Policy and Practice, Queen Mary College
- National Institutes of Health Working Group on Priority Setting: Setting research priorities at the National Institutes of Health. Publication number 97-4265. 2007, Bethesda MD: NIH
- Goodwin N, Allen P, Anderson S, Gagnon D, Grant R, Law S, Peckham S: Influencing policy makers and practitioners: towards a new model in the provision of evidence-based research to support decision-making. Paper given at Health Services Research Network Conference, Sheffield. 2007
- Ferlie E, McGivern G: Relationships between health care organisations: a critical overview of the literature and a research agenda. 2003, London: NCC SDO (SDO/19a/2002)
- Hunter D: Scoping study of the public health system in England. 2007, London: NCC SDO (SDO/150/2006)
- Gladman J: Specialist rehabilitation for neurological conditions: a literature review and mapping study. 2007, London: NCC SDO (SDO/132/2006)
- Burns T: Measuring outcomes for carers for people with mental health problems. 2004, London: NCC SDO (SDO/81/2004)
- Fulop N, Allen P: National listening exercise: report of the findings. 2000, London: NCC SDO
- Gulliford M, Morgan M, Hughes D, Beech R, Figeroa-Munoz J, Gibson B, Hudson M, Arumugam C, Connell P, Mohiddin A, Sedgwick J: Access to health care: report of a scoping exercise. 2001, London: NCC SDO (SDO/05/2000)
- Rosen R, Florin D, Dixon J: Access to health care: taking forward the findings from the scoping exercise. 2001, London: NCC SDO (SDO/12/2000)
- Carr-Hill R, Currie L, Dixon P: Skill mix in secondary care: a scoping exercise. 2003, London: NCC SDO (SDO/51/2003)
- Hewitt C, Lankshear A, Maynard A, Sheldon T, Smith K: Health service workforce and health outcomes: a scoping study. 2003, London: NCC SDO (SDO/50/2003)
- Elliott R, Scott A, Skatun D, Farrar S, Napper M: The impact of local labour market factors on the organisation and delivery of health services. 2003, London: NCC SDO (SDO/52/2003)
- Jones R: E-health policy context and consultation with stakeholders. 2003, London: NCC SDO (SDO/59/2003)
- Pagliari C: Literature review and conceptual map of the area of e-health. 2003, London: NCC SDO (SDO/60/2003)
- Cherry C: Review of scoping exercises commissioned by the SDO Programme 2000–02: opportunities for commissioning. 2000, London: NCC SDO
- Ross F, MacKenzie A, Smith E, Masterson A, Wood C: Identifying research priorities in nursing and midwifery service delivery and organisation. 2002, London: NCC SDO (SDO/20/2002)
- Keating F, Samele C, Furnish K, Greatley A: Research priorities for service user and carer-centred mental health services: a synthesis of the literature and policy documents. 2007, London: NCC SDO (SDO/85/2005)
- Naylor C, Wallcraft J, Samele C, Greatley A: Research Priorities for service user and carer-centred mental health services: a consultation report. 2007, London: NCC SDO (SDO/86/2005)
- Samele C, Wallcraft J, Naylor C, Keating F, Greatley A: Research priorities for service user and carer-centred mental health services: overview report. 2007, London: NCC SDO (SDO/85/86/2005)
- Gulliford M, Morgan M: Access to health care. 2003, London: Routledge
- Freeman G, Woloshynowych M, Baker R, Boulton M, Guthrie B, Car J, Haggerty J, Tarrant C: Continuity of Care 2006: What have we learned since 2000 and what are the policy imperatives now?. 2007, London:NCC SDO (SDO/138/2006)
- Freeman G, Sheppard S, Robinson I, Ehrich K, Richards S: Continuity of care: report of a scoping exercise. 2000, London: NCC SDO (SDO/02/2000)
- Gulliford M, Morgan M, Hughes D, Beech R, Figeroa-Munoz J, Gibson B, Hudson M, Arumugam C, Connell P, Mohiddin A, Sedgwick J: Access to health care: report of a scoping exercise. 2001, London: NCC SDO (SDO/05/2000)
- Horne R: Concordance, adherence and compliance in medicine taking: a scoping exercise. 2004, London: NCC SDO (SDO/76/2004)
- Lamb S, Gates S, Fisher J, Cooke M, Carter Y, McCabe C: Scoping exercise on fallers' clinics. 2007, London: NCC SDO (SDO/139/2006)
- Clarke A, Pillaye J: National policy initiatives: implications for the SDO Programme. 2002, London: NCC SDO
- Dalziel M: Using research to improve health care services. 2000, London: NCC SDO
- Fulop N, Allen P: National listening exercise: report of the findings. 2000, London: NCC SDO
- Cherry C, Anderson S: Refreshing the national listening exercise: report of the findings. 2002, London: NCC SDO
- Clarke A, Cherry C, Anderson S: Informing commissioning priorities: overview report. 2002, London: NCC SDO
- Lomas J, Fulop N, Gagnon D, Allen P: On being a good listener: setting priorities for applied health services research. Milbank Quarterly. 2003, 81 (3): 363-388. 10.1111/1468-0009.t01-1-00060. · DOI · Paper page
- Nicolson P: Eliciting users' views of the processes of health care: a scoping exercise. 2000, London: NCC SDO (SDO/03/2000)
- While A, Forbes A, Ullman R, Murgatroyd B: A scoping exercise to conceptualise the actual and potential contribution of nurses, midwives and health visitors to child health and child health services. 2004, London: NCC SDO (SDO/68/2004)
- Allen P, Fulop N, Clarke A: Commissioning scoping exercises: experience to date and lessons learnt. 2001, London: NCC SDO
- Ehrich K, Freeman G, Richards S, Robinson I, Shepperd S: How to do a scoping exercise: continuity of care. Research Policy and Planning. 2002, 20 (1): 25-29.
- Arksey H, O'Malley L: Scoping studies: towards a methodological framework. International Journal of Social Research Methodology. 2005, 8: 1-14. 10.1080/1364557032000119616. · DOI · Paper page
- Arksey H: Mental health scoping exercise: research on services to support carers for people with mental health problems. 2001, London: NCC SDO (SDO/15/2001)
- Gysels M, Higginson I, White P, Barclay S, Worth A, Murray S, Shipman C, Dewar S, Richardson A, Hotopf M, Lorenz K, Koffman J: A scoping exercise on generalist services for adults at the end of life: research, knowledge, policy and future research needs. 2007, London: NCC SDO (SDO/143/2006)
Cited by
Scoping studies: advancing the methodology
Levac, D., Colquhoun, H., & O'Brien, K. K. (2010) · Implementation Science · doi:10.1186/1748-5908-5-69 · Cited by 15,268
“While Arksey and O’Malley [6] do not indicate a team approach is imperative, we agree with others and suggest scoping studies involve multidisciplinary teams using a transparent and replicable process [2,10].”
InfluentialMunn, Z., Peters, M. D. J., Stern, C., Tufanaru, C., McArthur, A., & Aromataris, E. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0611-x · Cited by 11,385
Updated methodological guidance for the conduct of scoping reviews
Peters, M. D. J., Marnie, C., Tricco, A. C., Pollock, D., Munn, Z., Alexander, L., McInerney, P., Godfrey, C. M., & Khalil, H. (2020) · JBI Evidence Synthesis · doi:10.11124/jbies-20-00167 · Cited by 9,046
Guidance for conducting systematic scoping reviews
Peters, M. D. J., Godfrey, C. M., Khalil, H., McInerney, P., Parker, D., & Soares, C. B. (2015) · International Journal of Evidence-Based Healthcare · doi:10.1097/xeb.0000000000000050 · Cited by 7,472
“healthcare discipline or academic field).(4) The value of scoping reviews to evidence-based practice is the examination of a broader area to identify gaps in the research knowledge base,(9) clarify key concepts,(10) and report on the types of evidence that address and inform practice in the field.”
A scoping review of scoping reviews: advancing the approach and enhancing the consistency
Pham, M. T., Rajić, A., Greig, J. D., Sargeant, J. M., Papadopoulos, A., & McEwen, S. A. (2014) · Research Synthesis Methods · doi:10.1002/jrsm.1123 · Cited by 3,078
“(Anderson et al., 2008).”
Influential
Metadata
- Authors
- Anderson, S., Allen, P., Peckham, S., & Goodwin, N.
- Affiliations
- London School of Hygiene & Tropical Medicine (GB)
- Corresponding author
- Anderson, Stuart
- Venue
- Health Research Policy and Systems
- Published
- 2008-07-09
- Volume
- 6
- Issue
- 1
- Pages
- 7-7
- ISSN
- 1478-4505 (ISSN-L)
- PMC
- PMC2500008
- OpenAlex
- W1981119137
- Cited by
- 653
- References
- 44
- Citations by year
- 2026: 36 · 2025: 58 · 2024: 54 · 2023: 54 · 2022: 71 · 2021: 84 · 2020: 57 · 2019: 36 · 2018: 33 · 2017: 33 · 2016: 31 · 2015: 25 · 2014: 19 · 2013: 30 · 2012: 15
- FWCI
- 10.96
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/2.0 (tdm)
- Locations
- Health Research Policy and Systems (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (acceptedVersion)
- LSHTM Research Online (London School of Hygiene and Tropical Medicine) (acceptedVersion)
- Kent Academic Repository (University of Kent) (submittedVersion)
- Funders
- National Institute for Health and Care Research
- Service Delivery and Organisation Programme
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Health Professions › General Health Professions › Health Services Management and Policy
- Keywords
- scoping review, research commissioning, health care organization, health care delivery, literature mapping, conceptual mapping, policy mapping, stakeholder consultation, research questions, research impact
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Asking the right questions: Scoping studies in the commissioning of research on the organisation and delivery of health services. https://app.cochat.ai/papers/10.1186/1478-4505-6-7
- The paper
- Anderson, S., Allen, P., Peckham, S., & Goodwin, N. (2008). Asking the right questions: Scoping studies in the commissioning of research on the organisation and delivery of health services. Health Research Policy and Systems. https://doi.org/10.1186/1478-4505-6-7
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.