Cited by 195Health Policy Implementation ScienceDelphi Technique in ResearchClinical practice guidelines implementation

Improving the normalization of complex interventions: part 1 - development of the NoMAD instrument for assessing implementation work based on normalization process theory (NPT)

Open access
Rapley, Tim, Girling, Melissa, Mair, Frances S., Murray, Elizabeth, Treweek, Shaun, McColl, Elaine, Steen, Ian Nicholas, May, Carl R. +1 more·2018·BMC Medical Research Methodology

Abstract

BACKGROUND: Understanding and measuring implementation processes is a key challenge for implementation researchers. This study draws on Normalization Process Theory (NPT) to develop an instrument that can be applied to assess, monitor or measure factors likely to affect normalization from the perspective of implementation participants. METHODS: An iterative process of instrument development was undertaken using the following methods: theoretical elaboration, item generation and item reduction (team workshops); item appraisal (QAS-99); cognitive testing with complex intervention teams; theory re-validation with NPT experts; and pilot testing of instrument. RESULTS: We initially generated 112 potential questionnaire items; these were then reduced to 47 through team workshops and item appraisal. No concerns about item wording and construction were raised through the item appraisal process. We undertook three rounds of cognitive interviews with professionals (n = 30) involved in the development, evaluation, delivery or reception of complex interventions. We identified minor issues around wording of some items; universal issues around how to engage with people at different time points in an intervention; and conceptual issues around the types of people for whom the instrument should be designed. We managed these by adding extra items (n = 6) and including a new set of option responses: 'not relevant at this stage', 'not relevant to my role' and 'not relevant to this intervention' and decided to design an instrument explicitly for those people either delivering or receiving an intervention. This version of the instrument had 53 items. Twenty-three people with a good working knowledge of NPT reviewed the items for theoretical drift. Items that displayed a poor alignment with NPT sub-constructs were removed (n = 8) and others revised or combined (n = 6). The final instrument, with 43 items, was successfully piloted with five people, with a 100% completion rate of items. CONCLUSION: The process of moving through cycles of theoretical translation, item generation, cognitive testing, and theoretical (re)validation was essential for maintaining a balance between the theoretical integrity of the NPT concepts and the ease with which intended respondents could answer the questions. The final instrument could be easily understood and completed, while retaining theoretical validity. NoMAD represents a measure that can be used to understand implementation participants' experiences. It is intended as a measure that can be used alongside instruments that measure other dimensions of implementation activity, such as implementation fidelity, adoption, and readiness.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/s12874-018-0590-y

Cites

58 of 58 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Rapley, T., Girling, M., Mair, F. S., Murray, E., Treweek, S., McColl, E., Steen, I. N., May, C. R., & Finch, T. L.
Affiliations
  • Northumbria University (GB)
  • Newcastle University (GB)
  • University of Glasgow (GB)
  • The Royal Free Hospital (GB)
  • University of Aberdeen (GB)
  • London School of Hygiene & Tropical Medicine (GB)
Corresponding author
Rapley, Tim
Venue
BMC Medical Research Methodology
Published
2018-11-15
Volume
18
Issue
1
Pages
133-133
ISSN
1471-2288 (ISSN-L)
OpenAlex
W2982498515
Cited by
193
References
57
Citations by year
2026: 19 · 2025: 31 · 2024: 28 · 2023: 23 · 2022: 37 · 2021: 30 · 2020: 16 · 2019: 7 · 2018: 2
FWCI
21.00
Type
journal article
Language
en
Open access
Locations
  • BMC Medical Research Methodology (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, other-oa)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Northumbria Research Link (Northumbria University) (acceptedVersion)
  • ENLIGHTEN (Jurnal Bimbingan dan Konseling Islam) (acceptedVersion, cc-by-sa)
  • LSHTM Research Online (London School of Hygiene and Tropical Medicine) (acceptedVersion)
  • Aberdeen University Research Archive (Aberdeen University) (submittedVersion, other-oa)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • UCL Discovery (University College London) (submittedVersion)
Funders
  • Economic and Social Research Council (RES-062-23-3274)
  • National Institute for Health Research (Collaboration for Leadership in Applied Health Research and Care North Thames., School for Primary Care Research)
Topics
  • Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
  • Social Sciences › Sociology and Political Science › Delphi Technique in Research
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
Keywords
Normalization Process Theory, complex interventions, implementation research, NOMAD instrument, instrument development, cognitive interviewing, item generation, theory validation
MeSH terms
Implementation Science, Cognition, Delivery of Health Care, Health Services Research, Humans, Interview, Psychological, Pilot Projects, Surveys and Questionnaires

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review