CoChat
Sign Up
Cited by 116Health Policy Implementation ScienceHealth Education and ValidationPsychometric Methodologies and Testing

Psychometric properties of implementation measures for public health and community settings and mapping of constructs against the Consolidated Framework for Implementation Research: a systematic review

Open access
Clinton-McHarg, Tara, Yoong, Sze Lin, Tzelepis, Flora, Regan, Tim, Fielding, Alison, Skelton, Eliza, Kingsland, Melanie, Ooi, Jia Ying +1 more·2016·Implementation Science
DOI 10.1186/s13012-016-0512-5PMC PMC5100177
Open this paper in CoChat — free accountOpen PDF

Abstract

BACKGROUND: Recent reviews have synthesised the psychometric properties of measures developed to examine implementation science constructs in healthcare and mental health settings. However, no reviews have focussed primarily on the properties of measures developed to assess innovations in public health and community settings. This review identified quantitative measures developed in public health and community settings, examined their psychometric properties, and described how the domains of each measure align with the five domains and 37 constructs of the Consolidated Framework for Implementation Research (CFIR). METHODS: MEDLINE, PsycINFO, EMBASE, and CINAHL were searched to identify publications describing the development of measures to assess implementation science constructs in public health and community settings. The psychometric properties of each measure were assessed against recommended criteria for validity (face/content, construct, criterion), reliability (internal consistency, test-retest), responsiveness, acceptability, feasibility, and revalidation and cross-cultural adaptation. Relevant domains were mapped against implementation constructs defined by the CFIR. RESULTS: Fifty-one measures met the inclusion criteria. The majority of these were developed in schools, universities, or colleges and other workplaces or organisations. Overall, most measures did not adequately assess or report psychometric properties. Forty-six percent of measures using exploratory factor analysis reported >50 % of variance was explained by the final model; none of the measures assessed using confirmatory factor analysis reported root mean square error of approximation (<0.06) or comparative fit index (>0.95). Fifty percent of measures reported Cronbach's alpha of <0.70 for at least one domain; 6 % adequately assessed test-retest reliability; 16 % of measures adequately assessed criterion validity (i.e. known-groups); 2 % adequately assessed convergent validity (r > 0.40). Twenty-five percent of measures reported revalidation or cross-cultural validation. The CFIR constructs most frequently assessed by the included measures were relative advantage, available resources, knowledge and beliefs, complexity, implementation climate, and other personal resources (assessed by more than ten measures). Five CFIR constructs were not addressed by any measure. CONCLUSIONS: This review highlights gaps in the range of implementation constructs that are assessed by existing measures developed for use in public health and community settings. Moreover, measures with robust psychometric properties are lacking. Without rigorous tools, the factors associated with the successful implementation of innovations in these settings will remain unknown.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/s13012-016-0512-5

Cites

229 of 230 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

4 of the 234 references couldn’t be matched to a known paper.

Research works printed but not linked: 4

  1. [20]The SIRC Instrument Review Project (IRP): A systematic review and synthesis of implementation science instruments [ http://www.societyforimplementationresearchcollaboration.org/sirc-projects/sirc-instrument-project]
  2. [120]Rogers EM Diffusion of innovations 1983 3 New York Free Press
  3. [136]Moos RH The social climate scales: an overview 1974 Palo Alto Consulting Psychologists Press
  4. [218]Hackman JR Oldham GR Work redesign 1980 Reading Addison-Wesley

Cited by

Sorted by citations

Related

Authors
Clinton-McHarg, T., Yoong, S. L., Tzelepis, F., Regan, T., Fielding, A., Skelton, E., Kingsland, M., Ooi, J. Y., & Wolfenden, L.
Affiliations
  • Hunter Medical Research Institute (AU)
  • University of Newcastle Australia (AU)
  • Hunter New England Local Health District (AU)
Corresponding author
Wolfenden, Luke
ORCID
Venue
Implementation Science
Publisher
Springer Science and Business Media LLC
Published
2016-11-08
Volume
11
Issue
1
Pages
148-148
ISSN
1748-5908 (ISSN-L)
DOI
10.1186/s13012-016-0512-5
PMC
PMC5100177
OpenAlex
W2553855586
Cited by
116
References
234
Citations by year
2026: 6 · 2025: 9 · 2024: 11 · 2023: 16 · 2022: 12 · 2021: 15 · 2020: 14 · 2019: 15 · 2018: 11 · 2017: 6 · 2016: 1
FWCI
2.10
Type
journal article
Language
en
Open access
Licence
Locations
  • Implementation Science (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
  • Figshare (submittedVersion, cc-by)
  • Swinburne figshare (Swinburne University of Technology) (submittedVersion)
  • NOVA (University of Newcastle Australia) (submittedVersion, cc-by)
Funders
Hunter Medical Research Institute
Topics
  • Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
  • Social Sciences › Education › Health Education and Validation
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
Keywords
psychometric properties, Consolidated Framework for Implementation Research, public health settings, community settings, measurement validity, reliability statistics, construct validity, cross-cultural adaptation, implementation climate, implementation barriers and facilitators
MeSH terms
Community Health Services, Diffusion of Innovation, Feasibility Studies, Humans, Psychometrics, Public Health, Surveys and Questionnaires, Reproducibility of Results, Translational Research, Biomedical, Clinical Coding

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review