CoChat
Sign Up
Cited by 116Health Policy Implementation ScienceHealth Education and ValidationPsychometric Methodologies and Testing

The DOI matches its Crossref record; 229 of the 234 references it cites were found in a registry or an index, and 1 are not checked yet.

Psychometric properties of implementation measures for public health and community settings and mapping of constructs against the Consolidated Framework for Implementation Research: a systematic review

Open access
Clinton-McHarg, Tara, Yoong, Sze Lin, Tzelepis, Flora, Regan, Tim, Fielding, Alison, Skelton, Eliza, Kingsland, Melanie, Ooi, Jia Ying, Wolfenden, Luke·2016·Implementation Science
DOI 10.1186/s13012-016-0512-5PMC PMC5100177
Open this paper in CoChat — free accountOpen PDF

Abstract

BACKGROUND: Recent reviews have synthesised the psychometric properties of measures developed to examine implementation science constructs in healthcare and mental health settings. However, no reviews have focussed primarily on the properties of measures developed to assess innovations in public health and community settings. This review identified quantitative measures developed in public health and community settings, examined their psychometric properties, and described how the domains of each measure align with the five domains and 37 constructs of the Consolidated Framework for Implementation Research (CFIR). METHODS: MEDLINE, PsycINFO, EMBASE, and CINAHL were searched to identify publications describing the development of measures to assess implementation science constructs in public health and community settings. The psychometric properties of each measure were assessed against recommended criteria for validity (face/content, construct, criterion), reliability (internal consistency, test-retest), responsiveness, acceptability, feasibility, and revalidation and cross-cultural adaptation. Relevant domains were mapped against implementation constructs defined by the CFIR. RESULTS: Fifty-one measures met the inclusion criteria. The majority of these were developed in schools, universities, or colleges and other workplaces or organisations. Overall, most measures did not adequately assess or report psychometric properties. Forty-six percent of measures using exploratory factor analysis reported >50 % of variance was explained by the final model; none of the measures assessed using confirmatory factor analysis reported root mean square error of approximation (<0.06) or comparative fit index (>0.95). Fifty percent of measures reported Cronbach's alpha of <0.70 for at least one domain; 6 % adequately assessed test-retest reliability; 16 % of measures adequately assessed criterion validity (i.e. known-groups); 2 % adequately assessed convergent validity (r > 0.40). Twenty-five percent of measures reported revalidation or cross-cultural validation. The CFIR constructs most frequently assessed by the included measures were relative advantage, available resources, knowledge and beliefs, complexity, implementation climate, and other personal resources (assessed by more than ten measures). Five CFIR constructs were not addressed by any measure. CONCLUSIONS: This review highlights gaps in the range of implementation constructs that are assessed by existing measures developed for use in public health and community settings. Moreover, measures with robust psychometric properties are lacking. Without rigorous tools, the factors associated with the successful implementation of innovations in these settings will remain unknown.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/s13012-016-0512-5

Cites

229 of 230 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1 PRISMA flow diagram of the publication and measure inclusion process
Fig. 1 PRISMA flow diagram of the publication and measure inclusion processCC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Items 201–230 of 230

Research works printed but not linked: 14

  1. The SIRC Instrument Review Project (IRP): A systematic review and synthesis of implementation science instruments [ http://www.societyforimplementationresearchcollaboration.org/sirc-projects/sirc-instrument-project ]
  2. Hall GE, George A, Rutherford W. Measuring stages of concern about the innovation: a manual for use of the SoC questionnaire. Austin: Research and Development Center for Teacher Education, The University of Texas; 1979.
  3. Rogers EM. Diffusion of innovations. 3rd ed. New York: Free Press; 1983.
  4. Taylor J, Bowers D. Survey of organizations: a machine scored standardized questionnaire. Ann Arbor: Institute for Social Research, University of Michigan; 1972.
  5. Moos RH. The social climate scales: an overview. Palo Alto: Consulting Psychologists Press; 1974.
  6. Orthner DK, Cook PC, Sabah Y, Rosenfeld J. Measuring organizational learning in human services. Development and validation of the organizational learning capacity assessment. Miami: Workforce Issues in Social Work; 2005.
  7. Martino S, Ball SA, Gallon SL, Hall D, Garcia M, Ceperich S, Farentinos C, Hamilton J, Hausotter W. Motivational interviewing assessment: supervisory tools for enhancing proficiency. Salem: Northwest Frontier Addiction Technology Transfer Center, Oregon Health and Science University; 2006.
  8. Saksvik P, Nytro K, Tvedt SD. Healthy organizational change. In: Houdmont J, Leka S, editors. Occupational health psychology: European perspectives on research, education and practice, vol. 2. Nottingham: ISMAI Publishers; 2007. p. 81–90.
  9. Zaltman G. Metaphorically speaking: new technique uses multidisciplinary ideas to improve qualitative research. Mark Res. 1996;8:13.
  10. Newlove BW, Hall GE. A manual for assessing open-ended statements of concern. Austin: Research and Development Center for Teacher Education, The University of Texas; 1976.
  11. Venkatesh V, Davis FD. Extrinsic and intrinsic motivation to use computers in the work place. J Appl Psychol. 2000;22:1111–32.
  12. Anderson N, West MA. Team climate inventory: manual and user’s guide. Windsor: ASE; 1994.
  13. Hawe P, King L, Noort M, Jordens C, Lloyd B. Indicators to help with capacity building in health promotion. Sydney: NSW Health Department; 2000.
  14. Hackman JR, Oldham GR. Work redesign. Reading: Addison-Wesley; 1980.

Cited by

Sorted by citations

See all 116

Related

See all

Metadata

Authors
Clinton-McHarg, T., Yoong, S. L., Tzelepis, F., Regan, T., Fielding, A., Skelton, E., Kingsland, M., Ooi, J. Y., & Wolfenden, L.
Affiliations
  • Hunter Medical Research Institute (AU)
  • University of Newcastle Australia (AU)
  • Hunter New England Local Health District (AU)
Corresponding author
Wolfenden, Luke
ORCID
Venue
Implementation Science
Publisher
Springer Science and Business Media LLC
Published
2016-11-08
Volume
11
Issue
1
Pages
148-148
ISSN
1748-5908 (ISSN-L)
DOI
10.1186/s13012-016-0512-5
PMC
PMC5100177
OpenAlex
W2553855586
Cited by
116
References
234
Citations by year
2026: 6 · 2025: 9 · 2024: 11 · 2023: 16 · 2022: 12 · 2021: 15 · 2020: 14 · 2019: 15 · 2018: 11 · 2017: 6 · 2016: 1
FWCI
2.10
Type
journal article
Language
en
Open access
Licence
Locations
  • Implementation Science (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
  • Figshare (submittedVersion, cc-by)
  • Swinburne figshare (Swinburne University of Technology) (submittedVersion)
  • NOVA (University of Newcastle Australia) (submittedVersion, cc-by)
Funders
Hunter Medical Research Institute
Topics
  • Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
  • Health Sciences › Nursing › Research and Theory › Health Education and Validation
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
Keywords
psychometric properties, Consolidated Framework for Implementation Research, public health settings, community settings, measurement validity, reliability statistics, construct validity, cross-cultural adaptation, implementation climate, implementation barriers and facilitators
MeSH terms
Community Health Services, Diffusion of Innovation, Feasibility Studies, Humans, Psychometrics, Public Health, Surveys and Questionnaires, Reproducibility of Results, Translational Research, Biomedical, Clinical Coding

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review