Measuring readiness for implementation: A systematic review of measures’ psychometric and pragmatic properties
Open accessThe DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
Abstract
Background: Systematic measure reviews can facilitate advances in implementation research and practice by locating reliable, valid, pragmatic measures; identifying promising measures needing refinement and testing; and highlighting measurement gaps. This review identifies and evaluates the psychometric and pragmatic properties of measures of readiness for implementation and its sub-constructs as delineated in the Consolidated Framework for Implementation Research: leadership engagement, available resources, and access to knowledge and information. Methods: The systematic review methodology is described fully elsewhere. The review, which focused on measures used in mental or behavioral health, proceeded in three phases. Phase I, data collection, involved search string generation, title and abstract screening, full text review, construct assignment, and cited citation searches. Phase II, data extraction, involved coding relevant psychometric and pragmatic information. Phase III, data analysis, involved two trained specialists independently rating each measure using Psychometric and Pragmatic Evidence Rating Scales (PAPERS). Frequencies and central tendencies summarized information availability and PAPERS ratings. Results: Searches identified 9 measures of readiness for implementation, 24 measures of leadership engagement, 17 measures of available resources, and 6 measures of access to knowledge and information. Information about internal consistency was available for most measures. Information about other psychometric properties was often not available. Ratings for internal consistency were “adequate” or “good.” Ratings for other psychometric properties were less than “adequate.” Information on pragmatic properties was most often available regarding cost, language readability, and brevity. Information was less often available regarding training burden and interpretation burden. Cost and language readability generally exhibited “good” or “excellent” ratings, interpretation burden generally exhibiting “minimal” ratings, and training burden and brevity exhibiting mixed ratings across measures. Conclusion: Measures of readiness for implementation and its sub-constructs used in mental health and behavioral health care are unevenly distributed, exhibit unknown or low psychometric quality, and demonstrate mixed pragmatic properties. This review identified a few promising measures, but targeted efforts are needed to systematically develop and test measures that are useful for both research and practice. Plain language abstract: Successful implementation of effective mental health or behavioral health treatments in service delivery settings depends in part on the readiness of the service providers and administrators to implement the treatment; the engagement of organizational leaders in the implementation effort; the resources available to support implementation, such as time, money, space, and training; and the accessibility of knowledge and information among service providers about the treatment and how it works. It is important that the methods for measuring these factors are dependable, accurate, and practical; otherwise, we cannot assess their presence or strength with confidence or know whether efforts to increase their presence or strength have worked. This systematic review of published studies sought to identify and evaluate the quality of questionnaires (referred to as measures) that assess readiness for implementation, leadership engagement, available resources, and access to knowledge and information. We identified 56 measures of these factors and rated their quality in terms of how dependable, accurate, and practical they are. Our findings indicate there is much work to be done to improve the quality of available measures; we offer several recommendations for doing so.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009) · Implementation Science · doi:10.1186/1748-5908-4-50 · Cited by 14,496 · cited 7×
“Readiness for implementation, defined as “tangible and immediate indicators of organizational commitment to its decision to implement an intervention,” is considered a critical precursor to successful implementation (Damschroder et al., 2009; Gleeson, 2017; Kotter, 1996; Weiner, 2009).”
BackgroundInfluentialProctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., Griffey, R., & Hensley, M. (2011) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-010-0319-7 · Cited by 8,721
A theory of organizational readiness for change
Weiner, B. J. (2009) · Implementation Science · doi:10.1186/1748-5908-4-67 · Cited by 2,268
Self-citationTerwee, C. B., Mokkink, L. B., Knol, D. L., Ostelo, R. W. J. G., Bouter, L. M., & de Vet, H. C. W. (2012) · Quality of Life Research · doi:10.1007/s11136-011-9960-1 · Cited by 1,978
Weiner, B. J., Amick, H., & Lee, S.-Y. D. (2008) · Medical Care Research and Review · doi:10.1177/1077558708317802 · Cited by 786
Self-citationAarons, G. A., Ehrhart, M. G., & Farahnak, L. R. (2014) · Implementation Science · doi:10.1186/1748-5908-9-45 · Cited by 559 · cited 5×
“” Note the median rating of “2— adequate” for discriminant validity was based on the rating of just one measure: the Implementation Leadership Scale (Aarons et al., 2014).”
BackgroundMethodInfluentialA conceptual framework for transferring research to practice
Simpson, D. D. (2002) · Journal of Substance Abuse Treatment · doi:10.1016/s0740-5472(02)00231-3 · Cited by 538 · cited 6×
“, 2014) for leadership engagement, the Texas Christian University Program Training Needs Survey (Simpson, 2002) for available resources, and the Structured Interview of Evidence Use (Palinkas et al.”
MethodInfluentialEhrhart, M. G., Aarons, G. A., & Farahnak, L. R. (2014) · Implementation Science · doi:10.1186/s13012-014-0157-1 · Cited by 502 · cited 2×
“Assessing the organizational context for EBP implementation: The development and validity testing of the Implementation Climate Scale (ICS).”
MethodLewis, C. C., Fischer, S., Weiner, B. J., Stanick, C., Kim, M., & Martinez, R. G. (2015) · Implementation Science · doi:10.1186/s13012-015-0342-x · Cited by 430 · cited 2×
“negative assessment of a PAPERS property than the “worst counts” approach wherein a PAPERS property is assigned the lowest rating for that property when there are multiple ratings (Lewis et al., 2015; Terwee et al., 2012).”
BackgroundSelf-citationGlasgow, R. E., & Riley, W. T. (2013) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2013.03.010 · Cited by 318 · cited 1×
“Finally, neither review appraised the pragmatic quality of measures, that is, the extent to which they are relevant to practice or policy stakeholders and feasible to use in real-world settings (Glasgow, 2013; Glasgow & Riley, 2013; Stanick et al., 2019).”
Background
Cited by
Damschroder, L. J., Reardon, C. M., Opra Widerquist, M. A., & Lowery, J. (2022) · Implementation Science · doi:10.1186/s13012-021-01181-5 · Cited by 784
Hall, A., Shoesmith, A., Doherty, E., McEvoy, B., Mettert, K., Lewis, C. C., Wolfenden, L., Yoong, S., Kingsland, M., Shelton, R. C., et al. (2022) · Implementation Science · doi:10.1186/s13012-022-01252-1 · Cited by 81
“The resulting standardised measures include survey items tailored to specific characteristics, such as settings, populations, and/or EBIs, which have been calibrated to create standardised scores that are comparable across the tailored items [23].”
BackgroundMethodResultSelf-citationPowell, B. J., Mettert, K. D., Dorsey, C. N., Weiner, B. J., Stanick, C. F., Lengnick-Hall, R., Ehrhart, M. G., Aarons, G. A., Barwick, M. A., Damschroder, L. J., et al. (2021) · Implementation Research and Practice · doi:10.1177/26334895211018862 · Cited by 79
Self-citationLewis, C. C., Mettert, K., & Lyon, A. R. (2021) · Implementation Research and Practice · doi:10.1177/2633489521994197 · Cited by 38
Self-citationReflections on the measurement of implementation constructs
Wensing, M. (2021) · Implementation Research and Practice · doi:10.1177/26334895211020125 · Cited by 14
Metadata
- Authors
- Weiner, B. J., Mettert, K. D., Dorsey, C. N., Nolen, E. A., Stanick, C., Powell, B. J., & Lewis, C. C.
- Venue
- Implementation Research and Practice
- Publisher
- SAGE Publications
- Published
- 2020-01
- Volume
- 1
- Pages
- 2633489520933896-2633489520933896
- ISSN
- 2633-4895 (ISSN-L)
- OpenAlex
- W3080795028
- Cited by
- 58
- References
- 17
- FWCI
- 2.07
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Measuring readiness for implementation: A systematic review of measures’ psychometric and pragmatic properties. https://app.cochat.ai/papers/10.1177/2633489520933896
- The paper
- Weiner, B. J., Mettert, K. D., Dorsey, C. N., Nolen, E. A., Stanick, C., Powell, B. J., & Lewis, C. C. (2020). Measuring readiness for implementation: A systematic review of measures’ psychometric and pragmatic properties. Implementation Research and Practice. https://doi.org/10.1177/2633489520933896
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.