Organizational readiness to change assessment (ORCA): Development of an instrument based on the Promoting Action on Research in Health Services (PARIHS) framework
Open accessThe DOI matches its Crossref record; 46 of the 47 references it cites were found in a registry or an index.
Abstract
BACKGROUND: The Promoting Action on Research Implementation in Health Services, or PARIHS, framework is a theoretical framework widely promoted as a guide to implement evidence-based clinical practices. However, it has as yet no pool of validated measurement instruments that operationalize the constructs defined in the framework. The present article introduces an Organizational Readiness to Change Assessment instrument (ORCA), organized according to the core elements and sub-elements of the PARIHS framework, and reports on initial validation. METHODS: We conducted scale reliability and factor analyses on cross-sectional, secondary data from three quality improvement projects (n = 80) conducted in the Veterans Health Administration. In each project, identical 77-item ORCA instruments were administered to one or more staff from each facility involved in quality improvement projects. Items were organized into 19 subscales and three primary scales corresponding to the core elements of the PARIHS framework: (1) Strength and extent of evidence for the clinical practice changes represented by the QI program, assessed with four subscales, (2) Quality of the organizational context for the QI program, assessed with six subscales, and (3) Capacity for internal facilitation of the QI program, assessed with nine subscales. RESULTS: Cronbach's alpha for scale reliability were 0.74, 0.85 and 0.95 for the evidence, context and facilitation scales, respectively. The evidence scale and its three constituent subscales failed to meet the conventional threshold of 0.80 for reliability, and three individual items were eliminated from evidence subscales following reliability testing. In exploratory factor analysis, three factors were retained. Seven of the nine facilitation subscales loaded onto the first factor; five of the six context subscales loaded onto the second factor; and the three evidence subscales loaded on the third factor. Two subscales failed to load significantly on any factor. One measured resources in general (from the context scale), and one clinical champion role (from the facilitation scale). CONCLUSION: We find general support for the reliability and factor structure of the ORCA. However, there was poor reliability among measures of evidence, and factor analysis results for measures of general resources and clinical champion role did not conform to the PARIHS framework. Additional validation is needed, including criterion validation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The ORCA [27] assesses patient constructs in terms of the degree to which patient preferences are addressed in the available evidence supporting an innovation.”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Structural Equations with Latent Variables.
Clogg, C. C., & Bollen, K. A. (1991) · Contemporary Sociology · doi:10.2307/2072165 · Cited by 17,442
Structural Equations with Latent Variables
Bollen, K. A. (1989) · Wiley series in probability and statistics · doi:10.1002/9781118619179 · Cited by 17,277
Health Measurement Scales: A Practical Guide to Their Development and Use
Streiner, D. L., Norman, G. R., & Cairney, J. (1989) · Cited by 9,595
Social Research Methods: Qualitative and Quantitative Approaches
Djamba, Y. K., & Neuman, W. L. (2002) · Teaching Sociology · doi:10.2307/3211488 · Cited by 8,851
(1993) · Medical Entomology and Zoology · Cited by 7,690
Factor analysis in the development and refinement of clinical assessment instruments.
Floyd, F. J., & Widaman, K. F. (1995) · Psychological Assessment · doi:10.1037/1040-3590.7.3.286 · Cited by 3,970
Social Research Methods: Qualitative and Quantitative Approaches
Bernard, H. R. (2012) · Cited by 3,524
Enabling the implementation of evidence based practice: a conceptual framework.
Kitson, A., Harvey, G., & McCormack, B. (1998) · Quality in Health Care · doi:10.1136/qshc.7.3.149 · Cited by 1,985
On the nature and direction of relationships between constructs and measures.
Edwards, J. R., & Bagozzi, R. P. (2000) · Psychological Methods · doi:10.1037//1082-989x.5.2.155 · Cited by 1,745
On the Measurement of Organizational Slack
Bourgeois, L. J. (1981) · Academy of Management Review · doi:10.5465/amr.1981.4287985 · Cited by 1,545
Health Measurement Scales: A Practical Guide To Their Development And Use
Streiner, D. L., & Norman, G. R. (2003) · doi:10.1093/oso/9780198528470.001.0001 · Cited by 1,460
Kitson, A. L., Rycroft-Malone, J., Harvey, G., McCormack, B., Seers, K., & Titchen, A. (2008) · Implementation Science · doi:10.1186/1748-5908-3-1 · Cited by 1,314
Readiness for Organizational Change: The Systematic Development of a Scale
Holt, D. T., Armenakis, A. A., Feild, H. S., & Harris, S. G. (2007) · The Journal of Applied Behavioral Science · doi:10.1177/0021886306295295 · Cited by 1,243
What counts as evidence in evidence‐based practice?
Rycroft‐Malone, J., Seers, K., Titchen, A., Harvey, G., Kitson, A., & McCormack, B. (2004) · Journal of Advanced Nursing · doi:10.1111/j.1365-2648.2004.03068.x · Cited by 968
Planning and Studying Improvement in Patient Care: The Use of Theoretical Perspectives
GROL, R. P. T. M., BOSCH, M. C., HULSCHER, M. E. J. L., ECCLES, M. P., & WENSING, M. (2007) · The Milbank Quarterly · doi:10.1111/j.1468-0009.2007.00478.x · Cited by 943
The PARIHS Framework—A Framework for Guiding the Implementation of Evidence-based Practice
Rycroft-Malone, J. (2004) · Journal of Nursing Care Quality · doi:10.1097/00001786-200410000-00002 · Cited by 923
Eccles, M., Grimshaw, J., Walker, A., Johnston, M., & Pitts, N. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2004.09.002 · Cited by 917
On the Measurement of Organizational Slack
Bourgeois, L. J. (1981) · The Academy of Management Review · doi:10.2307/257138 · Cited by 896
Weiner, B. J., Amick, H., & Lee, S.-Y. D. (2008) · Medical Care Research and Review · doi:10.1177/1077558708317802 · Cited by 786
Getting evidence into practice: the meaning of `context'
McCormack, B., Kitson, A., Harvey, G., Rycroft‐Malone, J., Titchen, A., & Seers, K. (2002) · Journal of Advanced Nursing · doi:10.1046/j.1365-2648.2002.02150.x · Cited by 765
Getting evidence into practice: the role and function of facilitation
Harvey, G., Loftus‐Hills, A., Rycroft‐Malone, J., Titchen, A., Kitson, A., McCormack, B., & Seers, K. (2002) · Journal of Advanced Nursing · doi:10.1046/j.1365-2648.2002.02126.x · Cited by 730
Ingredients for change: revisiting a conceptual framework
Rycroft-Malone, J. (2002) · Quality and Safety in Health Care · doi:10.1136/qhc.11.2.174 · Cited by 701
Shortell, S. M., O'Brien, J. L., Carman, J. M., FOSTER, R. W. G., Hughes, E. F., Boerstler, H., & O’Connor, E. J. (1995) · Europe PMC (PubMed Central) · Cited by 678
An exploration of the factors that influence the implementation of evidence into practice
Rycroft‐Malone, J., Harvey, G., Seers, K., Kitson, A., McCormack, B., & Titchen, A. (2004) · Journal of Clinical Nursing · doi:10.1111/j.1365-2702.2004.01007.x · Cited by 623
Reconsidering formative measurement.
Howell, R. D., Breivik, E., & Wilcox, J. B. (2007) · Psychological Methods · doi:10.1037/1082-989x.12.2.205 · Cited by 432
Stetler, C. B., Legro, M. W., Rycroft-Malone, J., Bowman, C., Curran, G., Guihan, M., Hagedorn, H., Pineros, S., & Wallace, C. M. (2006) · Implementation Science · doi:10.1186/1748-5908-1-23 · Cited by 411
Designing theoretically-informed implementation interventions
The Improved Clinical Effectiveness through Behavioural Research Group (ICEBeRG) (2006) · Implementation Science · doi:10.1186/1748-5908-1-4 · Cited by 380
Shortell, S. M., Jones, R. H., Rademaker, A. W., Gillies, R. R., Dranove, D. S., Hughes, E. F. X., Budetti, P. P., Reynolds, K. S. E., & Huang, C.-F. (2000) · Medical Care · doi:10.1097/00005650-200002000-00010 · Cited by 285
Influence of Organizational Characteristics and Context on Research Utilization
Cummings, G. G., Estabrooks, C. A., Midodzi, W. K., Wallin, L., & Hayduk, L. (2007) · Nursing Research · doi:10.1097/01.nnr.0000280629.63654.95 · Cited by 277
(1995) · Journal of General Internal Medicine · doi:10.1007/bf02599863 · Cited by 214
How do patients' treatment preferences compare with those of clinicians?
Montgomery, A. A., & Fahey, T. (2001) · Quality and Safety in Health Care · doi:10.1136/qhc.0100039 · Cited by 166
Predicting Research Use in Nursing Organizations: A Multilevel Analysis
Estabrooks, C. A., Midodzi, W. K., Cummings, G. G., & Wallin, L. (2007) · Nursing Research · doi:10.1097/01.nnr.0000280647.18806.98 · Cited by 154
Development and Testing of the Context Assessment Index (CAI)
McCormack, B., McCarthy, G., Wright, J., & Coffey, A. (2009) · Worldviews on Evidence-Based Nursing · doi:10.1111/j.1741-6787.2008.00130.x · Cited by 126
Davis, R. E., Dolan, G., Thomas, S., Atwell, C., Mead, D., Nehammer, S., Moseley, L., Edwards, A., & Elwyn, G. (2003) · Health Expectations · doi:10.1046/j.1369-6513.2003.00235.x · Cited by 90
Do physicians take into account patients’ expectations?
Sanchez-Menegay, C., & Stalder, H. (1994) · Journal of General Internal Medicine · doi:10.1007/bf02629524 · Cited by 84
Managed Care Rebound? Recent Changes In Health Plans' Cost Containment Strategies
Mays, G. P., Claxton, G., & White, J. S. (2004) · Health Affairs · doi:10.1377/hlthaff.w4.427 · Cited by 68
Sharp, N. D., Pineros, S. L., Hsu, C., Starks, H., & Sales, A. E. (2004) · Worldviews on Evidence-Based Nursing · doi:10.1111/j.1741-6787.2004.04023.x · Cited by 50
Self-citationBrown, D., & McCormack, B. (2005) · Worldviews on Evidence-Based Nursing · doi:10.1111/j.1741-6787.2005.00024.x · Cited by 46
Bahtsevani, C., Willman, A., Khalaf, A., & Östman, M. (2008) · Journal of Evaluation in Clinical Practice · doi:10.1111/j.1365-2753.2007.00916.x · Cited by 41
Shevlin, M. E., Hunt, N., & Robbins, I. J. (2000) · Psychological Assessment · doi:10.1037/1040-3590.12.4.414 · Cited by 31
Young, G. J., Charns, M. P., & Heeren, T. (2004) · Academy of Management Journal · doi:10.2307/20159614 · Cited by 30
Every, N. R., Fihn, S. D., Sales, A. E., Keane, A., & Ritchie, J. R. (2000) · Medical Care · doi:10.1097/00005650-200006001-00006 · Cited by 29
Self-citationSeymour, J. (1993) · Cited by 27
Nr, E., Fihn, S. D., Sales, A., et al. (2000) · PubMed · Cited by 20
Self-citationSales, A. E., Helfrich, C. D., Ho, P. M., Hedeen, A. N., Plomondon, M. E., Li, Y.-F., Connors, A., & Rumsfeld, J. S. (2008) · Implementation Science · doi:10.1186/1748-5908-3-28 · Cited by 17
Self-citationPiñeros, S. L., Sales, A. E., Li, Y., & Sharp, N. D. (2004) · Worldviews on Evidence-Based Nursing · doi:10.1111/j.1524-475x.2004.04042.x · Cited by 15
Self-citation
Research works printed but not linked: 1
- Nunnally JC Bernstein IH Psychometric Theory 1994 3 New York, NY: McGraw-Hill Inc
Cited by
Bridging Research and Practice
Tabak, R. G., Khoong, E. C., Chambers, D. A., & Brownson, R. C. (2012) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2012.05.024 · Cited by 1,386
Harvey, G., & Kitson, A. (2015) · Implementation Science · doi:10.1186/s13012-016-0398-2 · Cited by 1,271
Pfadenhauer, L. M., Gerhardus, A., Mozygemba, K., Lysdahl, K. B., Booth, A., Hofmann, B., Wahlster, P., Polus, S., Burns, J., Brereton, L., et al. (2017) · Implementation Science · doi:10.1186/s13012-017-0552-5 · Cited by 1,016
Organizational readiness for implementing change: a psychometric assessment of a new measure
Shea, C. M., Jacobs, S. R., Esserman, D. A., Bruce, K., & Weiner, B. J. (2014) · Implementation Science · doi:10.1186/1748-5908-9-7 · Cited by 856
Chaudoir, S. R., Dugan, A. G., & Barr, C. H. (2013) · Implementation Science · doi:10.1186/1748-5908-8-22 · Cited by 800
Metadata
- Authors
- Helfrich, C. D., Li, Y.-F., Sharp, N. D., & Sales, A. E.
- Affiliations
- Health Services Research & Development (US)
- University of Washington (US)
- University of Alberta (CA)
- Corresponding author
- Helfrich, Christian D
- Venue
- Implementation Science
- Published
- 2009-07-14
- Volume
- 4
- Issue
- 1
- Pages
- 38-38
- ISSN
- 1748-5908 (ISSN-L)
- PMC
- PMC2716295
- OpenAlex
- W2011592435
- Cited by
- 564
- References
- 47
- Citations by year
- 2026: 18 · 2025: 40 · 2024: 47 · 2023: 53 · 2022: 60 · 2021: 78 · 2020: 52 · 2019: 25 · 2018: 37 · 2017: 23 · 2016: 18 · 2015: 30 · 2014: 22 · 2013: 20 · 2012: 16
- FWCI
- 23.60
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Implementation Science (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Funders
- U.S. Department of Veterans Affairs
- Health Services Research and Development
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Health Sciences › Health Professions › General Health Professions › Health Sciences Research and Education
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Healthcare Quality and Management
- Keywords
- PARIHS framework, organizational readiness to change, scale development, instrument validation, quality improvement, evidence-based practice implementation, organizational context, clinical champions, Veterans Health Administration, scale reliability, factor structure
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.