Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 31 of the 32 references it cites were found in a registry or an index.

Advancing implementation science through measure development and evaluation: a study protocol

Open access

Abstract

BACKGROUND: Significant gaps related to measurement issues are among the most critical barriers to advancing implementation science. Three issues motivated the study aims: (a) the lack of stakeholder involvement in defining pragmatic measure qualities; (b) the dearth of measures, particularly for implementation outcomes; and (c) unknown psychometric and pragmatic strength of existing measures. Aim 1: Establish a stakeholder-driven operationalization of pragmatic measures and develop reliable, valid rating criteria for assessing the construct. Aim 2: Develop reliable, valid, and pragmatic measures of three critical implementation outcomes, acceptability, appropriateness, and feasibility. Aim 3: Identify Consolidated Framework for Implementation Research and Implementation Outcome Framework-linked measures that demonstrate both psychometric and pragmatic strength. METHODS/DESIGN: For Aim 1, we will conduct (a) interviews with stakeholder panelists (N = 7) and complete a literature review to populate pragmatic measure construct criteria, (b) Q-sort activities (N = 20) to clarify the internal structure of the definition, (c) Delphi activities (N = 20) to achieve consensus on the dimension priorities, (d) test-retest and inter-rater reliability assessments of the emergent rating system, and (e) known-groups validity testing of the top three prioritized pragmatic criteria. For Aim 2, our systematic development process involves domain delineation, item generation, substantive validity assessment, structural validity assessment, reliability assessment, and predictive validity assessment. We will also assess discriminant validity, known-groups validity, structural invariance, sensitivity to change, and other pragmatic features. For Aim 3, we will refine our established evidence-based assessment (EBA) criteria, extract the relevant data from the literature, rate each measure using the EBA criteria, and summarize the data. DISCUSSION: The study outputs of each aim are expected to have a positive impact as they will establish and guide a comprehensive measurement-focused research agenda for implementation science and provide empirically supported measures, tools, and methods for accomplishing this work.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/s13012-015-0287-0

Cites

31 of 31 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1 Psychometric-pragmatic grid. Note. Predictions of psychometric and pragmatic strength for measures across domains
Fig. 1 Psychometric-pragmatic grid. Note. Predictions of psychometric and pragmatic strength for measures across domainsCC BY · PubMed Central
Fig. 2 Nomological network. Note. Included in our nomological network are antecedent constructs that have high relevance for only one focal construct
Fig. 2 Nomological network. Note. Included in our nomological network are antecedent constructs that have high relevance for only one focal constructCC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

1 more claim about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 32 references

Research works printed but not linked: 4

  1. Grimshaw J, Eccles M, Thomas R, MacLennan G, Ramsay C, Fraser C, et al. Toward evidence-based quality improvement. J Gen Intern Med. 2006;21:S14–20.
  2. Nunnally JC. Bernstein: psychometric theory. New York: McGraw-Hill; 1994.
  3. Donnelly J, Trochim W. The research methods knowledge base. Ohio: Atomic Dog Publishing; 2007.
  4. Muthén LK, Muthén BO. Mplus. Compr Model Program Appl Res Users Guide. 2012;5:E22.

Cited by

See all 115

Metadata

Authors
Lewis, C. C., Weiner, B. J., Stanick, C., & Fischer, S. M.
Affiliations
  • University of Washington (US)
  • University of North Carolina at Chapel Hill (US)
  • University of Montana (US)
  • Indiana University Bloomington (US)
Corresponding author
Lewis, Cara C.
Venue
Implementation Science
Published
2015-07-22
Volume
10
Issue
1
Pages
102-102
ISSN
1748-5908 (ISSN-L)
OpenAlex
W1522008408
Cited by
115
References
32
Citations by year
2026: 2 · 2025: 7 · 2024: 3 · 2023: 21 · 2022: 9 · 2021: 18 · 2020: 16 · 2019: 13 · 2018: 8 · 2017: 8 · 2016: 7 · 2015: 2
FWCI
15.20
Type
journal article
Language
en
Open access
Locations
  • Implementation Science (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Carolina Digital Repository (University of North Carolina at Chapel Hill) (submittedVersion)
  • UNC Libraries
Funders
  • Georgia Clinical and Translational Science Alliance
  • National Institutes of Health (2010-2015, UL1TR000083)
  • National Institute of Mental Health (R01MH106510)
  • National Center for Advancing Translational Sciences
Topics
  • Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
  • Social Sciences › Sociology and Political Science › Delphi Technique in Research
  • Health Sciences › Nursing › Research and Theory › Health Education and Validation
Keywords
implementation outcomes, measurement development, stakeholder engagement, acceptability, appropriateness, feasibility, psychometric properties, grading criteria, Consolidated Framework for Implementation Research, structural invariance, sensitivity to change, known-groups validity
MeSH terms
Humans, Research Design, Program Evaluation, Program Development

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Advancing implementation science through measure development and evaluation: a study protocol. https://app.cochat.ai/papers/10.1186/s13012-015-0287-0
The paper
Lewis, C. C., Weiner, B. J., Stanick, C., & Fischer, S. M. (2015). Advancing implementation science through measure development and evaluation: a study protocol. Implementation Science. https://doi.org/10.1186/s13012-015-0287-0

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.