The DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Operationalizing the ‘pragmatic’ measures construct using a stakeholder feedback and a multi-method approach
Open accessAbstract
CONTEXT: Implementation science measures are rarely used by stakeholders to inform and enhance clinical program change. Little is known about what makes implementation measures pragmatic (i.e., practical) for use in community settings; thus, the present study's objective was to generate a clinical stakeholder-driven operationalization of a pragmatic measures construct. EVIDENCE ACQUISITION: The pragmatic measures construct was defined using: 1) a systematic literature review to identify dimensions of the construct using PsycINFO and PubMed databases, and 2) interviews with an international stakeholder panel (N = 7) who were asked about their perspectives of pragmatic measures. EVIDENCE SYNTHESIS: Combined results from the systematic literature review and stakeholder interviews revealed a final list of 47 short statements (e.g., feasible, low cost, brief) describing pragmatic measures, which will allow for the development of a rigorous, stakeholder-driven conceptualization of the pragmatic measures construct. CONCLUSIONS: Results revealed significant overlap between terms related to the pragmatic construct in the existing literature and stakeholder interviews. However, a number of terms were unique to each methodology. This underscores the importance of understanding stakeholder perspectives of criteria measuring the pragmatic construct. These results will be used to inform future phases of the project where stakeholders will determine the relative importance and clarity of each dimension of the pragmatic construct, as well as their priorities for the pragmatic dimensions. Taken together, these results will be incorporated into a pragmatic rating system for existing implementation science measures to support implementation science and practice.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Lewis et al.’s measurement reviews on the original CFIR constructs [10] focused specifically on assessing questionnaires administered to healthcare professionals or leaders within behavioral health settings [97] using multi-dimensional criteria for validity and pragmatism [98].”
From The updated consolidated framework for implementation research based on user feedback (2022) · cites this paper as [98]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Qualitative research and evaluation methods
Patton, M. Q. (1980) · Cited by 35,733
Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009) · Implementation Science · doi:10.1186/1748-5908-4-50 · Cited by 14,496 · cited 2×
“Lewis CC, Stanick CF, Martinez RG, et al. The Society for Implementation Research Collaboration Instrument Review Project: a methodology to promote rigorous evaluation.”
MethodPalinkas, L. A., Horwitz, S. M., Green, C. A., Wisdom, J. P., Duan, N., & Hoagwood, K. (2015) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-013-0528-y · Cited by 12,038
Self-citationThe PHQ-9: A New Depression Diagnostic and Severity Measure
Kroenke, K., & Spitzer, R. L. (2002) · Psychiatric Annals · doi:10.3928/0048-5713-20020901-06 · Cited by 5,898 · cited 2×
“Importantly, work by Glasgow and colleagues [1, 20] and Kroenke and colleagues [2] were explicit in their attempts to describe pragmatic measure criteria and serve as the basis for the initial steps of our work.”
BackgroundGoulding, C. (2002) · SAGE Publications Ltd eBooks · doi:10.4135/9781849209236 · Cited by 4,455
Mokkink, L. B., Terwee, C. B., Patrick, D. L., Alonso, J., Stratford, P. W., Knol, D. L., Bouter, L. M., & de Vet, H. C. W. (2010) · Quality of Life Research · doi:10.1007/s11136-010-9606-8 · Cited by 4,260 · cited 1×
“Comparable rating criteria have over 1000 citations, despite their relatively recent development, suggesting the potential for significant impact [10, 11].”
BackgroundProctor, E. K., Landsverk, J., Aarons, G., Chambers, D., Glisson, C., & Mittman, B. (2009) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-008-0197-4 · Cited by 1,894
Qualitative Content Analysis: A Guide to Paths not Taken
Morgan, D. L. (1993) · Qualitative Health Research · doi:10.1177/104973239300300107 · Cited by 1,264
Glisson, C., Landsverk, J., Schoenwald, S., Kelleher, K., Hoagwood, K. E., Mayberg, S., Green, P., & The Research Network on Youth Mental Health (2008) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-007-0148-5 · Cited by 608
Hunsley, J., & Mash, E. J. (2007) · Annual Review of Clinical Psychology · doi:10.1146/annurev.clinpsy.3.022806.091419 · Cited by 528
Cited by
The updated consolidated framework for implementation research based on user feedback
Damschroder, L. J., Reardon, C. M., Widerquist, M. A. O., & Lowery, J. (2022) · Implementation Science · doi:10.1186/s13012-022-01245-0 · Cited by 4,452
“’s measurement reviews on the original CFIR constructs [10] focused specifically on assessing questionnaires administered to healthcare professionals or leaders within behavioral health settings [97] using multi-dimensional criteria for validity and pragmatism [98].”
BackgroundPsychometric and Pragmatic Properties of Social Risk Screening Tools: A Systematic Review
Henrikson, N. B., Blasi, P. R., Dorsey, C. N., Mettert, K. D., Nguyen, M. B., Walsh-Bailey, C., Macuiba, J., Gottlieb, L. M., & Lewis, C. C. (2019) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2019.07.012 · Cited by 165
Self-citationPowell, B. J., Stanick, C. F., Halko, H. M., Dorsey, C. N., Weiner, B. J., Barwick, M. A., Damschroder, L. J., Wensing, M., Wolfenden, L., & Lewis, C. C. (2017) · Implementation Science · doi:10.1186/s13012-017-0649-x · Cited by 144
Self-citationAn updated protocol for a systematic review of implementation-related measures
Lewis, C. C., Mettert, K. D., Dorsey, C. N., Martinez, R. G., Weiner, B. J., Nolen, E., Stanick, C., Halko, H., & Powell, B. J. (2018) · Systematic Reviews · doi:10.1186/s13643-018-0728-3 · Cited by 118
Self-citationStanick, C. F., Halko, H. M., Nolen, E. A., Powell, B. J., Dorsey, C. N., Mettert, K. D., Weiner, B. J., Barwick, M., Wolfenden, L., Damschroder, L. J., et al. (2021) · Translational Behavioral Medicine · doi:10.1093/tbm/ibz164 · Cited by 117
“, ensuring that all constructs that are sufficiently differentiated from similar constructs were identified), we systematically defined the properties of pragmatic measures using both deductive and inductive methods [5].”
MethodSelf-citation
Metadata
- Authors
- Stanick, C. F., Halko, H. M., Dorsey, C. N., Weiner, B. J., Powell, B. J., Palinkas, L. A., & Lewis, C. C.
- Affiliations
- University of Montana (US)
- Kaiser Permanente Washington Health Research Institute (US)
- University of Washington (US)
- University of North Carolina at Chapel Hill (US)
- University of Southern California (US)
- Corresponding author
- Lewis, Cara C.
- ORCID
- Venue
- BMC Health Services Research
- Published
- 2018-11-22
- Volume
- 18
- Issue
- 1
- Pages
- 882-882
- ISSN
- 1472-6963 (ISSN-L)
- PMC
- PMC6251134
- OpenAlex
- W2901927177
- Cited by
- 54
- References
- 25
- Citations by year
- 2026: 3 · 2025: 2 · 2024: 5 · 2023: 5 · 2022: 6 · 2021: 23 · 2020: 4 · 2019: 4 · 2018: 1
- FWCI
- 12.22
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0/ (tdm)
- creativecommons.org/licenses/by/4.0/ (vor)
- Locations
- BMC Health Services Research (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Carolina Digital Repository (University of North Carolina at Chapel Hill) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- UNC Libraries
- Funders
- National Institute of Mental Health (R01MH106510)
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Health Sciences › Health Professions › General Health Professions › Health Sciences Research and Education
- Keywords
- operationalization, stakeholder perspectives, community settings, feasibility, cost, brevity
- MeSH terms
- Implementation Science, Communication, Feedback, Female, Humans, Male, Middle Aged, Research Design
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Operationalizing the ‘pragmatic’ measures construct using a stakeholder feedback and a multi-method approach. https://app.cochat.ai/papers/10.1186/s12913-018-3709-2
- The paper
- Stanick, C. F., Halko, H. M., Dorsey, C. N., Weiner, B. J., Powell, B. J., Palinkas, L. A., & Lewis, C. C. (2018). Operationalizing the ‘pragmatic’ measures construct using a stakeholder feedback and a multi-method approach. BMC Health Services Research. https://doi.org/10.1186/s12913-018-3709-2
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.