The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index; 7 of the 26 sources its claims cite were checked for support.
Psychometric assessment of three newly developed implementation outcome measures
Open accessAbstract
BACKGROUND: Implementation outcome measures are essential for monitoring and evaluating the success of implementation efforts. Yet, currently available measures lack conceptual clarity and have largely unknown reliability and validity. This study developed and psychometrically assessed three new measures: the Acceptability of Intervention Measure (AIM), Intervention Appropriateness Measure (IAM), and Feasibility of Intervention Measure (FIM). METHODS: Thirty-six implementation scientists and 27 mental health professionals assigned 31 items to the constructs and rated their confidence in their assignments. The Wilcoxon one-sample signed rank test was used to assess substantive and discriminant content validity. Exploratory and confirmatory factor analysis (EFA and CFA) and Cronbach alphas were used to assess the validity of the conceptual model. Three hundred twenty-six mental health counselors read one of six randomly assigned vignettes depicting a therapist contemplating adopting an evidence-based practice (EBP). Participants used 15 items to rate the therapist's perceptions of the acceptability, appropriateness, and feasibility of adopting the EBP. CFA and Cronbach alphas were used to refine the scales, assess structural validity, and assess reliability. Analysis of variance (ANOVA) was used to assess known-groups validity. Finally, half of the counselors were randomly assigned to receive the same vignette and the other half the opposite vignette; and all were asked to re-rate acceptability, appropriateness, and feasibility. Pearson correlation coefficients were used to assess test-retest reliability and linear regression to assess sensitivity to change. RESULTS: All but five items exhibited substantive and discriminant content validity. A trimmed CFA with five items per construct exhibited acceptable model fit (CFI = 0.98, RMSEA = 0.08) and high factor loadings (0.79 to 0.94). The alphas for 5-item scales were between 0.87 and 0.89. Scale refinement based on measure-specific CFAs and Cronbach alphas using vignette data produced 4-item scales (α's from 0.85 to 0.91). A three-factor CFA exhibited acceptable fit (CFI = 0.96, RMSEA = 0.08) and high factor loadings (0.75 to 0.89), indicating structural validity. ANOVA showed significant main effects, indicating known-groups validity. Test-retest reliability coefficients ranged from 0.73 to 0.88. Regression analysis indicated each measure was sensitive to change in both directions. CONCLUSIONS: The AIM, IAM, and FIM demonstrate promising psychometric properties. Predictive validity assessment is planned.
Overview
CoChat Summary
## TL;DR
This paper developed and psychometrically tested three new implementation outcome measures: the Acceptability of Intervention Measure (AIM), Intervention Appropriateness Measure (IAM), and Feasibility of Intervention Measure (FIM). Across three studies, the measures showed promising content validity, factor structure, internal consistency, test-retest reliability, and sensitivity to change, though discriminant validity between acceptability and appropriateness remained uncertain.
## Key findings
- In study 1, judges showed high agreement on item assignments: ICC = 0.89 across all 31 items, with construct-specific ICCs of 0.82 for acceptability, 0.94 for appropriateness, and 0.87 for feasibility.
- For content validity, all but five items showed substantive and discriminant content validity; the paper specifically flags items such as “This EBP is good enough,” “This EBP will do,” “This EBP is pretty good,” “This EBP seems right,” and “This EBP seems right on the button” as lacking sufficient discriminant content validity.
- A 3-factor CFA supported the conceptual model: in study 1, fit was CFI = 0.98 and RMSEA = 0.08 with factor loadings from 0.79 to 0.94; in study 2, fit was CFI = 0.96 and RMSEA = 0.08 with factor loadings from 0.75 to 0.89.
- The final short scales were 4-item measures with good reliability: Cronbach alphas were 0.85, 0.91, and 0.89 in study 2 for acceptability, appropriateness, and feasibility, respectively; test-retest coefficients in study 3 were 0.80, 0.73, and 0.88.
- Known-groups validity was supported by ANOVA: the manipulated construct explained 42% of variance for acceptability, 37% for appropriateness, and 42% for feasibility; regression models explained 41%, 42%, and 46% of the variance in change scores, respectively.
## Methods
- The paper reports three studies: study 1 assessed substantive and discriminant content validity with implementation scientists and mental health professionals; study 2 tested structural validity, reliability, and known-groups validity in a vignette experiment; study 3 re-administered the vignette experiment weeks later to assess test-retest reliability and sensitivity to change.
- Study 1 included 36 implementation scientists and 27 mental health professionals who assigned 31 items to constructs and rated confidence; study 2 included 326 mental health counselors who rated 15 items across six randomly assigned vignettes; study 3 followed up with 192 respondents from study 2, with 95 used for test-retest and 97 for sensitivity analyses.
- Analyses included Wilcoxon one-sample signed rank tests, ICCs, EFA, CFA, Cronbach alphas, ANOVA, Pearson correlations corrected for attenuation, and linear regression.
## Limitations
- The paper states that correlations among acceptability, appropriateness, and feasibility were sometimes fairly high, and discriminant validity was not fully tested because of a survey design error.
- The study also states that replication is needed in samples with different backgrounds and characteristics, and with different methods or materials.
- Evident from the text: the vignette design was incomplete because two of the eight possible vignettes were not fielded, which limited the exploration of discriminant validity and interaction effects, especially between acceptability and appropriateness.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
![Fig. 1 Confirmatory factor analysis of substantive validity data N = 63. χ2(87) = 120.6, p =.01; CFI = 0.98 ; RMSEA = 0.08, CI [0.04-0.11]. All path loadings are significant at p < .05](https://figures.cochat.ai/paper-figures/3ef324243be846fdad2a0dd7bcb4982e/psychometric-assessment-of-three-newly-developed-implementation-outcome-measures-figure-1.jpg)
![Fig. 2 Confirmatory factor analysis of structural validity data N = 326. χ2(49) = 147.9, p = <.001; CFI = 0.96; RMSEA = 0.079, CI [0.06-0.09]. All path loadings are significant at p < .05](https://figures.cochat.ai/paper-figures/1d1bfaf351b94a6face5cf7d95fd6184/psychometric-assessment-of-three-newly-developed-implementation-outcome-measures-figure-2.jpg)
Verification
Claims the paper makes
5 Valid · 0 Not valid · 13 Not checked · 88 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“Despite their importance, existing measures of implementation outcomes have largely unknown psychometric and pragmatic qualities [2].”
§ Psychometric assessment of three newly developed implementation outcome measures > Background
What the sources say
- Supportedcites [2] Lewis CC, Fischer S, Weiner BJ, Stanick C, Kim M, Martinez RG. Outcomes for implementation science: an enhanced systematic review of instruments using evidence-based rating criteria. Implement Sci. 2015;10:155. doi: 10.1186/s13012-015-0342-x . source
The Results section 'Evidence-based assessment rating' states directly that 'availability of information on the psychometric and pragmatic properties across all 104 instruments was limited and variable,' with 46% missing data on four or more of six EBA criteria and reliability, structural validity, predictive validity and responsiveness unavailable for 51%, 74%, 82% and 96% of instruments respectively. The Background frames the same point as 'the quality of existing implementation outcome instruments remains unclear and perhaps one of the most critical gaps in the literature,' and the Discussion notes usability — the only pragmatic criterion, rated for all instruments via item count — is 'a crude metric for characterizing the pragmatic properties,' so the paper does not treat that universal coverage as making pragmatic qualities known. The citing sentence's 'despite their importance' also matches the Background's claim that implementation outcomes are 'perhaps the most critical factor in implementation science.' (LLM, conf=0.88)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“few if any have been assessed for other measurement properties that are of key importance to implementation researchers and other implementation stakeholders (e.g., intermediaries, policymakers, practice leaders), such as stability, responsiveness, and predictive validity [2–4].”
§ Psychometric assessment of three newly developed implementation outcome measures > Background
What the sources say
- Partly supportedcites [2] Lewis CC, Fischer S, Weiner BJ, Stanick C, Kim M, Martinez RG. Outcomes for implementation science: an enhanced systematic review of instruments using evidence-based rating criteria. Implement Sci. 2015;10:155. doi: 10.1186/s13012-015-0342-x . sourceExperimental — accuracy not yet measured
The Abstract Results and Discussion report that most of the 104 implementation outcome instruments lacked information on responsiveness or predictive validity, supporting that part of the claim. The paper does not report an assessment of stability; its reliability ratings concern internal consistency, so it does not establish the claim about stability. (LLM, conf=0.84)
- Partly supportedcites [3] Martinez RG, Lewis CC, Weiner BJ. Instrumentation issues in implementation science. Implement Sci. 2014;9:118. doi: 10.1186/s13012-014-0118-8 . sourceExperimental — accuracy not yet measured
The Discussion’s recommendations for validity reporting say there is a dearth of literature on instruments’ predictive validity. Its reliability recommendations discuss test–retest reliability, and its Language section says measures should be sensitive to change, but neither establishes that few, if any, implementation outcome measures have been assessed for stability or responsiveness. No page numbers are available in our copy. (LLM, conf=0.84)
- Partly supportedcites [4] Lewis CC, Weiner BJ, Stanick C, Fischer SM. Advancing implementation science through measure development and evaluation: a study protocol. Implement Sci. 2015;10:102. doi: 10.1186/s13012-015-0287-0 . sourceExperimental — accuracy not yet measured
The Introduction reports that, among 104 implementation outcome measures, 82% lacked information on predictive validity and 96% on responsiveness. It does not establish how many had been assessed for stability; the Methods/design section instead proposes test-retest reliability testing for new measures. The claim is therefore supported for two of the three named properties. (LLM, conf=0.84)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“These concerns extend to the measurement of implementation outcomes, as, despite their centrality to understanding the extent to which implementation is successful, valid and reliable measures are lacking [2].”
§ Psychometric assessment of three newly developed implementation outcome measures > Discussion > Contributions to implementation science and practice This study contributes to the literature by developing valid and reliable measures of important implementation outcomes [1]. The field of implementation science has been deemed a Tower of Babel given the lack of conceptual clarity and consistency in its terminology [26], and concerns about the state of measurement in the field have been well documented [3, 27, 28]. These concerns extend to the measurement of implementation outcomes, as, despite their centrality to understanding the extent to which implementation is successful, valid and reliable measures are lacking [2]. This study fills that gap by developing valid and reliable measures of three implementation outcomes that are salient to a wide range of implementation studies as well as a wide range of pilot, efficacy, and effectiveness studies. Indeed, with an increasing focus on designing for dissemination and implementation [29], and some going as far as to say that all effectiveness studies be Hybrid I effectiveness-implementation studies [30, 31], measuring constructs such as acceptability, appropriateness, and feasibility will be a ubiquitous need. These outcomes are relevant to assessing stakeholders’ perceptions of clinical and public health interventions, as well as assessing perceptions of implementation strategies, which are often complex interventions in and of themselves [1, 32, 33]. Assessing these outcomes early in the research process may ensure that interventions and implementation strategies are optimized and fit with end-users’ preferences. In addition to the need for valid and reliable measures of implementation-related constructs, there is a need for pragmatic measures [11]. Implementation stakeholders are unlikely to use measures unless they possess these qualities, which may include broad domains such as being (1) useful in informing decision-making, (2) compatible with the settings in which they are employed, (3) easy to use, and (4) acceptable [34]. Pragmatic measures are particularly important for low-resource settings [35]. There are several examples of recent efforts to develop pragmatic measures for implementation constructs such as organizational readiness for change [36], implementation leadership [37], and implementation climate [38, 39]. We sought to ensure that our measures were pragmatic, and we believe that we have accomplished that in three ways. First, we sought to develop measures that were brief. We began by developing 12 or fewer items for each construct, and our psychometric testing resulted in final measures with only four items per construct. Second, we made each item as general as possible by not specifying a specific context or clinical problem within the items. For example, the appropriateness items did not specify a purpose (e.g., “for treating depression”), person (e.g., “for my patients”), or situation (e.g., “for my organization”); those wishing to use these measures could add such referents to explore specific aspects of appropriateness (i.e., social or technical fit). Third, we purposefully made the measures “open access” to ensure that the scales are freely available to all who might wish to use them. Our hope is that developing a measure that is free, brief, easy to use and not context- or treatment-specific will increase the chances of its use broadly in implementation research and practice. One of the unfortunate realities in implementation science is that, to date, the majority of current measures are developed for the purpose of a single study (usually with minimal conceptual clarity and psychometric testing) and then never used again. This state of affairs precludes our ability to develop generalizable knowledge in implementation science and, more specifically, knowledge about how current measures perform across a wide range of contexts. Of course, whether these measures ultimately demonstrate predictive validity within the field of mental health and other clinical settings is yet to be determined. Finally, we have laid out a systematic process for measure development and testing that we believe is both replicable and feasible. In doing so, we stress the importance of clearly defining constructs and engaging in domain delineation processes that ensure that constructs are sufficiently differentiated from similar constructs. We also stress careful psychometric testing, and lay out a process for establishing substantive validity, discriminate validity, structural validity, discriminant validity, known-groups validity, test-retest reliability, and sensitivity to change. The measurement development and testing process took 15 months, suggesting that the process could be completed within the period of a grant-funded implementation study. We encourage other teams to replicate this methodology and suggest further refinements that may enhance the efficiency and effectiveness of this process.
What the sources say
- Supportedcites [2] Lewis CC, Fischer S, Weiner BJ, Stanick C, Kim M, Martinez RG. Outcomes for implementation science: an enhanced systematic review of instruments using evidence-based rating criteria. Implement Sci. 2015;10:155. doi: 10.1186/s13012-015-0342-x . source
The paper's Conclusions state that 'the majority of implementation outcomes are underdeveloped, yielding few instruments or those without evidence of psychometric strength' and that 'Without high-quality instruments, it will be difficult to determine predictors, moderators, and mediators of implementation success' (Conclusions), matching both halves of the claim — centrality to understanding implementation success and the lack of valid/reliable measures. The Abstract Conclusions repeat that 'Implementation outcomes instrumentation is underdeveloped with respect to both the sheer number of available instruments and the psychometric quality of existing instruments', and the Background frames implementation outcomes as 'perhaps the most critical factor in implementation science' with instrument quality 'one of the most critical gaps'. The Results quantify the shortfall (only 47% good/excellent for reliability, 17% structural validity, 9% predictive validity; Discussion notes 46% of 104 instruments missing data on four or more psychometric properties). (LLM, conf=0.90)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The ICC for all participants for each construct was as follows: 0.82 (95% CI 0.63–0.94) for acceptability, 0.94 (95% CI 0.86–0.98) for appropriateness, and 0.87 (95% CI 0.72–0.96) for feasibility.”
§ Psychometric assessment of three newly developed implementation outcome measures > Study 1 > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The correlations among the three factors were high, ranging from 0.77 to 0.91.”
§ Psychometric assessment of three newly developed implementation outcome measures > Study 1 > Results
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
48 more claims
Create a free account to see all 106 claims and to verify the 19 references still open.
- It may state two separate claims as one. 35
- The quote only partly states this claim. 1
- Probably not a substantial claim. 24
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Among therapists, acceptability of MAST-V was assessed using four-item validated measures of intervention acceptability and appropriateness (α = .90, 92; Weiner et al., 2017).”
From Brief modular anxiety intervention for primary care: Hybrid I pilot randomized controlled trial of feasibility, acceptability, effectiveness, and implementation potential (2024) · cites this paper as Weiner et al., 2017
The cited source says what the claim attributes to it.
In the citing paper
“These measures have been reported to have good psychometric properties [30].”
The cited source says what the claim attributes to it.
In the citing paper
“Weiner and colleagues’ study of the psychometric properties for measures of three key implementation outcomes [12].”
From Ten years of implementation outcomes research: a scoping review (2023) · cites this paper as [12]
The cited source says what the claim attributes to it.
In the citing paper
“Survey questions are based on AIM, IAM, and FIM outcome measures to assess the acceptability, appropriateness, and feasibility of the ICU-PAUSE implementation along a 5-point Likert scale (Table 1) [21].”
From Improving Communication in Intensive Care Unit to Ward Transitions: Protocol for Multisite National Implementation of the ICU-PAUSE Handoff Tool (2023) · cites this paper as [21]
In the citing paper
“The 12-item quantitative survey consisted of three validated scales (Acceptability of Intervention Measure, Intervention Appropriateness Measure, Feasibility of Intervention Measure)(Weiner et al., 2017) measured on a 5-point Likert scale (1 = completely disagree, 5 = completely agree).”
From Mobile technology and task shifting to improve access to alcohol treatment services in Mozambique (2022) · cites this paper as Weiner et al., 2017
Citations
Cites
Hu, L., & Bentler, P. M. (1999) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705519909540118 · Cited by 108,788
Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009) · Implementation Science · doi:10.1186/1748-5908-4-50 · Cited by 14,496
Power analysis and determination of sample size for covariance structure modeling.
MacCallum, R. C., Browne, M. W., & Sugawara, H. M. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.2.130 · Cited by 10,418
Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., Griffey, R., & Hensley, M. (2011) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-010-0319-7 · Cited by 8,721
The proof and measurement of association between two things
Spearman, C. (1904) · The American Journal of Psychology · doi:10.2307/1412159 · Cited by 8,039
Reporting Structural Equation Modeling and Confirmatory Factor Analysis Results: A Review
Schreiber, J. B., Nora, A., Stage, F. K., Barlow, E. A., & King, J. (2006) · The Journal of Educational Research · doi:10.3200/joer.99.6.323-338 · Cited by 6,937
Powell, B. J., Waltz, T. J., Chinman, M. J., Damschroder, L. J., Smith, J. L., Matthieu, M. M., Proctor, E. K., & Kirchner, J. E. (2015) · Implementation Science · doi:10.1186/s13012-015-0209-1 · Cited by 4,963
Self-citationA sharper Bonferroni procedure for multiple tests of significance
HOCHBERG, Y. (1988) · Biometrika · doi:10.1093/biomet/75.4.800 · Cited by 4,829
Messick, S. (1995) · American Psychologist · doi:10.1037/0003-066x.50.9.741 · Cited by 4,198
A Brief Tutorial on the Development of Measures for Use in Survey Questionnaires
Hinkin, T. R. (1998) · Organizational Research Methods · doi:10.1177/109442819800100106 · Cited by 4,095
Research works printed but not linked: 5
- Muthén LK, Muthén BO. Mplus user’s guide. seventh edition ed. Muthén & Muthén: Los Angeles; 2012.
- Cohen J. Statistical power analysis for the behavioral sciences. Second edition ed. Lawrence Earlbaum Associates: Hillsdale; 1988.
- WMK T, JP D. The research methods knowledge base. 3rd ed. Cincinnati: Atomic Dog Publishing; 2006.
- Proctor EK, Powell BJ, Feely M. Measurement in dissemination and implementation science. In: Kendall RSBPC, editor. Dissemination and implementation of evidence-based practices in child and adolescent mental health. New York: Oxford University Press; 2014. p. 22–43.
- Powell BJ, Weiner BJ, Stanick CF, Halko H, Dorsey C, Lewis CC. Stakeholders’ perceptions of criteria for pragmatic measurement in implementation: a concept mapping approach (oral presentation). 9th Annual Conference on the Science of Dissemination & Implementation. Washington, D. C: Academy Health and the National Institutes of Health; 2016.
Cited by
Damschroder, L. J., Reardon, C. M., Opra Widerquist, M. A., & Lowery, J. (2022) · Implementation Science · doi:10.1186/s13012-021-01181-5 · Cited by 784
Guidance for conducting feasibility and pilot studies for implementation trials
Pearson, N., Naylor, P.-J., Ashe, M. C., Fernandez, M., Yoong, S. L., & Wolfenden, L. (2020) · Pilot and Feasibility Studies · doi:10.1186/s40814-020-00634-w · Cited by 475
Ten years of implementation outcomes research: a scoping review
Proctor, E. K., Bunger, A. C., Lengnick-Hall, R., Gerke, D. R., Martin, J. K., Phillips, R. J., & Swanson, J. C. (2023) · Implementation Science · doi:10.1186/s13012-023-01286-z · Cited by 424
“Weiner and colleagues’ study of the psychometric properties for measures of three key implementation outcomes [12].”
BackgroundCook, C. R., Lyon, A. R., Locke, J., Waltz, T., & Powell, B. J. (2019) · Prevention Science · doi:10.1007/s11121-019-01017-1 · Cited by 336
Self-citationStover, A. M., Haverman, L., van Oers, H. A., Greenhalgh, J., Potter, C. M., On behalf of the ISOQOL PROMs/PREMs in Clinical Practice Implementation Science Work Group, Ahmed, S., Greenhalgh, J., Gibbons, E., Haverman, L., et al. (2021) · Quality of Life Research · doi:10.1007/s11136-020-02564-9 · Cited by 308
Metadata
- Authors
- Weiner, B. J., Lewis, C. C., Stanick, C., Powell, B. J., Dorsey, C. N., Clary, A. S., Boynton, M. H., & Halko, H.
- Affiliations
- University of Washington (US)
- Kaiser Permanente Washington Health Research Institute (US)
- University of North Carolina at Chapel Hill (US)
- University of Montana (US)
- Corresponding author
- Weiner, Bryan J.
- ORCID
- Venue
- Implementation Science
- Published
- 2017-08-29
- Volume
- 12
- Issue
- 1
- Pages
- 108-108
- ISSN
- 1748-5908 (ISSN-L)
- PMC
- PMC5576104
- OpenAlex
- W2750615195
- Cited by
- 2,819
- References
- 35
- Citations by year
- 2026: 549 · 2025: 655 · 2024: 548 · 2023: 376 · 2022: 298 · 2021: 229 · 2020: 98 · 2019: 45 · 2018: 14 · 2017: 2
- FWCI
- 69.79
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0/ (tdm)
- creativecommons.org/licenses/by/4.0/ (vor)
- Locations
- Implementation Science (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Open Scholarship Institutional Repository (Washington University in St. Louis) (submittedVersion, other-oa)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Funders
- National Institute of Mental Health (R01MH106510)
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Health Sciences › Nursing › Research and Theory › Health Education and Validation
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Keywords
- acceptability, appropriateness, feasibility, evidence-based practice adoption, mental health counselors, scale development, content validity, structural validity, known-groups validity, test-retest reliability, sensitivity to change, vignettes, psychometric properties, reliability and validity, mental health services
- MeSH terms
- Factor Analysis, Statistical, Feasibility Studies, Female, Health Plan Implementation, Humans, Male, Psychometrics, Surveys and Questionnaires, Reproducibility of Results, Outcome Assessment, Health Care
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Psychometric assessment of three newly developed implementation outcome measures. https://app.cochat.ai/papers/10.1186/s13012-017-0635-3
- The paper
- Weiner, B. J., Lewis, C. C., Stanick, C., Powell, B. J., Dorsey, C. N., Clary, A. S., Boynton, M. H., & Halko, H. (2017). Psychometric assessment of three newly developed implementation outcome measures. Implementation Science. https://doi.org/10.1186/s13012-017-0635-3
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.