Reviews of attitude research in implementation science require comprehensiveness, accuracy, and specificity
Open accessThe DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Abstract
© The Author(s) 2022. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http:// creat iveco mmons. org/ licen ses/ by/4. 0/. The Creative Commons Public Domain Dedication waiver (http:// creat iveco mmons. org/ publi cdoma in/ zero/1. 0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data. Dear Editors-in-Chief, Implementation Science: I write to voice concerns regarding the article by Fishman, Yang, and Mandell (2021, Vol 16, No. 87) recently published in Implementation Science. The concerns include attributions, interpretation of the meaning of measures, and an overly narrow consideration of the extant literature on attitudes in implementation research. The first concern is attribution regarding the 15-item Evidence-Based Practice Attitude Scale (EBPAS) [1]. The authors state that “The EBPAS developers acknowledge that the EBPAS assesses other constructs, such as knowledge” (pg. 6), but they proceed to cite a paper not by the EBPAS developer [2] for support. The cited paper used the expanded EBPAS-50 [3] that was subsequently made more pragmatic in the EBPAS-36, [4] both of which assess eight additional dimensions of attitudes toward evidence-based practices (EBPs). The EBPAS assesses attitudes, not knowledge. The authors incorrectly attributed their assertion to the EBPAS developer (i.e., this letter’s author). If the attribution were true, appropriate evidence should be clearly documented. Second, the authors offer an example of an EBPAS item that they assert assesses “knowledge,” but this is incorrect. They give the example of attitudes toward implementation of EBP given requirements to do so by supervisors, an organization, or a state (e.g., policy level) (i.e., EBPAS Requirements subscale). These items were intended to, and do, assess attitudes conditional on the source of directives to use EBP in work with patients and clients. This is a critical issue in implementation as directives to adopt and use new practices often come from those who manage, supervise, or set policy for direct services. This is part of the complex outer and inner contexts of implementation represented in frameworks that address such issues [5, 6]. The EBPAS Requirements subscale squarely focuses on attitudes conditional on the nuances of clinicians’ work environments in practice. Fishman et al.’s claim suggests a misunderstanding or misinterpretation of the EBPAS and a potential misconstrual of differences between assessment of attitudes and knowledge. Third, the authors state that “EBPAS items refer only to vaguely defined behavioral goals, such as adoption of new practices” (pg. 6). Again, the authors misinterpret the EBPAS, as this widely used instrument was designed to assess attitudes relevant to EBP implementation [1, 7]. The original 15-item EBPAS focuses on EBPs for three subscales, and more general attitudes of openness to new practices in one subscale (Openness subscale). The EBPAS is easily adapted for attitudes toward specific innovations in health care such as attitudes toward cognitive behavioral therapy, measurement-based care, or medication-assisted treatment, and behavioral targets such as EBP use or fidelity. Fishman et al.’s conclusions suggest a cursory evaluation of the EBPAS and ignore the breadth of empirical work conducted on attitudes using the EBPAS and its application since its publication in 2004. For example, an informal search on Google Scholar Open Access
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Our paper complies with recommendations that review articles in implementation science be rigorous, comprehensive of the questions being asked, and provide accurate attributions [66].”
From Ten years of implementation outcomes research: a scoping review (2023) · cites this paper as [66]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The theory of planned behavior: Frequently asked questions
Ajzen, I. (2020) · Human Behavior and Emerging Technologies · doi:10.1002/hbe2.195 · Cited by 3,347 · cited 1×
“For example, Azjen [11] notes that “feedback effects” (where the experience of a behavior can influence subsequent attitudes) is a frequently raised issue in research using the Theory of Planned Behavior.”
BackgroundAdvancing a Conceptual Model of Evidence-Based Practice Implementation in Public Service Sectors
Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-010-0327-7 · Cited by 3,240 · cited 2×
“Attitudes operate across multiple phases in complex implementation processes and contexts [5].”
BackgroundSelf-citationAarons, G. A. (2004) · Mental Health Services Research · doi:10.1023/b:mhsr.0000024351.12294.65 · Cited by 1,272 · cited 2×
“The first concern is attribution regarding the 15-item Evidence-Based Practice Attitude Scale (EBPAS) [1].”
BackgroundSelf-citationSystematic review of the Exploration, Preparation, Implementation, Sustainment (EPIS) framework
Moullin, J. C., Dickson, K. S., Stadnick, N. A., Rabin, B., & Aarons, G. A. (2019) · Implementation Science · doi:10.1186/s13012-018-0842-6 · Cited by 1,223 · cited 1×
“This is part of the complex outer and inner contexts of implementation represented in frameworks that address such issues [5, 6].”
BackgroundSelf-citationA systematic review of empirical studies examining mechanisms of implementation in health
Lewis, C. C., Boyd, M. R., Walsh-Bailey, C., Lyon, A. R., Beidas, R., Mittman, B., Aarons, G. A., Weiner, B. J., & Chambers, D. A. (2020) · Implementation Science · doi:10.1186/s13012-020-00983-3 · Cited by 363 · cited 1×
“The decision to exclude articles not covered in the Lewis et al review [8] (studies from 1990 to 2018) resulted in a narrow and unrepresentative sample of attitude studies and missed relevant studies including more recent work.”
ResultSelf-citationAarons, G. A., Research Network on Youth Mental Health, Glisson, C., Hoagwood, K., Kelleher, K., Landsverk, J., & Cafri, G. (2010) · Psychological Assessment · doi:10.1037/a0019188 · Cited by 316
Self-citationFarahnak, L. R., Ehrhart, M. G., Torres, E. M., & Aarons, G. A. (2020) · Journal of Leadership & Organizational Studies · doi:10.1177/1548051818824529 · Cited by 261 · cited 1×
“, 2020) of attitudes as part of a multilevel causal model examining leadership and attitudes in relation to implementation success [9].”
BackgroundSelf-citationCausality in the Theory of Planned Behavior
Sussman, R., & Gifford, R. (2019) · Personality and Social Psychology Bulletin · doi:10.1177/0146167218801363 · Cited by 237 · cited 1×
“Empirical evidence for “reversecausal relations” in the Theory of Planned Behavior suggests that reciprocal causal relations should be considered in attitude focused implementation research [12].”
BackgroundAarons, G. A., Cafri, G., Lugo, L., & Sawitzky, A. (2012) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-010-0302-3 · Cited by 225 · cited 1×
“The cited paper used the expanded EBPAS-50 [3] that was subsequently made more pragmatic in the EBPAS-36, [4] both of which assess eight additional dimensions of attitudes toward evidence-based practices (EBPs).”
MethodSelf-citationFischer, R. D., & Karl, J. A. (2021) · Social Psychological and Personality Science · doi:10.1177/19485506211019844 · Cited by 165 · cited 1×
“However, recent research demonstrates the relevance of the roles of policy, attitudes, and intentions during the COVID-19 health crisis [13].”
Background
Research works printed but not linked: 1
- Pearl J. Structural and probalistic causality. Psychol Learn. 1996;34:393–435.
Cited by
Ten years of implementation outcomes research: a scoping review
Proctor, E. K., Bunger, A. C., Lengnick-Hall, R., Gerke, D. R., Martin, J. K., Phillips, R. J., & Swanson, J. C. (2023) · Implementation Science · doi:10.1186/s13012-023-01286-z · Cited by 424
“Our paper complies with recommendations that review articles in implementation science be rigorous, comprehensive of the questions being asked, and provide accurate attributions [66].”
Additional file 1 of Ten years of implementation outcomes research: a scoping review
Proctor, E. K., Bunger, A. C., Lengnick‐Hall, R., Gerke, D. R., Martin, J. K., Phillips, R. J., & Swanson, J. C. (2024) · Figshare · doi:10.6084/m9.figshare.26606383
Metadata
- Authors
- Aarons, G. A.
- Affiliations
- UC San Diego Health System (US)
- Corresponding author
- Aarons, Gregory A.
- ORCID
- Aarons, Gregory A.
- Venue
- Implementation Science
- Published
- 2022-05-10
- Volume
- 17
- Issue
- 1
- Pages
- 28-28
- ISSN
- 1748-5908 (ISSN-L)
- PMC
- PMC9084932
- OpenAlex
- W4229451145
- Semantic Scholar
- d0626fc5572857059061cf27b1fbd1ba0c71e9c2
- Cited by
- 2
- References
- 15
- Citations by year
- 2024: 1 · 2023: 1
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Implementation Science (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Institute of Mental Health (K01MH001695)
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Applied Psychology › Behavioral Health and Interventions
- Keywords
- attitude research
- MeSH terms
- Implementation Science, Data Collection, Humans
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Reviews of attitude research in implementation science require comprehensiveness, accuracy, and specificity. https://app.cochat.ai/papers/10.1186/s13012-022-01198-4
- The paper
- Aarons, G. A. (2022). Reviews of attitude research in implementation science require comprehensiveness, accuracy, and specificity. Implementation Science. https://doi.org/10.1186/s13012-022-01198-4
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
