The DOI matches its Crossref record; 85 of the 85 references it cites were found in a registry or an index.
Measuring determinants of implementation behavior: psychometric properties of a questionnaire based on the theoretical domains framework
Open accessAbstract
BACKGROUND: To be able to design effective strategies to improve healthcare professionals' implementation behaviors, a valid and reliable questionnaire is needed to assess potential implementation determinants. The present study describes the development of the Determinants of Implementation Behavior Questionnaire (DIBQ) and investigates the reliability and validity of this Theoretical Domains Framework (TDF)-based questionnaire. METHODS: The DIBQ was developed to measure the potential behavioral determinants of the 12-domain version of the TDF (Michie et al., 2005). We identified existing questionnaires including items assessing constructs within TDF domains and developed new items where needed. Confirmatory factor analysis was used to examine whether the predefined structure of the TDF-based questionnaire was supported by the data. Cronbach's alpha was calculated to assess internal consistency reliability of the questionnaire, and domains' discriminant validity was investigated. RESULTS: We developed an initial questionnaire containing 100 items assessing 12 domains. Results obtained from confirmatory factor analysis and Cronbach's alpha resulted in the final questionnaire consisting of 93 items assessing 18 domains, explaining 63.3% of the variance, and internal consistency reliability values ranging from .68 to .93. Domains demonstrated good discriminant validity, although the domains 'Knowledge' and 'Skills' and the domains 'Skills' and 'Social/professional role and identity' were highly correlated. CONCLUSIONS: We have developed a valid and reliable questionnaire that can be used to assess potential determinants of healthcare professional implementation behavior following the theoretical domains of the TDF. The DIBQ can be used by researchers and practitioners who are interested in identifying determinants of implementation behaviors in order to be able to develop effective strategies to improve healthcare professionals' implementation behaviors. Furthermore, the findings provide a novel validation of the TDF and indicate that the domain 'Environmental context and resources' might be divided into several environment-related domains.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2025) · doi:10.32614/r.manuals · Cited by 352,181
The theory of planned behavior
Ajzen, I. (1991) · Organizational Behavior and Human Decision Processes · doi:10.1016/0749-5978(91)90020-t · Cited by 90,587
Coefficient Alpha and the Internal Structure of Tests
Cronbach, L. J. (1951) · Psychometrika · doi:10.1007/bf02310555 · Cited by 43,931 · cited 1×
“Cronbach’s alpha [58] was computed to assess the internal consistency reliability of the items assessing each domain (i.”
MethodStructural equation modeling in practice: A review and recommended two-step approach
Anderson, J. C., & Gerbing, D. W. (1988) · Psychological Bulletin · doi:10.1037/0033-2909.103.3.411 · Cited by 41,941
Rogers, E. M., Singhal, A., & Quinlan, M. M. (2019) · An Integrated Approach to Communication Theory and Research · doi:10.4324/9780203710753-35 · Cited by 37,322
Development and validation of brief measures of positive and negative affect: The PANAS scales.
Watson, D., Clark, L. A., & Tellegen, A. (1988) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.54.6.1063 · Cited by 32,433 · cited 1×
“In addition, dividing the domain ‘Emotion’ into the domains ‘Positive emotions’ and ‘Negative emotions’ could be explained by previous research that indicated that positive and negative affect are two relatively independent constructs that can be measured discriminately [69,70].”
BackgroundIntrinsic Motivation and Self-Determination in Human Behavior
Deci, E. L., & Ryan, R. M. (1985) · doi:10.1007/978-1-4899-2271-7 · Cited by 26,772
Convergent and discriminant validation by the multitrait-multimethod matrix.
Campbell, D. T., & Fiske, D. W. (1959) · Psychological Bulletin · doi:10.1037/h0046016 · Cited by 17,116
Intrinsic Motivation and Self-Determination in Human Behavior
Miller, K. A., Deci, E. L., & Ryan, R. M. (1988) · Contemporary Sociology · doi:10.2307/2070638 · Cited by 15,312
The measurement of experienced burnout
Maslach, C., & Jackson, S. E. (1981) · Journal of Organizational Behavior · doi:10.1002/job.4030020205 · Cited by 14,496
Applied Multivariate Statistics for the Social Sciences
Stevens, J. P. (2012) · Routledge eBooks · doi:10.4324/9780203843130 · Cited by 10,648
Analysis of a complex of statistical variables into principal components.
Hotelling, H. (1933) · Journal of Educational Psychology · doi:10.1037/h0071325 · Cited by 9,633
The proof and measurement of association between two things
Spearman, C. (1904) · The American Journal of Psychology · doi:10.2307/1412159 · Cited by 8,039
Scheier, M. F., Carver, C. S., & Bridges, M. W. (1994) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.67.6.1063 · Cited by 7,456
Diffusion of Innovations in Service Organizations: Systematic Review and Recommendations
GREENHALGH, T., ROBERT, G., MACFARLANE, F., BATE, P., & KYRIAKIDOU, O. (2004) · The Milbank Quarterly · doi:10.1111/j.0887-378x.2004.00325.x · Cited by 7,448 · cited 1×
“This adjustment is consistent with leading theoretical models on the introduction of innovations in healthcare [6,8-12].”
ResultThe diffusion of innovations: Some examples of finnish civil servant's professional tours in Europe
Hietala, M. (1983) · Scandinavian Journal of History · doi:10.1080/03468758308579015 · Cited by 6,162
Applied multivariate statistics for the social sciences
Stevens, J. P., & Stevens, J. P. (2001) · doi:10.4324/9781410604491 · Cited by 4,993
Cane, J., O’Connor, D., & Michie, S. (2012) · Implementation Science · doi:10.1186/1748-5908-7-37 · Cited by 4,888 · cited 6×
“Furthermore, the adjustment corresponds with the results of the recent validation of the TDF [29].”
BackgroundMethodResultInfluentialFrom best evidence to best practice: effective implementation of change in patients' care
Grol, R., & Grimshaw, J. (2003) · The Lancet · doi:10.1016/s0140-6736(03)14546-1 · Cited by 4,531
Making psychological theory useful for implementing evidence based practice: a consensus approach
Michie, S. (2005) · Quality and Safety in Health Care · doi:10.1136/qshc.2004.011155 · Cited by 3,440
Cialdini, R. B., Kallgren, C. A., & Reno, R. R. (1991) · Advances in Experimental Social Psychology · doi:10.1016/s0065-2601(08)60330-5 · Cited by 3,143 · cited 1×
“Items measuring the constructs within the domains ‘Knowledge’, ‘Beliefs about capabilities’, ‘Social influences’, ‘Emotion’, ‘Behavioral regulation’, and ‘Nature of the behaviors’ [37,41,42,44-49] were adapted from previously published questionnaires.”
Health promotion from the perspective of Social Cognitive Theory
Bandura, A. (1998) · Psychology & Health · doi:10.1080/08870449808407422 · Cited by 3,087
Effectiveness and efficiency of guideline dissemination and implementation strategies
Grimshaw, J., Thomas, R., MacLennan, G., Fraser, C., Ramsay, C., Vale, L., Whitty, P., Eccles, M., Matowe, L., Shirran, L., et al. (2004) · Health Technology Assessment · doi:10.3310/hta8060 · Cited by 3,044
Crawford, J. R., & Henry, J. D. (2004) · British Journal of Clinical Psychology · doi:10.1348/0144665031752934 · Cited by 2,957 · cited 1×
“In addition, dividing the domain ‘Emotion’ into the domains ‘Positive emotions’ and ‘Negative emotions’ could be explained by previous research that indicated that positive and negative affect are two relatively independent constructs that can be measured discriminately [69,70].”
BackgroundA conceptual framework for implementation fidelity
Carroll, C., Patterson, M., Wood, S., Booth, A., Rick, J., & Balain, S. (2007) · Implementation Science · doi:10.1186/1748-5908-2-40 · Cited by 2,559
A General Approach to Confirmatory Maximum Likelihood Factor Analysis
Jöreskog, K. G. (1969) · Psychometrika · doi:10.1007/bf02289343 · Cited by 2,489
Knowledge translation of research findings
Grimshaw, J. M., Eccles, M. P., Lavis, J. N., Hill, S. J., & Squires, J. E. (2012) · Implementation Science · doi:10.1186/1748-5908-7-50 · Cited by 2,407
Glasgow, R. E., Lichtenstein, E., & Marcus, A. C. (2003) · American Journal of Public Health · doi:10.2105/ajph.93.8.1261 · Cited by 1,836
Michie, S., Johnston, M., Francis, J., Hardeman, W., & Eccles, M. (2008) · Applied Psychology · doi:10.1111/j.1464-0597.2008.00341.x · Cited by 1,789 · cited 3×
“Identifying the key factors associated with HCP implementation behavior can inform the development of strategies to promote evidence-based behavior [6,14-19].”
Background(2001) · PubMed · Cited by 1,709
The proof and measurement of association between two things
Spearman, C. (2010) · International Journal of Epidemiology · doi:10.1093/ije/dyq191 · Cited by 1,681
Applied Multivariate Statistics for the Social Sciences
Schumm, W. R., & Stevens, J. (1993) · The American Statistician · doi:10.2307/2685203 · Cited by 1,523
French, S. D., Green, S. E., O’Connor, D. A., McKenzie, J. E., Francis, J. J., Michie, S., Buchbinder, R., Schattner, P., Spike, N., & Grimshaw, J. M. (2012) · Implementation Science · doi:10.1186/1748-5908-7-38 · Cited by 1,338
Flottorp, S. A., Oxman, A. D., Krause, J., Musila, N. R., Wensing, M., Godycki-Cwirko, M., Baker, R., & Eccles, M. P. (2013) · Implementation Science · doi:10.1186/1748-5908-8-35 · Cited by 1,138
The proof and measurement of association between two things. By C. Spearman, 1904.
Spearman, C. (1987) · PubMed · Cited by 1,023
Action planning and coping planning for long-term lifestyle change: theory and assessment
Sniehotta, F. F., Schwarzer, R., Scholz, U., & Schüz, B. (2005) · European Journal of Social Psychology · doi:10.1002/ejsp.258 · Cited by 981
Planning and Studying Improvement in Patient Care: The Use of Theoretical Perspectives
GROL, R. P. T. M., BOSCH, M. C., HULSCHER, M. E. J. L., ECCLES, M. P., & WENSING, M. (2007) · The Milbank Quarterly · doi:10.1111/j.1468-0009.2007.00478.x · Cited by 943 · cited 3×
“To enhance the effective development of implementation strategies, therefore, many advocate using a theoretical approach to guide the investigation of implementation determinants [14,17,23-25].”
BackgroundEccles, M., Grimshaw, J., Walker, A., Johnston, M., & Pitts, N. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2004.09.002 · Cited by 917 · cited 3×
“Identifying the key factors associated with HCP implementation behavior can inform the development of strategies to promote evidence-based behavior [6,14-19].”
Background(2007) · Reference Reviews · doi:10.1108/09504120710737987 · Cited by 893
Effectiveness and efficiency of guideline dissemination and implementation strategies
Grimshaw, J. M., Thomas, R. E., MacLennan, G., Fraser, C., Ramsay, C. R., Vale, L., Whitty, P., Eccles, M. P., Matowe, L., Shirran, L., et al. (2005) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462305290190 · Cited by 864
Gardner, B., Abraham, C., Lally, P., & de Bruijn, G.-J. (2012) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/1479-5868-9-102 · Cited by 855
Chaudoir, S. R., Dugan, A. G., & Barr, C. H. (2013) · Implementation Science · doi:10.1186/1748-5908-8-22 · Cited by 800
Changing Provider Behavior: An Overview of Systematic Reviews of Interventions
Grimshaw, J. M., Shirran, L., Thomas, R., Mowatt, G., Fraser, C., Bero, L., Grilli, R., Harvey, E., Oxman, A., & O’Brien, M. A. (2001) · Medical Care · doi:10.1097/00005650-200108002-00002 · Cited by 778
Determinants of innovation within health care organizations: Literature review and Delphi study
Fleuren, M. (2004) · International Journal for Quality in Health Care · doi:10.1093/intqhc/mzh030 · Cited by 732
(2007) · Choice Reviews Online · doi:10.5860/choice.44-3029 · Cited by 691
Glasgow, R. E., Klesges, L. M., Dzewaltowski, D. A., Bull, S. S., & Estabrooks, P. (2004) · Annals of Behavioral Medicine · doi:10.1207/s15324796abm2701_2 · Cited by 635
Davies, P., Walker, A. E., & Grimshaw, J. M. (2010) · Implementation Science · doi:10.1186/1748-5908-5-14 · Cited by 619
Wood, W., & Neal, D. T. (2009) · Journal of Consumer Psychology · doi:10.1016/j.jcps.2009.08.003 · Cited by 601
Improving Patient Care: The Implementation of Change in Clinical Practice
Cosby, J. L. (2006) · Quality and Safety in Health Care · doi:10.1136/qshc.2005.016824 · Cited by 557
Improving Patient Care: The Implementation of Change in Clinical Practice
Grol, R. P. T. M., Wensing, M., Eccles, M., Davis, D., & Wensign, M. (2005) · Cited by 553
Bridging the implementation gap between knowledge and action for health.
(2004) · Europe PMC (PubMed Central) · doi:10.1590/s0042-96862004001000005 · Cited by 527
An implementation research agenda
Eccles, M. P., Armstrong, D., Baker, R., Cleary, K., Davies, H., Davies, S., Glasziou, P., Ilott, I., Kinmonth, A.-L., Leng, G., et al. (2009) · Implementation Science · doi:10.1186/1748-5908-4-18 · Cited by 481
Huijg, J. M., Gebhardt, W. A., Crone, M. R., Dusseldorp, E., & Presseau, J. (2014) · Implementation Science · doi:10.1186/1748-5908-9-11 · Cited by 385
Self-citationDesigning theoretically-informed implementation interventions
The Improved Clinical Effectiveness through Behavioural Research Group (ICEBeRG) (2006) · Implementation Science · doi:10.1186/1748-5908-1-4 · Cited by 380
Frese, M. (1999) · Journal of Occupational Health Psychology · doi:10.1037//1076-8998.4.3.179 · Cited by 327
Rothman, A. J. (2004) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/1479-5868-1-11 · Cited by 322
the Canada PRIME Plus Team, Patey, A. M., Islam, R., Francis, J. J., Bryson, G. L., & Grimshaw, J. M. (2012) · Implementation Science · doi:10.1186/1748-5908-7-52 · Cited by 237 · cited 1×
“, [30-35]) and for the identification of suitable theories to further investigate these behaviors [27,36].”
BackgroundFrancis, J. J., Stockton, C., Eccles, M. P., Johnston, M., Cuthbertson, B. H., Grimshaw, J. M., Hyde, C., Tinmouth, A., & Stanworth, S. J. (2009) · British Journal of Health Psychology · doi:10.1348/135910708x397025 · Cited by 230
Mazza, D., Chapman, A., & Michie, S. (2013) · BMC Health Services Research · doi:10.1186/1472-6963-13-36 · Cited by 208
Frankmann, J. P., & Adams, J. A. (1962) · Psychological Bulletin · doi:10.1037/h0046142 · Cited by 201
WÄNNSTRÖM, I., PETERSON, U., ÅSBERG, M., NYGREN, Å., & GUSTAVSSON, J. P. (2009) · Scandinavian Journal of Psychology · doi:10.1111/j.1467-9450.2008.00697.x · Cited by 190
Islam, R., Tinmouth, A. T., Francis, J. J., Brehaut, J. C., Born, J., Stockton, C., Stanworth, S. J., Eccles, M. P., Cuthbertson, B. H., Hyde, C., et al. (2012) · Implementation Science · doi:10.1186/1748-5908-7-93 · Cited by 181
Shirley, D., van der Ploeg, H. P., & Bauman, A. E. (2010) · Physical Therapy · doi:10.2522/ptj.20090383 · Cited by 169
Dzewaltowski, D. A., Estabrooks, P. A., & Glasgow, R. E. (2004) · Exercise and Sport Sciences Reviews · doi:10.1097/00003677-200404000-00004 · Cited by 169
Michie, S., Pilling, S., Garety, P., Whitty, P., Eccles, M. P., Johnston, M., & Simmons, J. (2007) · Implementation Science · doi:10.1186/1748-5908-2-8 · Cited by 161
Testing a Simple Structure Hypothesis in Factor Analysis
Jöreskog, K. G. (1966) · Psychometrika · doi:10.1007/bf02289505 · Cited by 160
Dyson, J., Lawton, R., Jackson, C., & Cheater, F. (2011) · Journal of Infection Prevention · doi:10.1177/1757177410384300 · Cited by 142 · cited 1×
“, [30-35]) and for the identification of suitable theories to further investigate these behaviors [27,36].”
BackgroundBeenstock, J., Sniehotta, F. F., White, M., Bell, R., Milne, E. M., & Araujo-Soares, V. (2012) · Implementation Science · doi:10.1186/1748-5908-7-36 · Cited by 138 · cited 5×
“[38] and Taylor et al.”
BackgroundMethodResultInfluentialWhat supports physiotherapists’ use of research in clinical practice? A qualitative study in Sweden
Dannapfel, P., Peolsson, A. L. C., & Nilsén, P. (2013) · Implementation Science · doi:10.1186/1748-5908-8-31 · Cited by 138
McSherry, L. A., Dombrowski, S. U., Francis, J. J., Murphy, J., Martin, C. M., O’Leary, J. J., Sharp, L., & ATHENS Group (2012) · Implementation Science · doi:10.1186/1748-5908-7-73 · Cited by 129
Curran, J. A., Brehaut, J., Patey, A. M., Osmond, M., Stiell, I., & Grimshaw, J. M. (2013) · Implementation Science · doi:10.1186/1748-5908-8-25 · Cited by 107
Why does primary care need more implementation research?
Foy, R., Eccles, M., & Grimshaw, J. (2001) · Family Practice · doi:10.1093/fampra/18.4.353 · Cited by 96
Development and initial validation of the determinants of physical activity questionnaire
Taylor, N., Lawton, R., & Conner, M. (2013) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/1479-5868-10-74 · Cited by 96
Amemori, M., Michie, S., Korhonen, T., Murtomaa, H., & Kinnunen, T. H. (2011) · Implementation Science · doi:10.1186/1748-5908-6-50 · Cited by 82
Theory-based predictors of multiple clinician behaviors in the management of diabetes
Presseau, J., Johnston, M., Francis, J., Hrisos, S., Stamp, E., Steen, N., Hawthorne, G., Grimshaw, J., Elovainio, M. J., Hunter, M., et al. (2013) · Journal of Behavioral Medicine · doi:10.1007/s10865-013-9513-x · Cited by 82
Barrett, E., Darker, C. D., & Hussey, J. M. (2012) · Journal of Public Health · doi:10.1007/s10389-012-0512-0 · Cited by 63
Huijg, J. M., Crone, M. R., Verheijden, M. W., van der Zouwe, N., Middelkoop, B. J., & Gebhardt, W. A. (2013) · BMC Family Practice · doi:10.1186/1471-2296-14-142 · Cited by 63
Self-citationA Simple Method of Factor Analysis
Holzinger, K. J. (1944) · Psychometrika · doi:10.1007/bf02288737 · Cited by 62
Francis, J. J., Tinmouth, A., Stanworth, S. J., Grimshaw, J. M., Johnston, M., Hyde, C., Stockton, C., Brehaut, J. C., Fergusson, D., & Eccles, M. P. (2009) · Implementation Science · doi:10.1186/1748-5908-4-70 · Cited by 60
Development and initial validation of the Influences on Patient Safety Behaviours Questionnaire
Taylor, N., Parveen, S., Robins, V., Slater, B., & Lawton, R. (2013) · Implementation Science · doi:10.1186/1748-5908-8-81 · Cited by 58
Stuive, I., Kiers, H. A. L., Timmerman, M. E., & ten Berge, J. M. F. (2008) · Educational and Psychological Measurement · doi:10.1177/0013164408315264 · Cited by 50
Huijg, J. M., van der Zouwe, N., Crone, M. R., Verheijden, M. W., Middelkoop, B. J., & Gebhardt, W. A. (2014) · International Journal of Behavioral Medicine · doi:10.1007/s12529-014-9411-9 · Cited by 47
Self-citationForsman, H., Rudman, A., Gustavsson, P. J., Ehrenberg, A., & Wallin, L. (2012) · Implementation Science · doi:10.1186/1748-5908-7-46 · Cited by 47
van der Wees, P. J., Zagers, C., de Die, S. E., Hendriks, E., der Sanden, M. W. G. N., & de Bie, R. A. (2013) · BMC Health Services Research · doi:10.1186/1472-6963-13-159 · Cited by 34
Stuive, I., Kiers, H. A. L., & Timmerman, M. E. (2009) · Educational and Psychological Measurement · doi:10.1177/0013164409332226 · Cited by 32
Cited by
Finch, T. L., Girling, M., May, C. R., Mair, F. S., Murray, E., Treweek, S., McColl, E., Steen, I. N., Cook, C., Vernazza, C. R., et al. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0591-x · Cited by 337
Palinkas, L. A., Chou, C.-P., Spear, S. E., Mendon, S. J., Villamar, J., & Brown, C. H. (2020) · Implementation Science · doi:10.1186/s13012-020-01030-x · Cited by 70
Becoming an eCoach: Training therapists in online cognitive-behavioral therapy for chronic pain
Terpstra, J. A., van der Vaart, R., Spillekom-van Koulil, S., van Dam, A., Rosmalen, J. G. M., Knoop, H., van Middendorp, H., & Evers, A. W. M. (2018) · Patient Education and Counseling · doi:10.1016/j.pec.2018.03.029 · Cited by 17
Metadata
- Authors
- Huijg, J. M., Gebhardt, W. A., Dusseldorp, E., Verheijden, M. W., van der Zouwe, N., Middelkoop, B. J., & Crone, M. R.
- Venue
- Implementation Science
- Published
- 2014-03-19
- Volume
- 9
- Issue
- 1
- Pages
- 33-33
- ISSN
- 1748-5908 (ISSN-L)
- OpenAlex
- W2159177038
- Cited by
- 352
- References
- 85
- FWCI
- 15.68
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.