Improving the normalization of complex interventions: part 1 - development of the NoMAD instrument for assessing implementation work based on normalization process theory (NPT)
Open accessAbstract
BACKGROUND: Understanding and measuring implementation processes is a key challenge for implementation researchers. This study draws on Normalization Process Theory (NPT) to develop an instrument that can be applied to assess, monitor or measure factors likely to affect normalization from the perspective of implementation participants. METHODS: An iterative process of instrument development was undertaken using the following methods: theoretical elaboration, item generation and item reduction (team workshops); item appraisal (QAS-99); cognitive testing with complex intervention teams; theory re-validation with NPT experts; and pilot testing of instrument. RESULTS: We initially generated 112 potential questionnaire items; these were then reduced to 47 through team workshops and item appraisal. No concerns about item wording and construction were raised through the item appraisal process. We undertook three rounds of cognitive interviews with professionals (n = 30) involved in the development, evaluation, delivery or reception of complex interventions. We identified minor issues around wording of some items; universal issues around how to engage with people at different time points in an intervention; and conceptual issues around the types of people for whom the instrument should be designed. We managed these by adding extra items (n = 6) and including a new set of option responses: 'not relevant at this stage', 'not relevant to my role' and 'not relevant to this intervention' and decided to design an instrument explicitly for those people either delivering or receiving an intervention. This version of the instrument had 53 items. Twenty-three people with a good working knowledge of NPT reviewed the items for theoretical drift. Items that displayed a poor alignment with NPT sub-constructs were removed (n = 8) and others revised or combined (n = 6). The final instrument, with 43 items, was successfully piloted with five people, with a 100% completion rate of items. CONCLUSION: The process of moving through cycles of theoretical translation, item generation, cognitive testing, and theoretical (re)validation was essential for maintaining a balance between the theoretical integrity of the NPT concepts and the ease with which intended respondents could answer the questions. The final instrument could be easily understood and completed, while retaining theoretical validity. NoMAD represents a measure that can be used to understand implementation participants' experiences. It is intended as a measure that can be used alongside instruments that measure other dimensions of implementation activity, such as implementation fidelity, adoption, and readiness.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The Normalization Measure Development Questionnaire (NoMAD) has a Cronbach's Alpha of 0.89.
“The measure is characterized by a Cronbach's Alpha of 0.89 (83).”
From Clinical and Cost-Effectiveness of PSYCHOnlineTHERAPY: Study Protocol of a Multicenter Blended Outpatient Psychotherapy Cluster Randomized Controlled Trial for Patients With Depressive and Anxiety Disorders (2021) · cites this paper as [83]
The Background identifies this as the development paper and says a companion paper presents the validation results. Neither the supplied passages nor searches for “Cronbach's Alpha 0.89” and internal-consistency reliability found an alpha value in our copy, so the cited paper does not verify the claimed 0.89. (LLM, conf=0.76)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009) · Implementation Science · doi:10.1186/1748-5908-4-50 · Cited by 14,496 · see citing works
Developing and evaluating complex interventions: the new Medical Research Council guidance
Craig, P., Dieppe, P., Macintyre, S., Michie, S., Nazareth, I., & Petticrew, M. (2008) · BMJ · doi:10.1136/bmj.a1655 · Cited by 10,995 · see citing works
Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., Griffey, R., & Hensley, M. (2011) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-010-0319-7 · Cited by 8,721 · see citing works
Diffusion of Innovations in Service Organizations: Systematic Review and Recommendations
GREENHALGH, T., ROBERT, G., MACFARLANE, F., BATE, P., & KYRIAKIDOU, O. (2004) · The Milbank Quarterly · doi:10.1111/j.0887-378x.2004.00325.x · Cited by 7,448 · see citing works
Making sense of implementation theories, models and frameworks
Nilsen, P. (2015) · Implementation Science · doi:10.1186/s13012-015-0242-0 · Cited by 4,447 · see citing works
Implementing, Embedding, and Integrating Practices: An Outline of Normalization Process Theory
May, C., & Finch, T. (2009) · Sociology · doi:10.1177/0038038509103208 · Cited by 2,115 · see citing works
Self-citationThe Meaning of Translational Research and Why It Matters
Woolf, S. H. (2008) · JAMA · doi:10.1001/jama.2007.26 · Cited by 1,910 · see citing works
Willis, G. (2005) · doi:10.4135/9781412983655 · Cited by 1,778 · see citing works
Development of a theory of implementation and integration: Normalization Process Theory
May, C. R., Mair, F., Finch, T., MacFarlane, A., Dowrick, C., Treweek, S., Rapley, T., Ballini, L., Ong, B. N., Rogers, A., et al. (2009) · Implementation Science · doi:10.1186/1748-5908-4-29 · Cited by 1,409 · see citing works
Self-citationPlanning and Studying Improvement in Patient Care: The Use of Theoretical Perspectives
GROL, R. P. T. M., BOSCH, M. C., HULSCHER, M. E. J. L., ECCLES, M. P., & WENSING, M. (2007) · The Milbank Quarterly · doi:10.1111/j.1468-0009.2007.00478.x · Cited by 943 · see citing works
Organizational readiness for implementing change: a psychometric assessment of a new measure
Shea, C. M., Jacobs, S., Esserman, D. A., Bruce, K., & Weiner, B. J. (2014) · Implementation Science · doi:10.1186/1748-5908-9-7 · Cited by 863 · see citing works
Chaudoir, S. R., Dugan, A. G., & Barr, C. H. (2013) · Implementation Science · doi:10.1186/1748-5908-8-22 · Cited by 800 · see citing works
Weiner, B. J., Amick, H., & Lee, S.-Y. D. (2008) · Medical Care Research and Review · doi:10.1177/1077558708317802 · Cited by 786 · see citing works
May, C., Finch, T., Mair, F., Ballini, L., Dowrick, C., Eccles, M., Gask, L., MacFarlane, A., Murray, E., Rapley, T., et al. (2007) · BMC Health Services Research · doi:10.1186/1472-6963-7-148 · Cited by 768 · see citing works
Self-citationStewart, D. W., Schwarz, N., & Sudman, S. (1996) · Journal of Marketing Research · doi:10.2307/3152152 · Cited by 692 · see citing works
Towards a general theory of implementation
May, C. (2013) · Implementation Science · doi:10.1186/1748-5908-8-18 · Cited by 656 · see citing works
Self-citationAarons, G. A., Ehrhart, M. G., & Farahnak, L. R. (2014) · Implementation Science · doi:10.1186/1748-5908-9-45 · Cited by 559 · see citing works
Lewis, C. C., Fischer, S., Weiner, B. J., Stanick, C., Kim, M., & Martinez, R. G. (2015) · Implementation Science · doi:10.1186/s13012-015-0342-x · Cited by 430 · see citing works
McEvoy, R., Ballini, L., Maltoni, S., O’Donnell, C. A., Mair, F. S., & MacFarlane, A. (2014) · Implementation Science · doi:10.1186/1748-5908-9-2 · Cited by 413 · see citing works
Self-citationGallacher, K., May, C. R., Montori, V. M., & Mair, F. S. (2011) · The Annals of Family Medicine · doi:10.1370/afm.1249 · Cited by 356 · see citing works
Self-citationThe analysis and interpretation of cognitive interviews for instrument development
Knafl, K. A., Deatrick, J. A., Gallo, A. M., Holcombe, G., Bakitas, M. A., Dixon, J. K., & Grey, M. J. (2007) · Research in Nursing & Health · doi:10.1002/nur.20195 · Cited by 342 · see citing works
Finch, T. L., Girling, M., May, C. R., Mair, F. S., Murray, E., Treweek, S., McColl, E., Steen, I. N., Cook, C., Vernazza, C. R., et al. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0591-x · Cited by 337 · see citing works
Self-citationGlasgow, R. E., & Riley, W. T. (2013) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2013.03.010 · Cited by 318 · see citing works
Why is it difficult to implement e-health initiatives? A qualitative study
Murray, E., Burns, J., May, C., Finch, T., O'Donnell, C., Wallace, P., & Mair, F. (2011) · Implementation Science · doi:10.1186/1748-5908-6-6 · Cited by 270 · see citing works
Self-citationInstrumentation issues in implementation science
Martinez, R. G., Lewis, C. C., & Weiner, B. J. (2014) · Implementation Science · doi:10.1186/s13012-014-0118-8 · Cited by 243 · see citing works
May, C. R., Finch, T. L., Ballini, L., MacFarlane, A. E., Mair, F. S., Murray, E., Treweek, S., & Rapley, T. (2011) · Arrow@dit (Dublin Institute of Technology) · doi:10.34961/10032 · Cited by 235 · see citing works
Self-citationUsing a Theory-Driven Conceptual Framework in Qualitative Health Research
MacFarlane, A. E., & Brún, M. O.-D. (2011) · Qualitative Health Research · doi:10.1177/1049732311431898 · Cited by 229 · see citing works
Context matters: measuring implementation climate among individuals and groups
Jacobs, S., Weiner, B. J., & Bunger, A. C. (2014) · Implementation Science · doi:10.1186/1748-5908-9-46 · Cited by 189 · see citing works
Finch, T. L., Rapley, T., Girling, M., Mair, F. S., Murray, E., Treweek, S., McColl, E., Steen, I. N., & May, C. R. (2013) · Implementation Science · doi:10.1186/1748-5908-8-43 · Cited by 177 · see citing works
Self-citationElwyn, G., Légaré, F., Weijden, T. V. D., Edwards, A., & May, C. (2008) · Implementation Science · doi:10.1186/1748-5908-3-57 · Cited by 159 · see citing works
Self-citationFoy, R., Ovretveit, J., Shekelle, P. G., Pronovost, P. J., Taylor, S. L., Dy, S., Hempel, S., McDonald, K. M., Rubenstein, L. V., & Wachter, R. M. (2011) · BMJ Quality & Safety · doi:10.1136/bmjqs.2010.047993 · Cited by 148 · see citing works
Gagnon, M., Attieh, R., Ghandour, E. K., Légaré, F., Ouimet, M., Estabrooks, C. A., & Grimshaw, J. (2014) · PLoS ONE · doi:10.1371/journal.pone.0114338 · Cited by 145 · see citing works
Rogers, A. E., Vassilev, I., Sanders, C., Kirk, S. A., Chew-Graham, C. A., Kennedy, A. P., Protheroe, J., Bower, P., Blickem, C., Reeves, D., et al. (2011) · Implementation Science · doi:10.1186/1748-5908-6-56 · Cited by 129 · see citing works
Finch, T. L., Mair, F. S., O’Donnell, C., Murray, E., & May, C. R. (2012) · BMC Medical Research Methodology · doi:10.1186/1471-2288-12-69 · Cited by 127 · see citing works
Self-citationImplementation research: state of the art and future directions
Linton, J. D. (2002) · Technovation · doi:10.1016/s0166-4972(01)00075-x · Cited by 126 · see citing works
Clinton-McHarg, T., Yoong, S. L., Tzelepis, F., Regan, T., Fielding, A., Skelton, E., Kingsland, M., Ooi, J. Y., & Wolfenden, L. (2016) · Implementation Science · doi:10.1186/s13012-016-0512-5 · Cited by 116 · see citing works
Understanding factors that inhibit or promote the utilization of telecare in chronic lung disease
Mair, F. S., Hiscock, J., & Beaton, S. C. (2008) · Chronic Illness · doi:10.1177/1742395308092482 · Cited by 115 · see citing works
Self-citationImplementing a stepped-care approach in primary care: results of a qualitative study
Franx, G., Oud, M., de Lange, J., Wensing, M., & Grol, R. (2012) · Implementation Science · doi:10.1186/1748-5908-7-8 · Cited by 106 · see citing works
Understanding the implementation and integration of e-health services
Mair, F., May, C., Finch, T., Murray, E., Anderson, G. K., Sullivan, F., O'Donnell, C., Wallace, P., & Epstein, O. (2007) · Journal of Telemedicine and Telecare · doi:10.1258/135763307781645112 · Cited by 105 · see citing works
Self-citationSquires, J. E., Estabrooks, C. A., O'Rourke, H. M., Gustavsson, P., Newburn-Cook, C. V., & Wallin, L. (2011) · Implementation Science · doi:10.1186/1748-5908-6-83 · Cited by 101 · see citing works
Development and formative evaluation of the e-Health Implementation Toolkit (e-HIT)
Murray, E., May, C., & Mair, F. (2010) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-10-61 · Cited by 96 · see citing works
Self-citationBlakeman, T., Protheroe, J., Chew-Graham, C., Rogers, A., & Kennedy, A. (2012) · British Journal of General Practice · doi:10.3399/bjgp12x636056 · Cited by 96 · see citing works
Sanders, T., Foster, N. E., & Ong, B. N. (2011) · BMC Medicine · doi:10.1186/1741-7015-9-49 · Cited by 84 · see citing works
Dyson, J., Lawton, R., Jackson, C., & Cheater, F. (2013) · Implementation Science · doi:10.1186/1748-5908-8-111 · Cited by 81 · see citing works
Exploring implementation and sustainability of models of care: can theory help?
Forster, D. A., Newton, M. S., McLachlan, H., & Willis, K. F. (2011) · BMC Public Health · doi:10.1186/1471-2458-11-s5-s8 · Cited by 76 · see citing works
Embedding effective depression care: using theory for primary care organisational and systems change
Gunn, J. M., Palmer, V. J., Dowrick, C. F., Herrman, H., Griffiths, F. E., Kokanović, R., Blashki, G. A., Hegarty, K. L., Johnson, C., Potiriadis, M., et al. (2010) · Implementation Science · doi:10.1186/1748-5908-5-62 · Cited by 75 · see citing works
Self-citationGask, L., Bower, P., Lovell, K., Escott, D., Archer, J., Gilbody, S., Lankshear, A. J., Simpson, A. E., & Richards, D. A. (2010) · Implementation Science · doi:10.1186/1748-5908-5-15 · Cited by 75 · see citing works
Kennedy, A., Chew-Graham, C., Blakeman, T., Bowen, A., Gardner, C., Protheroe, J., Rogers, A., & Gask, L. (2010) · Implementation Science · doi:10.1186/1748-5908-5-7 · Cited by 66 · see citing works
A coding system for appraising questionnaires.
Lessler, J. T., & Forsyth, B. (1996) · Cited by 64 · see citing works
Measurement of a model of implementation for health care: toward a testable theory
Cook, J. M., O’Donnell, C., Dinnen, S., Coyne, J. C., Ruzek, J. I., & Schnurr, P. P. (2012) · Implementation Science · doi:10.1186/1748-5908-7-59 · Cited by 59 · see citing works
QUAID: A questionnaire evaluation aid for survey methodologists
Graesser, A. C., Wiemer-Hastings, K., Kreuz, R. J., Hastings, P., & Marquis, K. H. (2000) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03207792 · Cited by 56 · see citing works
Beyond the limits of clinical governance? The case of mental health in English primary care
Gask, L. L., Rogers, A. E., Campbell, S., & Sheaff, R. (2008) · BMC Health Services Research · doi:10.1186/1472-6963-8-63 · Cited by 55 · see citing works
Atkins, S., Lewin, S., Ringsberg, K. C., & Thorson, A. (2011) · BMC Health Services Research · doi:10.1186/1472-6963-11-275 · Cited by 42 · see citing works
Stamatakis, K. A., McQueen, A., Filler, C., Boland, E., Dreisinger, M., Brownson, R. C., & Luke, D. A. (2012) · Implementation Science · doi:10.1186/1748-5908-7-65 · Cited by 42 · see citing works
James, D. M. (2011) · Implementation Science · doi:10.1186/1748-5908-6-95 · Cited by 22 · see citing works
Information systems development: a normalisation process theory perspective
Sooklal, R., Παπαδόπουλος, Θ., & Ojiako, U. (2011) · Industrial Management & Data Systems · doi:10.1108/02635571111170794 · Cited by 18 · see citing works
Process evaluation of infertility management in primary care: has open access HSG been normalized?
Wilkes, S., & Rubin, G. P. (2009) · Primary Health Care Research & Development · doi:10.1017/s1463423609990168 · Cited by 10 · see citing works
May, C. R., Finch, T., Ballini, L., MacFarlane, A., Mair, F., Murray, E., Treweek, S., & Rapley, T. (2011) · BMC Health Services Research · doi:10.1186/1472-6963-11-245 · Cited by 3 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Action, actor, context, target, time (AACTT): a framework for specifying behaviour
Presseau, J., McCleary, N., Lorencatto, F., Patey, A. M., Grimshaw, J. M., & Francis, J. J. (2019) · Implementation Science · doi:10.1186/s13012-019-0951-x · Cited by 411 · see citing works
Finch, T. L., Girling, M., May, C. R., Mair, F. S., Murray, E., Treweek, S., McColl, E., Steen, I. N., Cook, C., Vernazza, C. R., et al. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0591-x · Cited by 337 · see citing works
Self-citationMay, C. R., Albers, B., Bracher, M., Finch, T. L., Gilbert, A. W., Girling, M., Greenwood, K., MacFarlane, A. E., Mair, F. S., May, C., et al. (2022) · Implementation Science · doi:10.1186/s13012-022-01191-x · Cited by 251 · see citing works
Self-citationVis, C., Ruwaard, J., Finch, T. L., Rapley, T., Beurs, D. P. D., van Stel, H. F., van Lettow, B., Mol, M., Kleiboer, A. M., Riper, H. M., et al. (2019) · Journal of Medical Internet Research · doi:10.2196/12376 · Cited by 84 · see citing works
Self-citationHall, A., Shoesmith, A., Doherty, E., McEvoy, B., Mettert, K., Lewis, C. C., Wolfenden, L., Yoong, S., Kingsland, M., Shelton, R. C., et al. (2022) · Implementation Science · doi:10.1186/s13012-022-01252-1 · Cited by 81 · see citing works
Elf, M., Nordmark, S., Lyhagen, J., Lindberg, I., Finch, T. L., & Åberg, A. C. (2018) · Implementation Science · doi:10.1186/s13012-018-0835-5 · Cited by 75 · see citing works
Self-citationDanner, M., Geiger, F., Wehkamp, K., Rueffer, J. U., Kuch, C., Sundmacher, L., Skjelbakken, T., Rummer, A., Novelli, A., Debrouwere, M., et al. (2020) · BMJ Open · doi:10.1136/bmjopen-2020-037575 · Cited by 67 · see citing works
Sermeus, W., Aiken, L. H., Ball, J., Bridges, J., Bruyneel, L., Busse, R., Witte, H. D., Dello, S., Drennan, J., Eriksson, L. E., et al. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-059159 · Cited by 54 · see citing works
Bührmann, L., Schuurmans, J., Ruwaard, J., Fleuren, M. A. H., Etzelmueller, A., Piera-Jiménez, J., Finch, T. L., Rapley, T., Potthoff, S., Aouizerate, B., et al. (2020) · Trials · doi:10.1186/s13063-020-04686-4 · Cited by 51 · see citing works
Self-citationWolfenden, L., McCrabb, S., Barnes, C., O’Brien, K. M., Ng, K., Nathan, N., Sutherland, R., Hodder, R. K., Tzelepis, F., Nolan, E., et al. (2022) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011677.pub3 · Cited by 49 · see citing works
Implementation in rehabilitation: a roadmap for practitioners and researchers
Morris, J. H., Bernhardsson, S., Bird, M., Connell, L., Lynch, E. A., Jarvis, K. A., Kayes, N. M., Miller, K. J., Mudge, S., & Fisher, R. J. (2019) · Disability and Rehabilitation · doi:10.1080/09638288.2019.1587013 · Cited by 45 · see citing works
Changing healthcare professionals' non-reflective processes to improve the quality of care
Potthoff, S., Kwaśnicka, D., Avery, L., Finch, T. L., Gardner, B., Hankonen, N. E., Johnston, D. W., Johnston, M., Kok, G., Lally, P. J., et al. (2022) · Social Science & Medicine · doi:10.1016/j.socscimed.2022.114840 · Cited by 47 · see citing works
Self-citationVis, C., Schuurmans, J., Aouizerate, B., Craggs, M. A., Batterham, P. J., Bührmann, L., Calear, A. L., Pashoja, A. C., Christensen, H. M., Dozeman, E., et al. (2022) · Journal of Medical Internet Research · doi:10.2196/41532 · Cited by 42 · see citing works
Self-citationHafid, S., Howard, M., Guenter, D., Elston, D., Fikree, S., Gallagher, E., Winemaker, S., & Waters, H. (2021) · BMC Palliative Care · doi:10.1186/s12904-021-00817-z · Cited by 45 · see citing works
Baumeister, H., Bauereiss, N., Zarski, A.-C., Braun, L., Buntrock, C., Hoherz, C., Idrees, A. R., Kraft, R., Meyer, P., Nguyen, T. B. D., et al. (2021) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2021.660534 · Cited by 30 · see citing works
Lewis, N. V., Dowrick, A., Sohal, A. H., Feder, G. S., & Griffiths, C. (2019) · Health & Social Care in the Community · doi:10.1111/hsc.12733 · Cited by 32 · see citing works
A cross-sectional survey of the pharmacy workforce in general practice in Scotland
Stewart, D. C., MacLure, K., Newham, R., Smith, K. G., Bruce, R., Cunningham, S., MacLure, A., Fry, S., Mackerrow, J., & Bennie, M. (2019) · Family Practice · doi:10.1093/fampra/cmz052 · Cited by 31 · see citing works
Salisbury, T. T., Kohrt, B. A., Bakolis, I., Jordans, M. J. D., Hull, L., Luitel, N. P., McCrone, P., Sevdalis, N., Pokhrel, P., Carswell, K., et al. (2021) · JMIR Research Protocols · doi:10.2196/24115 · Cited by 31 · see citing works
Bissett, S. M., Rapley, T., Preshaw, P. M., & Presseau, J. (2020) · BMJ Open · doi:10.1136/bmjopen-2019-032369 · Cited by 26 · see citing works
Self-citationPerry, L. M., Morken, V., Peipert, J. D., Yanez, B. R., Garcia, S. F., Barnard, C., Hirschhorn, L. R., Linder, J. A., Jordan, N., Ackermann, R. T., et al. (2022) · JMIR Research Protocols · doi:10.2196/38461 · Cited by 25 · see citing works
Cross, S. P. M., Nicholas, J., Mangelsdorf, S. N., Valentine, L., Baker, S. T. E., McGorry, P. D., Gleeson, J., & Álvarez‐Jiménez, M. (2023) · JMIR Formative Research · doi:10.2196/49846 · Cited by 24 · see citing works
The implementation and validation of the NoMAD during a complex primary care intervention
Lamarche, L., Clark, R. E., Parascandalo, F., & Mangin, D. A. (2022) · BMC Medical Research Methodology · doi:10.1186/s12874-022-01655-0 · Cited by 22 · see citing works
Cross-cultural adaptation of the NoMAD questionnaire to Brazilian Portuguese
Loch, A. P., Finch, T. L., Fonsi, M., & de Soárez, P. C. (2020) · Revista da Associação Médica Brasileira · doi:10.1590/1806-9282.66.10.1383 · Cited by 21 · see citing works
Self-citationSchubbe, D. C., Yen, R. W., Saunders, C. H., Elwyn, G. J., Forcino, R. C., O’Malley, A. J., Politi, M. C., Margenthaler, J. A., Volk, R. J., Sepucha, K., et al. (2021) · Implementation Science · doi:10.1186/s13012-021-01115-1 · Cited by 21 · see citing works
Depla, A. L., Pluut, B., Ruiter, M. L.-D., Kersten, A. W., Evers, I. M., Franx, A., & Bekker, M. N. (2023) · Journal of Patient-Reported Outcomes · doi:10.1186/s41687-023-00568-w · Cited by 20 · see citing works
Newnam, S., Goode, N., Read, G. J. M., & Salmon, P. M. (2020) · Safety Science · doi:10.1016/j.ssci.2020.104844 · Cited by 20 · see citing works
Seipp, H., Haasenritter, J., Hach, M., Becker, D., Schütze, D., Engler, J., Bösner, S., & Kuss, K. (2022) · BMC Palliative Care · doi:10.1186/s12904-022-01109-w · Cited by 19 · see citing works
Jurczuk, M., Bidwell, P., Martinez, D., Silverton, L. I., van der Meulen, J., Wolstenholme, D., Thakar, R., Gurol‐Urganci, I., & Sevdalis, N. (2021) · Implementation Science · doi:10.1186/s13012-021-01125-z · Cited by 18 · see citing works
Nilsen, P., Potthoff, S., & Birken, S. A. (2022) · Frontiers in Health Services · doi:10.3389/frhs.2021.795144 · Cited by 17 · see citing works
Cullington, H. E., Kitterick, P. T., Darnton, P., Finch, T. L., Greenwell, K., Riggs, C., Weal, M. J., Walker, D., & Sibley, A. (2021) · JMIR Research Protocols · doi:10.2196/27207 · Cited by 15 · see citing works
Self-citationEn Route to Implementation Science 3.0
Albers, B., Shlonsky, A., & Mildon, R. L. (2020) · Springer eBooks · doi:10.1007/978-3-030-03874-8_1 · Cited by 15 · see citing works
Meloncelli, N. J. L., Barnett, A. G., & de Jersey, S. (2020) · BMC Pregnancy and Childbirth · doi:10.1186/s12884-020-03352-6 · Cited by 15 · see citing works
Bjerk, M., Flottorp, S. A., Pripp, A. H., Øien, H., Hansen, T. M., Foy, R., Close, J., Linnerud, S., Brovold, T., Solli, R., et al. (2024) · Implementation Science · doi:10.1186/s13012-024-01334-2 · Cited by 15 · see citing works
Freund, J., Piotrowski, A., Bührmann, L., Oehler, C., Titzler, I., Netter, A.-L., Potthoff, S., Ebert, D. D., Finch, T. L., Köberlein–Neu, J., et al. (2023) · Implementation Science Communications · doi:10.1186/s43058-023-00505-4 · Cited by 14 · see citing works
Self-citationLee, D. C. W., O’Brien, K. M., McCrabb, S., Wolfenden, L., Tzelepis, F., Barnes, C., Yoong, S., Bartlem, K. M., & Hodder, R. K. (2024) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011677.pub4 · Cited by 15 · see citing works
Hultstrand, J. N., Engström, E., Målqvist, M., Tydén, T., Maseko, N., & Jonsson, M. (2020) · PLoS ONE · doi:10.1371/journal.pone.0236712 · Cited by 14 · see citing works
Vos, J. A. M., de Best, R., Duineveld, L. A. M., van Weert, H. C. P. M., & van Asselt, K. M. (2022) · BMC Family Practice · doi:10.1186/s12875-021-01610-w · Cited by 14 · see citing works
Garcia, S. F., Smith, J. D., Kallen, M. A., Webster, K., Lyleroehr, M., Kircher, S. M., Bass, M., Cella, D. F., & Penedo, F. J. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-059563 · Cited by 14 · see citing works
Pritchard, L., Thompson‐Hodgetts, S., McKillop, A. B., Rosychuk, R. J., Mrklas, K., Zwaigenbaum, L., Zwicker, J. D., Andersen, J. C., King, G. A., & Firouzeh, P. (2022) · BMC Pediatrics · doi:10.1186/s12887-022-03381-4 · Cited by 16 · see citing works
Scott, J., Finch, T. L., Bevan, M. T., Maniatopoulos, G., Gibbins, C., Yates, B., Narayanan, K., Sheerin, N., & Kanagasundaram, N. S. (2019) · BMJ Open · doi:10.1136/bmjopen-2019-032925 · Cited by 14 · see citing works
Self-citationLoch, A. P., Rocha, S. Q., Fonsi, M., Caraciolo, J. M. D. M., Kalichman, A. O., Souza, R. D. A., Gianna, M. C., Gonçalves, A., Short, D., Pimenta, S. L., et al. (2021) · PLoS ONE · doi:10.1371/journal.pone.0250060 · Cited by 13 · see citing works
Geriatric Assessment in CKD Care: An Implementation Study
Voorend, C. G. N., Berkhout-Byrne, N. C., van Bodegom‐Vos, L., Diepenbroek, A., Franssen, C. F. M., Joosten, H., Mooijaart, S. P., Bos, W. J. W., van Buren, M., van Alphen, A., et al. (2024) · Kidney Medicine · doi:10.1016/j.xkme.2024.100809 · Cited by 13 · see citing works
Validity and reliability of the Chinese version of the Normalization MeAsure Development(NoMAD)
Jiang, M., Wáng, Q., Finch, T. L., She, D., Zhou, Y., Chung, Y. F., Chen, J., & Han, L. (2022) · BMC Health Services Research · doi:10.1186/s12913-022-08737-1 · Cited by 12 · see citing works
Self-citationCarter, H. M., Beard, D. J., Harvey, A., Leighton, P. A., Moffatt, F., Smith, B. E., Webster, K. E., & Logan, P. (2023) · Implementation Science Communications · doi:10.1186/s43058-023-00499-z · Cited by 12 · see citing works
Engeltjes, B., Rosman, A. N., Scheele, F., Vis, C., & Wouters, E. (2022) · JMIR Formative Research · doi:10.2196/33709 · Cited by 12 · see citing works
Quint, E. E., Haanstra, A. J., van der Veen, Y., Maring, H., Berger, S. P., Ranchor, A. V., Bakker, S. J. L., Finnema, E., Pol, R. A., & Annema, C. (2023) · BMJ Open · doi:10.1136/bmjopen-2023-072805 · Cited by 12 · see citing works
Bernhardsson, S., Boman, C., Lundqvist, S., Arvidsson, D., Börjesson, M., Larsson, M. E. H., Lundh, H., Melin, K., Nilsén, P., & Lauruschkus, K. (2022) · Pilot and Feasibility Studies · doi:10.1186/s40814-022-01075-3 · Cited by 12 · see citing works
Jones, A. S. K., Harding, S., Seaton, N., Hudson, J. L., Duff, A., Wroe, A. L., Singh, H., Norton, S., Picariello, F., & Moss-Morris, R. E. (2024) · Contemporary Clinical Trials · doi:10.1016/j.cct.2024.107658 · Cited by 11 · see citing works
Richards, D. A., Sugg, H. V. R., Cockcroft, E., Cooper, J. E., Cruickshank, S., Doris, F. E., Hulme, C. T., Logan, P., Iles‐Smith, H., Meléndez‐Torres, G. J., et al. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-046436 · Cited by 11 · see citing works
Jia, M., Gu, Y., Chen, Y., Tu, J., Liu, Y., Lu, H., Huang, S., Li, J., & Zhou, H. (2021) · Nursing Open · doi:10.1002/nop2.1121 · Cited by 11 · see citing works
Leinert, C., Brefka, S., Braisch, U., Denninger, N.-E., Müller, M., Benzinger, P., Bauer, J., Bahrmann, A., Frey, N., Katus, H. A., et al. (2021) · BMC Geriatrics · doi:10.1186/s12877-021-02585-0 · Cited by 11 · see citing works
Cook, C., Finch, T. L., Sharma, S., Girling, M., Rapley, T., & Vernazza, C. R. (2019) · European Journal Of Dental Education · doi:10.1111/eje.12481 · Cited by 10 · see citing works
Self-citationRidgeway, J. L., Branda, M. E., Gravholt, D., Brito, J. P., Hargraves, I. G., Hartasanchez, S. A., Leppin, A. L., Gomez, Y. L., Mann, D. M., Nautiyal, V., et al. (2021) · Implementation Science Communications · doi:10.1186/s43058-021-00145-6 · Cited by 10 · see citing works
Kvæl, L. A. H., Gulbrandsen, P., Werner, A., & Bergland, A. (2024) · BMC Health Services Research · doi:10.1186/s12913-024-11647-z · Cited by 10 · see citing works
On behalf of the RESPECT 2 study group, van Hoorn, F., Koster, M. P. H., Kwee, A., Groenendaal, F., Franx, A., & Bekker, M. N. (2021) · BMC Pregnancy and Childbirth · doi:10.1186/s12884-021-03749-x · Cited by 9 · see citing works
Strid, E. N., Wallin, L., & Nilsagård, Y. E. (2022) · JMIR Research Protocols · doi:10.2196/37634 · Cited by 10 · see citing works
Upscaling telemonitoring in Dutch University Medical Centres: A baseline measurement
Gijsbers, H., Kleiss, J., Nurmohamed, A. S., van de Belt, T. H., Schijven, M. P., Bekker, M. N., Buijs, M. A., Buskens, E., Chavannes, N. H., van Dam, D., et al. (2023) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2023.105085 · Cited by 9 · see citing works
Perspectives of general practitioners about a collaborative asthma care model in primary care
Qazi, A., Armour, C. L., & Saini, B. (2020) · Journal of Asthma · doi:10.1080/02770903.2020.1823408 · Cited by 9 · see citing works
Hjort (2021) · doi:10.1136/bmjopen-2020-042728 · Cited by 11 · see citing works
Gooberman‐Hill, R., Wylde, V., Bertram, W., Moore, A. J., Pinedo‐Villanueva, R., Sanderson, E., Dennis, J. A., Harris, S. R. S., Judge, A., Noble, S., et al. (2023) · Programme Grants for Applied Research · doi:10.3310/watm4500 · Cited by 10 · see citing works
Boman, C., Bernhardsson, S., Lauruschkus, K., Lundqvist, S., & Melin, K. (2023) · Frontiers in Health Services · doi:10.3389/frhs.2022.1102328 · Cited by 8 · see citing works
Batterham, P. J., Allenhof, C., Pashoja, A. C., Etzelmueller, A., Fanaj, N., Finch, T. L., Freund, J., Hanssen, D. J. C., Mathiasen, K., Piera-Jiménez, J., et al. (2024) · Implementation Research and Practice · doi:10.1177/26334895241245448 · Cited by 9 · see citing works
Self-citationLupu, D., Aldous, A. M., Harbert, G., Tamura, M. K., Holdsworth, L. M., Nicklas, A. C., Vinson, B., & Moss, A. H. (2020) · Kidney360 · doi:10.34067/kid.0005892020 · Cited by 8 · see citing works
Nicoll, A., Maxwell, M., & Williams, B. (2021) · Implementation Science Communications · doi:10.1186/s43058-021-00159-0 · Cited by 8 · see citing works
Gordon, K., Dainty, K., Gray, C. S., DeLacy, J., Shah, A., & Seto, E. (2022) · JMIR Nursing · doi:10.2196/36346 · Cited by 8 · see citing works
Schubbe, D. C., Yen, R. W., Leavitt, H. B., Forcino, R. C., Jacobs, C., Friedman, E. B., McEvoy, M. P., Rosenkranz, K. M., Rojas, K. E., Bradley, A., et al. (2023) · Implementation Science Communications · doi:10.1186/s43058-023-00453-z · Cited by 7 · see citing works
Mainstream and special schools’ use of well‐being programmes: A regional survey
Nisar, A., Hastings, R., Watkins, R. C., & Williams, S. (2023) · British Educational Research Journal · doi:10.1002/berj.3903 · Cited by 6 · see citing works
Pallan, M. J., Murphy, M., Morrison, B., Pokhilenko, I., Sitch, A. J., Frew, E. J., Rawdin, C., Adams, R. L., Adamson, A., Bartington, S., et al. (2023) · Public Health Research · doi:10.3310/ttpl8570 · Cited by 8 · see citing works
Pease, A. S., Lambert, B., Ingram, J. C., Bradley, N., Fleming, P., Blair, P. S., & Farr, M. (2025) · BMJ Open · doi:10.1136/bmjopen-2024-091414 · Cited by 7 · see citing works
Augustsson, H., Costea, V.-A., Eriksson, L., Hasson, H., Bäck, A., Åhström, M., & Bergström, A. (2021) · Implementation Science Communications · doi:10.1186/s43058-021-00233-7 · Cited by 7 · see citing works
Batten, M., Kosari, S., Lewis, J. M., Naunton, M., & Strickland, K. (2023) · Health & Social Care in the Community · doi:10.1155/2023/4370884 · Cited by 6 · see citing works
Bennett, C. L., Fraser, C., Hannigan, B., Sawle, L., Williams, J., Jones, A., Pryjmachuk, S., Elliott, M. B., Elliott, M., Bennett, N., et al. (2025) · BMC Health Services Research · doi:10.1186/s12913-025-13446-6 · Cited by 6 · see citing works
Moore, A. J., Wylde, V., Bertram, W., Beswick, A. D., Howells, N., & Gooberman‐Hill, R. (2023) · PLoS ONE · doi:10.1371/journal.pone.0284406 · Cited by 7 · see citing works
Vos, J. A. M., Wollersheim, B. M., van Weert, H. C. P. M., van de Poll-Franse, L. V., & van Asselt, K. M. (2024) · JCO Oncology Practice · doi:10.1200/op-24-00491 · Cited by 6 · see citing works
Netter, A.-L., Etzelmueller, A., Kircher, T. T. J., Rapley, T., Ebert, D. D., & Brakemeier, E. (2022) · Journal of Technology in Behavioral Science · doi:10.1007/s41347-021-00237-z · Cited by 5 · see citing works
Self-citationGijsbers, H., Nurmohamed, A. S., van de Belt, T. H., Schijven, M. P., & The Citrien 2 Project Leaders and Steering Group (2023) · JMIR Research Protocols · doi:10.2196/45201 · Cited by 6 · see citing works
Bradley, P. T., & Patterson, J. M. (2021) · Journal of Voice · doi:10.1016/j.jvoice.2021.06.027 · Cited by 6 · see citing works
Anyango, J. F., Yost, J., Dobson, A., Nkalubo, J., & McKeever, A. E. (2024) · Journal of Advanced Nursing · doi:10.1111/jan.16198 · Cited by 5 · see citing works
Smith, J. D., Mohanty, N., Davis, M. M., Knapp, A. A., Tedla, Y. G., Carroll, A. J., Price, H. E., Villamar, J. A., Padilla, R., Jordan, N., et al. (2020) · Implementation Science Communications · doi:10.1186/s43058-020-00039-z · Cited by 5 · see citing works
May, C. R., Albers, B., Bracher, M., Finch, T. L., Gilbert, A. W., Girling, M., Greenwood, K., MacFarlane, A. E., Mair, F. S., May, C., et al. (2021) · Research Square · doi:10.21203/rs.3.rs-870733/v1 · Cited by 5 · see citing works
Self-citationKuharić, M., Merle, J. L., Cella, D. F., Mitchell, S. A., DiMartino, L., Ridgeway, J. L., Dizon, D. S., Paudel, R., Austin, J. D., Wong, S. L., et al. (2025) · Implementation Science Communications · doi:10.1186/s43058-025-00756-3 · Cited by 4 · see citing works
Kennedy, K., Snow, A. L., Mills, W. L., Haigh, S. V., Mochel, A. L., Curyto, K. J., Bishop, T. K., Hartmann, C. W., Camp, C. J., & Hilgeman, M. M. (2024) · Dementia · doi:10.1177/14713012241263712 · Cited by 4 · see citing works
Potthoff, S., Brendryen, H., Bosnic, H., Njå, A. L. M., Finch, T. L., & Lid, T. G. (2024) · Frontiers in Health Services · doi:10.3389/frhs.2024.1343568 · Cited by 4 · see citing works
Self-citationHuang, S., Xiao, J., Wang, Q., Chen, L., Tan, Y., Chen, Y., Xie, W., & Xu, P. (2025) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.70226 · Cited by 4 · see citing works
Bjerk, M., Pripp, A. H., Skyrud, K. D., Foy, R., Helbostad, J. L., Skelton, D. A., Close, J., Øien, H., Linnerud, S., Brovold, T., et al. (2025) · Age and Ageing · doi:10.1093/ageing/afaf312 · Cited by 4 · see citing works
Holdsworth, L. M., Stedman, M. R., Saliba‐Gustafsson, E. A., Han, J., Asch, S. M., Harbert, G., Lorenz, K., Lupu, D., Malcolm, E. J., Moss, A. H., et al. (2023) · BMC Health Services Research · doi:10.1186/s12913-023-10360-7 · Cited by 4 · see citing works
Shoesmith, A., Nathan, N., Lum, M., Yoong, S., Nolan, E., Wolfenden, L., Shelton, R. C., Cooper, B. R., Lane, C., Grady, A., et al. (2024) · Implementation Science · doi:10.1186/s13012-024-01372-w · Cited by 4 · see citing works
Beasant, L., Chakkarapani, E., Horwood, J. P., Odd, D., Stocks, S., Parker, D., & Ingram, J. C. (2024) · BMJ Open · doi:10.1136/bmjopen-2024-088228 · Cited by 4 · see citing works
Sturgiss, E. A., Gunatillaka, N., Ball, L., Lam, T., Nielsen, S., O’Donnell, R., Barton, C., Skouteris, H., Tam, C. W. M., Jacka, D., et al. (2021) · BJGP Open · doi:10.3399/bjgpo.2021.0037 · Cited by 4 · see citing works
Read, S., Morgan, J., Gillespie, D., Nollett, C. L., Weiss, M., Allen, D. A., Anderson, P., & Waterman, H. A. (2021) · PLoS ONE · doi:10.1371/journal.pone.0255564 · Cited by 4 · see citing works
Plaisance, A., Heyland, D. K., Laflamme, B., Morin, M., Pageau, F., Girard, A., & LeBlanc, A. (2023) · Mortality · doi:10.1080/13576275.2023.2178291 · Cited by 4 · see citing works
Talla, P. K., Thomas, A., Ataman, R., Auger, C., McKerral, M., Wittich, W., Poncet, F., & Ahmed, S. (2023) · BMJ Open · doi:10.1136/bmjopen-2022-068866 · Cited by 4 · see citing works
Walugembe, D. R., Plamondon, K., Kaharuza, F. M., Waiswa, P., Wylie, L., Wathen, C. N., & Kothari, A. R. (2024) · Health Research Policy and Systems · doi:10.1186/s12961-024-01191-x · Cited by 3 · see citing works
Lorente-Català, R., Gili, M., del Hoyo, Y. L., Mayoral, F., Pérez‐Aranda, A., Castro, A., Moreno, E. V., Baños, R., Roca, M., Monreal‐Bartolomé, A., et al. (2022) · Internet Interventions · doi:10.1016/j.invent.2022.100581 · Cited by 4 · see citing works
Ossenberg, C., Brunelli, V. N., Mitchell, M. L., & Henderson, A. J. (2025) · Journal of Advanced Nursing · doi:10.1111/jan.16768 · Cited by 3 · see citing works
Rai, B., Dixit, K., Dhital, R., Rishal, P., Gurung, S. C., Paudel, P. R., Mishra, G., Bonnett, L. J., de Siqueira-Filha, N. T., Khanal, M. N., et al. (2022) · Wellcome Open Research · doi:10.12688/wellcomeopenres.17669.3 · Cited by 3 · see citing works
Struhar, J., Walters, T., Gracz, K., Sheth, M., Fernández, A., Lopez, C., & Jesus, T. S. (2024) · The International Journal of Health Planning and Management · doi:10.1002/hpm.3832 · Cited by 3 · see citing works
Blütgen, S., Krug, K., Graf, K., Betz, U., Böhm, J., Jäger, E., Krell, V., Müller, J., Pahl, A., Voland, A., et al. (2025) · BMC Cancer · doi:10.1186/s12885-025-13797-7 · Cited by 3 · see citing works
Leysen, D., Reich, B., Carrozzo, E., Crutzen, R., Grote, V., Kumar, D., Mayr, B., Niebauer, J., Pfannerstill, F., Propst, E. M., et al. (2025) · JMIR Research Protocols · doi:10.2196/73704 · Cited by 3 · see citing works
Treating self-harm behaviors in prisons: implementation of the STEPPS program
Lorente-Català, R., Jaén, I., Buscà-Huertas, X., Ferrer, B. F., & García‐Palacios, A. (2025) · BMC Psychology · doi:10.1186/s40359-024-02342-z · Cited by 2 · see citing works
- Authors
- Rapley, T., Girling, M., Mair, F. S., Murray, E., Treweek, S., McColl, E., Steen, I. N., May, C. R., & Finch, T. L.
- Affiliations
- Northumbria University (GB)
- Newcastle University (GB)
- University of Glasgow (GB)
- The Royal Free Hospital (GB)
- University of Aberdeen (GB)
- London School of Hygiene & Tropical Medicine (GB)
- Corresponding author
- Rapley, Tim
- ORCID
- Venue
- BMC Medical Research Methodology
- Published
- 2018-11-15
- Volume
- 18
- Issue
- 1
- Pages
- 133-133
- ISSN
- 1471-2288 (ISSN-L)
- PMC
- PMC6238361
- OpenAlex
- W2982498515
- Cited by
- 193
- References
- 57
- Citations by year
- 2026: 19 · 2025: 31 · 2024: 28 · 2023: 23 · 2022: 37 · 2021: 30 · 2020: 16 · 2019: 7 · 2018: 2
- FWCI
- 21.00
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- BMC Medical Research Methodology (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, other-oa)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Northumbria Research Link (Northumbria University) (acceptedVersion)
- ENLIGHTEN (Jurnal Bimbingan dan Konseling Islam) (acceptedVersion, cc-by-sa)
- LSHTM Research Online (London School of Hygiene and Tropical Medicine) (acceptedVersion)
- Aberdeen University Research Archive (Aberdeen University) (submittedVersion, other-oa)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- UCL Discovery (University College London) (submittedVersion)
- Funders
- Economic and Social Research Council (RES-062-23-3274)
- National Institute for Health Research (Collaboration for Leadership in Applied Health Research and Care North Thames., School for Primary Care Research)
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- Normalization Process Theory, complex interventions, implementation research, NOMAD instrument, instrument development, cognitive interviewing, item generation, theory validation
- MeSH terms
- Implementation Science, Cognition, Delivery of Health Care, Health Services Research, Humans, Interview, Psychological, Pilot Projects, Surveys and Questionnaires
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review