The e-health implementation toolkit: qualitative evaluation across four European countries
Open accessAbstract
BACKGROUND: Implementation researchers have attempted to overcome the research-practice gap in e-health by developing tools that summarize and synthesize research evidence of factors that impede or facilitate implementation of innovation in healthcare settings. The e-Health Implementation Toolkit (e-HIT) is an example of such a tool that was designed within the context of the United Kingdom National Health Service to promote implementation of e-health services. Its utility in international settings is unknown. METHODS: We conducted a qualitative evaluation of the e-HIT in use across four countries--Finland, Norway, Scotland, and Sweden. Data were generated using a combination of interview approaches (n = 22) to document e-HIT users' experiences of the tool to guide decision making about the selection of e-health pilot services and to monitor their progress over time. RESULTS: e-HIT users evaluated the tool positively in terms of its scope to organize and enhance their critical thinking about their implementation work and, importantly, to facilitate discussion between those involved in that work. It was easy to use in either its paper- or web-based format, and its visual elements were positively received. There were some minor criticisms of the e-HIT with some suggestions for content changes and comments about its design as a generic tool (rather than specific to sites and e-health services). However, overall, e-HIT users considered it to be a highly workable tool that they found useful, which they would use again, and which they would recommend to other e-health implementers. CONCLUSION: The use of the e-HIT is feasible and acceptable in a range of international contexts by a range of professionals for a range of different e-health systems.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The eHealth Implementation Toolkit (e-HIT) is aimed specifically at eHealth interventions and has been developed for use by managers and other staff charged with implementing eHealth initiatives.
“A similar toolkit, the eHealth Implementation Toolkit (e-HIT), is aimed specifically at eHealth interventions and has been developed for use by managers and other staff charged with implementing eHealth initiatives (ie, implementers) [120-122].”
From Web-Based Interventions for Behavior Change and Self-Management: Potential, Pitfalls, and Progress (2012) · cites this paper as [122]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
(1998) · Sage eBooks · Cited by 38,631 · see citing works
(1990) · Cited by 24,217 · see citing works
Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory
Wuetherick, B. (2010) · Canadian Journal of University Continuing Education · doi:10.21225/d5g01t · Cited by 8,327 · see citing works
Qualitative research. Introducing focus groups.
Kitzinger, J. (1995) · BMJ · doi:10.1136/bmj.311.7000.299 · Cited by 6,939 · see citing works
From best evidence to best practice: effective implementation of change in patients' care
Grol, R., & Grimshaw, J. (2003) · The Lancet · doi:10.1016/s0140-6736(03)14546-1 · Cited by 4,531 · see citing works
Morse, J. M. (2000) · Qualitative Health Research · doi:10.1177/104973200129118183 · Cited by 3,244 · see citing works
Chaudhry, B., Wang, J., Wu, S., Maglione, M., Mojica, W., Roth, E., Morton, S. C., & Shekelle, P. G. (2006) · Annals of Internal Medicine · doi:10.7326/0003-4819-144-10-200605160-00125 · Cited by 3,159 · see citing works
Implementing, Embedding, and Integrating Practices: An Outline of Normalization Process Theory
May, C., & Finch, T. (2009) · Sociology · doi:10.1177/0038038509103208 · Cited by 2,115 · see citing works
Currivan, D. B., & Gilbert, N. (1994) · Teaching Sociology · doi:10.2307/1318574 · Cited by 1,736 · see citing works
Kitson, A. L., Rycroft-Malone, J., Harvey, G., McCormack, B., Seers, K., & Titchen, A. (2008) · Implementation Science · doi:10.1186/1748-5908-3-1 · Cited by 1,314 · see citing works
Using Reflexivity to Optimize Teamwork in Qualitative Research
Barry, C. A., Britten, N., Barber, N., Bradley, C., & Stevenson, F. A. (1999) · Qualitative Health Research · doi:10.1177/104973299129121677 · Cited by 774 · see citing works
Implementing information systems in health care organizations: myths and challenges
Berg, M. (2001) · International Journal of Medical Informatics · doi:10.1016/s1386-5056(01)00200-3 · Cited by 715 · see citing works
Helfrich, C. D., Li, Y.-F., Sharp, N. D., & Sales, A. E. (2009) · Implementation Science · doi:10.1186/1748-5908-4-38 · Cited by 564 · see citing works
The World Health Report 1999. Making a Difference
Gómez‐Dantés, O. (1999) · Salud Pública de México · doi:10.1590/s0036-36341999000300012 · Cited by 559 · see citing works
Costs and benefits of health information technology.
Shekelle, P. G., Morton, S. C., & Keeler, E. B. (2006) · doi:10.23970/ahrqepcerta132 · Cited by 501 · see citing works
Evidence for public health policy on inequalities: 1: The reality according to policymakers
Petticrew, M. P., Whitehead, M., Macintyre, S. J., Graham, H., & Egan, M. (2004) · Journal of Epidemiology & Community Health · doi:10.1136/jech.2003.015289 · Cited by 380 · see citing works
ICT in Health Care: Sociotechnical Approaches
Berg, M., Aarts, J., & van der Lei, J. (2003) · Methods of Information in Medicine · doi:10.1055/s-0038-1634221 · Cited by 261 · see citing works
May, C. R., Finch, T. L., Ballini, L., MacFarlane, A. E., Mair, F. S., Murray, E., Treweek, S., & Rapley, T. (2011) · Arrow@dit (Dublin Institute of Technology) · doi:10.34961/10032 · Cited by 235 · see citing works
Self-citationAnalytic Strategies and Sample Size
Morse, J. M. (2015) · Qualitative Health Research · doi:10.1177/1049732315602867 · Cited by 194 · see citing works
Integrating Quantitative and Qualitative Methods in Patient Care Information System Evaluation
Berg, M., & Stoop, A. P. (2003) · Methods of Information in Medicine · doi:10.1055/s-0038-1634349 · Cited by 117 · see citing works
Understanding the implementation and integration of e-health services
Mair, F., May, C., Finch, T., Murray, E., Anderson, G. K., Sullivan, F., O'Donnell, C., Wallace, P., & Epstein, O. (2007) · Journal of Telemedicine and Telecare · doi:10.1258/135763307781645112 · Cited by 105 · see citing works
Self-citationDevelopment and formative evaluation of the e-Health Implementation Toolkit (e-HIT)
Murray, E., May, C., & Mair, F. (2010) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-10-61 · Cited by 96 · see citing works
Self-citationPresseau, J., Sniehotta, F. F., Francis, J., & Campbell, N. C. (2009) · Implementation Science · doi:10.1186/1748-5908-4-77 · Cited by 79 · see citing works
Embedding effective depression care: using theory for primary care organisational and systems change
Gunn, J. M., Palmer, V. J., Dowrick, C. F., Herrman, H., Griffiths, F. E., Kokanović, R., Blashki, G. A., Hegarty, K. L., Johnson, C., Potiriadis, M., et al. (2010) · Implementation Science · doi:10.1186/1748-5908-5-62 · Cited by 75 · see citing works
The National Programme for IT in the NHS.
Hassey, A. (2005) · PubMed · Cited by 63 · see citing works
Balka, E. (2003) · Methods of Information in Medicine · doi:10.1055/s-0038-1634225 · Cited by 60 · see citing works
Big is Beautiful: Electronic Patient Records in Large Norwegian Hospitals 1980s – 2001
Ellingsen, G., & Monteiro, E. (2003) · Methods of Information in Medicine · doi:10.1055/s-0038-1634231 · Cited by 45 · see citing works
Wen, K., Gustafson, D. H., Hawkins, R. P., Brennan, P. F., Dinauer, S., Johnson, P. R., & Siegler, T. (2010) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2010.005546 · Cited by 33 · see citing works
Short- and long-term effects of a quality improvement collaborative on diabetes management
Schouten, L. M. T., Hulscher, M., van Everdingen, J. J., Huijsman, R., Niessen, L. W., & Grol, R. P. T. M. (2010) · Implementation Science · doi:10.1186/1748-5908-5-94 · Cited by 26 · see citing works
the ReBEQI group, Akl, E. A., Treweek, S. P., Foy, R., Francis, J., & Oxman, A. D. (2007) · Implementation Science · doi:10.1186/1748-5908-2-19 · Cited by 21 · see citing works
Harrop, N., Wood‐Harper, T., & Gillies, A. C. (2006) · Health Informatics Journal · doi:10.1177/1460458206069759 · Cited by 14 · see citing works
Roberts, A., Reponen, J., Pesola, U.-M., Waterworth, E., Larsen, F., Mäkiniemi, M., Heaney, D., Wakeling, M., McFarlane, A., Winblad, I., et al. (2010) · Telemedicine Journal and e-Health · doi:10.1089/tmj.2010.0075 · Cited by 12 · see citing works
Self-citationMay, C. R., Finch, T., Ballini, L., MacFarlane, A., Mair, F., Murray, E., Treweek, S., & Rapley, T. (2011) · BMC Health Services Research · doi:10.1186/1472-6963-11-245 · Cited by 3 · see citing works
Self-citationThe Telemedicine Tool Kit: A Workbook for NHS Doctors, Nurses and Managers
Lilley, R. (1999) · Cited by 2 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Elbert, N. J., van Os‐Medendorp, H., van Renselaar, W., Ekeland, A. G., Roijen, L. H., Raat, H., Nijsten, T. E. C., & Pasmans, S. G. M. A. (2014) · Journal of Medical Internet Research · doi:10.2196/jmir.2790 · Cited by 350 · see citing works
Web-Based Interventions for Behavior Change and Self-Management: Potential, Pitfalls, and Progress
Murray, E. (2012) · Medicine 2.0 · doi:10.2196/med20.1741 · Cited by 219 · see citing works
Self-citationeHealth and quality in health care: implementation time
Ossebaard, H. C., & Gemert-Pijnen, L. V. (2016) · International Journal for Quality in Health Care · doi:10.1093/intqhc/mzw032 · Cited by 141 · see citing works
Ariëns, L. F. M., Schussler-Raymakers, F. M., Frima, C., Flinterman, A. E., Hamminga, E. A., Arents, B. W. M., Bruijnzeel-Koomen, C. A. F. M., de Bruin-Weller, M. S., & van Os‐Medendorp, H. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.7512 · Cited by 110 · see citing works
Devlin, A. M., Lennon, M., O’Donnell, C. A., Bouamrane, M.-M., Agbakoba, R., O’Connor, S., Grieve, E., Finch, T. L., Wyke, S., Watson, N., et al. (2015) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocv097 · Cited by 89 · see citing works
Optimizing telehealth pain care after COVID-19
Tauben, D. J., Langford, D. J., Sturgeon, J. A., Rundell, S. D., Towle, C., Bockman, C., & Nicholas, M. (2020) · Pain · doi:10.1097/j.pain.0000000000002048 · Cited by 75 · see citing works
Evans, B. A., Snooks, H. A., Howson, H., & Davies, M. (2013) · Implementation Science · doi:10.1186/1748-5908-8-17 · Cited by 74 · see citing works
MacFarlane, A. E., O’Donnell, C. A., Mair, F. S., Brún, M. O.-D., de Brún, T., Spiegel, W., van den Muijsenbergh, M. E. T. C., van Weel-Baumgarten, E. M., Lionis, C. D., Burns, N., et al. (2012) · Implementation Science · doi:10.1186/1748-5908-7-111 · Cited by 47 · see citing works
Self-citationScott, I. A., Sullivan, C. M., & Staib, A. N. (2018) · Australian Health Review · doi:10.1071/ah17153 · Cited by 39 · see citing works
Jones, R. (2013) · Medicine 2.0 · doi:10.2196/med20.2559 · Cited by 34 · see citing works
Toolkits for implementing and evaluating digital health: A systematic review of rigor and reporting
Godinho, M. A., Ansari, S., Guo, G. N., & Liaw, S. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocab010 · Cited by 32 · see citing works
Richardson, J. E., Abramson, E. L., Pfoh, E. R., & Kaushal, R. (2012) · PubMed Central · Cited by 30 · see citing works
Petkovic, J., Welch, V. A., Jacob, M. H., Yoganathan, M., Ayala, A. P., Cunningham, H., & Tugwell, P. (2018) · Campbell Systematic Reviews · doi:10.4073/csr.2018.8 · Cited by 21 · see citing works
Dannapfel, P., Thomas, K., Chakhunashvili, A., Melin, J., & Lagerros, Y. T. (2021) · JMIR Formative Research · doi:10.2196/17568 · Cited by 19 · see citing works
Abdelaziz, S., Garfield, S., Neves, A. L., Lloyd, J., Norton, J., van Dael, J., Wheeler, C., McLeod, M., & Franklin, B. D. (2024) · BMJ Open · doi:10.1136/bmjopen-2024-089026 · Cited by 19 · see citing works
Weimar, S. N., Martjan, R. S., & Terzidis, O. (2024) · Management Review Quarterly · doi:10.1007/s11301-024-00417-0 · Cited by 19 · see citing works
Digital Health Systems in Integrated Care
Gray, C. S., Gagnon, D., Guldemond, N., & Kenealy, T. W. (2021) · Springer eBooks · doi:10.1007/978-3-030-69262-9_28 · Cited by 9 · see citing works
Validity and Reliability of the eHealth Analysis and Steering Instrument
Henkemans, O. B., Dusseldorp, E., Keijsers, J. F. E. M., Kessens, J. M., Neerincx, M. A., & Otten, W. (2013) · Medicine 2 0 · doi:10.2196/med20.2571 · Cited by 8 · see citing works
Collombon, E. H. G. M., Bolman, C. A. W., Peels, D. A., de Bruijn, G., de Groot, R. H. M., & Lechner, L. (2022) · JMIR Research Protocols · doi:10.2196/31677 · Cited by 7 · see citing works
Lee, L., Cresswell, K. M., Slee, A., Slight, S. P., Coleman, J. J., & Sheikh, A. (2014) · JRSM Open · doi:10.1177/2054270414551658 · Cited by 7 · see citing works
Understanding Stakeholder Interests and Perspectives in Evaluations of Health IT
Lee, L., & Sheikh, A. (2016) · Studies in health technology and informatics · doi:10.3233/978-1-61499-635-4-53 · Cited by 7 · see citing works
[Advances in eHealth in Colombia: adoption of the National Cancer Information System].
Rivillas, J. C., Huertas-Quintero, J. A., Montaño-Caicedo, J. I., & Martinez, M. L. (2014) · PubMed · Cited by 5 · see citing works
The many faces of implementation
Vimarlund, V., & Keller, C. (2014) · Cited by 4 · see citing works
Digital Health Enabling Integrated Care
Gray, C. S., Gagnon, D., Guldemond, N., & Kenealy, T. W. (2021) · Emerald Group Publishing Limited eBooks · doi:10.1108/978-1-83867-527-120211012 · Cited by 3 · see citing works
Changing Places: The Need to Alter the Start Point for Information Security Design
Coles-Kemp, L., & Williams, P. A. H. (2014) · Australasian Journal of Paramedicine · Cited by 2 · see citing works
Crossing Boundaries : Transferring eHealth services across the Northern Periphery
Pesola, U.-M. (2013) · KTH Publication Database DiVA (KTH Royal Institute of Technology) · Cited by 2 · see citing works
Self-citationColes-Kemp, E., & Williams, P. A. H. (2012) · Australasian Journal of Paramedicine · doi:10.4225/75/5795b34e3cf0e · Cited by 1 · see citing works
Johansson, A. M. (2015) · Epubl LTU · Cited by 1 · see citing works
Cornelis, G. W., Ramon, S. F., & Fabienne, A. (2015) · RePEc: Research Papers in Economics · doi:10.2791/31536
Kłak, A., Raciborski, F., & Samolinski, B. K. (2018) · Polish Annals of Medicine · doi:10.29089/2017.17.00015
Kalsy, M., Sward, K. A., Bray, B. E., Redd, A. M., & Eilbeck, K. L. (2019) · Annals of Circulation · doi:10.17352/ac.000014
Digital Health Systems in Integrated Care
Gray, C. S., Gagnon, D., Guldemond, N., & Kenealy, T. W. (2025) · Springer eBooks · doi:10.1007/978-3-031-96286-8_28 · Cited by 1 · see citing works
Cohen, C. (2014) · The University of Bath Online Publications Store (The University of Bath)
Digital Health Systems in Integrated Care
Gray, C. S., Gagnon, D., Guldemond, N., & Kenealy, T. (2024) · Springer eBooks · doi:10.1007/978-3-031-25376-8_28-1
- Authors
- MacFarlane, A., Clerkin, P., Murray, E., Heaney, D. J., Wakeling, M., Pesola, U.-M., Waterworth, E. L., Larsen, F., Makiniemi, M., & Winblad, I.
- Affiliations
- University of Limerick (IE)
- Ollscoil na Gaillimhe – University of Galway (IE)
- The Royal Free Hospital (GB)
- University of Aberdeen (GB)
- Umeå University (SE)
- University Hospital of North Norway (NO)
- University of Oulu (FI)
- Corresponding author
- MacFarlane, Anne
- Venue
- Implementation Science
- Published
- 2011-11-19
- Volume
- 6
- Issue
- 1
- Pages
- 122-122
- ISSN
- 1748-5908 (ISSN-L)
- PMC
- PMC3283514
- OpenAlex
- W2135197859
- Cited by
- 34
- References
- 34
- Citations by year
- 2025: 1 · 2024: 3 · 2022: 1 · 2021: 4 · 2020: 1 · 2019: 1 · 2018: 3 · 2017: 1 · 2016: 2 · 2015: 3 · 2014: 6 · 2013: 4 · 2012: 4
- FWCI
- 8.32
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Implementation Science (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Arrow@dit (Dublin Institute of Technology) (acceptedVersion)
- Arrow@dit (Dublin Institute of Technology) (submittedVersion)
- KTH Publication Database DiVA (KTH Royal Institute of Technology) (submittedVersion, other-oa)
- ARAN (University of Galway Research Repository) (Ollscoil na Gaillimhe – University of Galway) (submittedVersion, cc-by-nc-nd)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- UCL Discovery (University College London) (submittedVersion)
- University of Limerick Institutional Repository (University of Limerick) (submittedVersion, cc-by-nc-sa)
- Lenus, The Irish Health Repository (Dr Steevens Hospital Library) (submittedVersion)
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Telemedicine and Telehealth Implementation
- Keywords
- eHealth implementation, research-practice gap, implementation barriers and facilitators, implementation monitoring, user experience
- MeSH terms
- Decision Making, Delivery of Health Care, Europe, Evaluation Studies as Topic, Finland, Health Plan Implementation, Humans, Interviews as Topic, Medical Informatics Applications, Norway, Pilot Projects, Scotland, Sweden, Program Evaluation, Internet
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
