Development and formative evaluation of the e-Health Implementation Toolkit (e-HIT)
Open accessAbstract
BACKGROUND: The use of Information and Communication Technology (ICT) or e-Health is seen as essential for a modern, cost-effective health service. However, there are well documented problems with implementation of e-Health initiatives, despite the existence of a great deal of research into how best to implement e-Health (an example of the gap between research and practice). This paper reports on the development and formative evaluation of an e-Health Implementation Toolkit (e-HIT) which aims to summarise and synthesise new and existing research on implementation of e-Health initiatives, and present it to senior managers in a user-friendly format. RESULTS: The content of the e-HIT was derived by combining data from a systematic review of reviews of barriers and facilitators to implementation of e-Health initiatives with qualitative data derived from interviews of "implementers", that is people who had been charged with implementing an e-Health initiative. These data were summarised, synthesised and combined with the constructs from the Normalisation Process Model. The software for the toolkit was developed by a commercial company (RocketScience). Formative evaluation was undertaken by obtaining user feedback. There are three components to the toolkit--a section on background and instructions for use aimed at novice users; the toolkit itself; and the report generated by completing the toolkit. It is available to download from http://www.ucl.ac.uk/pcph/research/ehealth/documents/e-HIT.xls. CONCLUSIONS: The e-HIT shows potential as a tool for enhancing future e-Health implementations. Further work is needed to make it fully web-enabled, and to determine its predictive potential for future implementations.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The e-HIT tool was developed in previous research by members of the team and is designed to help promote understanding of digital health implementation issues.
“This tool was developed in previous research by members of the team [28] and is designed to help promote understanding of digital health implementation issues.”
The eHealth Implementation Toolkit (e-HIT) is aimed specifically at eHealth interventions and has been developed for use by managers and other staff charged with implementing eHealth initiatives.
“A similar toolkit, the eHealth Implementation Toolkit (e-HIT), is aimed specifically at eHealth interventions and has been developed for use by managers and other staff charged with implementing eHealth initiatives (ie, implementers) [120-122].”
From Web-Based Interventions for Behavior Change and Self-Management: Potential, Pitfalls, and Progress (2012) · cites this paper as [121]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Older Adults' Reasons for Using Technology while Aging in Place
Peek, S. T. M., Luijkx, K. G., Rijnaard, M. D., Nieboer, M. E., van der Voort, C. S., Aarts, S., van Hoof, J., Vrijhoef, H. J., & Wouters, E. (2015) · Gerontology · doi:10.1159/000430949 · Cited by 19,803 · see citing works
Developing and evaluating complex interventions: the new Medical Research Council guidance
Craig, P., Dieppe, P., Macintyre, S., Michie, S., Nazareth, I., & Petticrew, M. (2008) · BMJ · doi:10.1136/bmj.a1655 · Cited by 10,995 · see citing works
Developing and Evaluating Complex Interventions: New Guidance
Craig, P., Dieppe, P. A., MacIntyre, S., Michie, S., Nazareth, I., & Petticrew, M. P. (2006) · UCL Discovery (University College London) · Cited by 1,877 · see citing works
Designing and evaluating complex interventions to improve health care
Campbell, N. C., Murray, E., Darbyshire, J., Emery, J., Farmer, A., Griffiths, F., Guthrie, B., Lester, H., Wilson, P., & Kinmonth, A. L. (2007) · BMJ · doi:10.1136/bmj.39108.379965.be · Cited by 1,331 · see citing works
Self-citationKitson, A. L., Rycroft-Malone, J., Harvey, G., McCormack, B., Seers, K., & Titchen, A. (2008) · Implementation Science · doi:10.1186/1748-5908-3-1 · Cited by 1,314 · see citing works
Electronic health records in ambulatory care—a national survey of physicians
DesRoches, C. M., Campbell, E. G., Rao, S. R., Donelan, K., Ferris, T. G., Jha, A., Kaushal, R., Levy, D. E., Rosenbaum, S., Shields, A. E., et al. (2008) · New England Journal of Medicine · doi:10.1056/nejmsa0802005 · Cited by 1,111 · see citing works
What Is eHealth (3): A Systematic Review of Published Definitions
Oh, H., Jadad, A., Rizo, C., Enkin, M., Powell, J., & Pagliari, C. (2005) · Journal of Medical Internet Research · doi:10.2196/jmir.7.1.e1 · Cited by 1,015 · see citing works
Helfrich, C. D., Li, Y.-F., Sharp, N. D., & Sales, A. E. (2009) · Implementation Science · doi:10.1186/1748-5908-4-38 · Cited by 564 · see citing works
A rational model for assessing and evaluating complex interventions in health care
May, C. (2006) · BMC Health Services Research · doi:10.1186/1472-6963-6-86 · Cited by 447 · see citing works
Self-citationDesigning theoretically-informed implementation interventions
The Improved Clinical Effectiveness through Behavioural Research Group (ICEBeRG) (2006) · Implementation Science · doi:10.1186/1748-5908-1-4 · Cited by 380 · see citing works
Evidence for public health policy on inequalities: 1: The reality according to policymakers
Petticrew, M. P., Whitehead, M., Macintyre, S. J., Graham, H., & Egan, M. (2004) · Journal of Epidemiology & Community Health · doi:10.1136/jech.2003.015289 · Cited by 380 · see citing works
(2004) · UCL Discovery (University College London) · Cited by 246 · see citing works
Gagnon, M.-P., Légaré, F., Labrecque, M., Frémont, P., Pluye, P., Gagnon, J., Car, J., Pagliari, C., Desmartis, M., Turcot, L., et al. (2009) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006093.pub2 · Cited by 157 · see citing works
Health technology assessment in its local contexts: studies of telehealthcare
May, C., Mort, M., Williams, T., Mair, F., & Gask, L. (2003) · Social Science & Medicine · doi:10.1016/s0277-9536(02)00419-7 · Cited by 147 · cited 1× · see citing works
“It was initially derived from an extensive series of qualitative studies of telemedicine initiatives [18], and subsequently refined and expanded to apply to the implementation of all complex interventions in health care.”
BackgroundSelf-citationPublic Health Decision‐Makers' Informational Needs and Preferences for Receiving Research Evidence
Dobbins, M., Jack, S. M., Thomas, H., & Kothari, A. R. (2007) · Worldviews on Evidence-Based Nursing · doi:10.1111/j.1741-6787.2007.00089.x · Cited by 142 · see citing works
Implementation research: state of the art and future directions
Linton, J. D. (2002) · Technovation · doi:10.1016/s0166-4972(01)00075-x · Cited by 126 · see citing works
The relationship between busyness and research utilization: it is about time
Thompson, D. S., O’Leary, K., Jensen, E. N., Scott, S. D., O’Brien‐Pallas, L., & Estabrooks, C. A. (2008) · Journal of Clinical Nursing · doi:10.1111/j.1365-2702.2007.01981.x · Cited by 113 · see citing works
Understanding the implementation and integration of e-health services
Mair, F., May, C., Finch, T., Murray, E., Anderson, G. K., Sullivan, F., O'Donnell, C., Wallace, P., & Epstein, O. (2007) · Journal of Telemedicine and Telecare · doi:10.1258/135763307781645112 · Cited by 105 · see citing works
Self-citationImplementing the NHS information technology programme: qualitative study of progress in acute trusts
Hendy, J., Fulop, N., Reeves, B. C., Hutchings, A., & Collin, S. M. (2007) · BMJ · doi:10.1136/bmj.39195.598461.551 · Cited by 77 · see citing works
The National Programme for IT in the NHS.
Hassey, A. (2005) · PubMed · Cited by 63 · see citing works
Pharmacologic Management of Atrial Fibrillation: Established and Emerging Options
Kalus, J. S. (2009) · Journal of Managed Care Pharmacy · doi:10.18553/jmcp.2009.15.s6-b.10 · Cited by 57 · see citing works
(2003) · UCL Discovery (University College London) · Cited by 40 · see citing works
Health Information Technology—Results From a Roundtable Discussion
Balfour, D. C., Evans, S. W., Januska, J., Lee, H. Y., Lewis, S. J., Nolan, S. R., Noga, M., Stemple, C., & Thapar, K. (2009) · Journal of Managed Care Pharmacy · doi:10.18553/jmcp.2009.15.s1.10 · Cited by 35 · see citing works
the ReBEQI group, Akl, E. A., Treweek, S. P., Foy, R., Francis, J., & Oxman, A. D. (2007) · Implementation Science · doi:10.1186/1748-5908-2-19 · Cited by 21 · see citing works
Cross, M. (2007) · BMJ · doi:10.1136/bmj.39259.445035.34 · Cited by 6 · see citing works
The Telemedicine Tool Kit: A Workbook for NHS Doctors, Nurses and Managers
Lilley, R. (1999) · Cited by 2 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
May, C. R., Cummings, A., Girling, M., Bracher, M., Mair, F. S., May, C. M., Murray, E., Myall, M., Rapley, T., & Finch, T. (2018) · Implementation Science · doi:10.1186/s13012-018-0758-1 · Cited by 683 · see citing works
Self-citationMcEvoy, R., Ballini, L., Maltoni, S., O’Donnell, C. A., Mair, F. S., & MacFarlane, A. (2014) · Implementation Science · doi:10.1186/1748-5908-9-2 · Cited by 413 · see citing works
Self-citationFinch, T. L., Girling, M., May, C. R., Mair, F. S., Murray, E., Treweek, S., McColl, E., Steen, I. N., Cook, C., Vernazza, C. R., et al. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0591-x · Cited by 337 · see citing works
Self-citationLennon, M. R., Bouamrane, M.-M., Devlin, A. M., O'Connor, S., O'Donnell, C., Chetty, U., Agbakoba, R., Bikker, A., Grieve, E., Finch, T., et al. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.6900 · Cited by 264 · see citing works
Self-citationWeb-Based Interventions for Behavior Change and Self-Management: Potential, Pitfalls, and Progress
Murray, E. (2012) · Medicine 2.0 · doi:10.2196/med20.1741 · Cited by 219 · see citing works
Self-citationRapley, T., Girling, M., Mair, F. S., Murray, E., Treweek, S., McColl, E., Steen, I. N., May, C. R., & Finch, T. L. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0590-y · Cited by 193 · see citing works
Self-citationFinch, T. L., Rapley, T., Girling, M., Mair, F. S., Murray, E., Treweek, S., McColl, E., Steen, I. N., & May, C. R. (2013) · Implementation Science · doi:10.1186/1748-5908-8-43 · Cited by 177 · see citing works
Self-citationFinch, T. L., Mair, F. S., O’Donnell, C., Murray, E., & May, C. R. (2012) · BMC Medical Research Methodology · doi:10.1186/1471-2288-12-69 · Cited by 127 · see citing works
Self-citationPatient-focused Internet interventions in reproductive medicine: a scoping review
Aarts, J. W. M., van den Haak, P., Nelen, W. L. D. M., Tuil, W. S., Faber, M. J., & Kremer, J. A. M. (2012) · Human Reproduction Update · doi:10.1093/humupd/dmr045 · Cited by 52 · see citing works
Nordin, C. A., Michaelson, P., Gard, G., & Eriksson, M. K. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.5634 · Cited by 38 · see citing works
The e-health implementation toolkit: qualitative evaluation across four European countries
MacFarlane, A., Clerkin, P., Murray, E., Heaney, D. J., Wakeling, M., Pesola, U.-M., Waterworth, E. L., Larsen, F., Makiniemi, M., & Winblad, I. (2011) · Implementation Science · doi:10.1186/1748-5908-6-122 · Cited by 34 · see citing works
Self-citationMay, C. R., Finch, T., Ballini, L., MacFarlane, A., Mair, F., Murray, E., Treweek, S., & Rapley, T. (2011) · BMC Health Services Research · doi:10.1186/1472-6963-11-245 · Cited by 3 · see citing works
Self-citation
- Authors
- Murray, E., May, C., & Mair, F.
- Affiliations
- The Royal Free Hospital (GB)
- University of Southampton (GB)
- University of Glasgow (GB)
- Corresponding author
- Murray, Elizabeth
- Venue
- BMC Medical Informatics and Decision Making
- Published
- 2010-10-18
- Volume
- 10
- Issue
- 1
- Pages
- 61-61
- ISSN
- 1472-6947 (ISSN-L)
- PMC
- PMC2967499
- OpenAlex
- W2061328880
- Cited by
- 96
- References
- 26
- Citations by year
- 2026: 4 · 2025: 3 · 2024: 6 · 2023: 4 · 2022: 5 · 2021: 16 · 2020: 4 · 2019: 9 · 2018: 8 · 2017: 5 · 2016: 4 · 2015: 5 · 2014: 7 · 2013: 7 · 2012: 5
- FWCI
- 9.56
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- BMC Medical Informatics and Decision Making (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- ENLIGHTEN (Jurnal Bimbingan dan Konseling Islam) (acceptedVersion)
- LSHTM Research Online (London School of Hygiene and Tropical Medicine) (acceptedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- UCL Discovery (University College London) (submittedVersion)
- Funders
- National Institute for Health and Care Research
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Telemedicine and Telehealth Implementation
- Keywords
- eHealth implementation, implementation toolkit, barriers and facilitators, health managers, research-practice gap, user feedback
- MeSH terms
- Administrative Personnel, United Kingdom, Health Plan Implementation, Humans, Interviews as Topic, Medical Informatics Applications, Software, User-Computer Interface, Private Sector, Telemedicine, Guidelines as Topic, Internet
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
