Readiness for Delivering Digital Health at Scale: Lessons From a Longitudinal Qualitative Evaluation of a National Digital Health Innovation Program in the United Kingdom

May combine two claims.

Obtaining accreditation for the eRedBook was labor and time intensive and posed a barrier for implementation.

“This was because endorsement by the relevant medical association was seen as a prerequisite to ensure uptake by lay users and health professionals; obtaining such accreditation was labor and time intensive and posed a barrier for implementation but once accreditation or endorsement was achieved, this was perceived as a potential facilitator of uptake.”

May combine two claims.

Companies often wanted to sell a technology kit, whereas the statutory sector wanted a contract linking the use of technology to data services and outcomes.

“Companies often wanted to sell a ‟technology kit,” whereas the statutory sector wanted a different type of ‟contract,” linking the use of technology to data services and outcomes.”

May combine two claims.

Commercial companies often perceive open standards and interoperability as a threat to their business model, because interoperability was not a priority compared with increasing their own individual market share.

“Commercial companies often perceive open standards and interoperability as a threat to their business model, since it was not a priority in comparison with efforts to increase their own individual market share as the market develops and matures.”

May combine two claims.

Companies were reluctant to take on risk until it was clear what the ethical guidelines and responsibilities were around collection and use of lifestyle data, and this was another barrier to implementation.

“This was a risk which companies were reluctant to take on in the current market until it was clear what the ethical guidelines and responsibilities were around collection and use of lifestyle data and was another barrier to implementation.”

May combine two claims.

Although policy and legislation relating to data sharing has been reviewed and clear recommendations made, this has not yet translated to local contexts and was reported as a key barrier to deployment of the dallas program.

“Although policy and legislation relating to data sharing has been reviewed and clear recommendations made [36], this has not yet translated to local contexts and was reported as a key barrier to deployment of the dallas program across its lifetime.”

• cites [36] HealthDGov.UK20132017-02-03Caldicott Review: information governance in health and care system, Information: To share or not to share? https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/192572/2900774_InfoGovernance_accv2.pdfNot checked yet

  Not checked in the free run.

May combine two claims.

The UK digital health market is challenging to navigate, with a lack of clear access or entry points for the retail sectors.

“All of this makes the UK digital health market challenging to navigate, with a lack of clear access or entry points for the retail sectors.”

May combine two claims.

Accessing Twitter or similar sites using hospital computer systems was often not permissible, which meant that integrating social networking platforms within the health care arena was problematic.

“For example, accessing ‟Twitter” or similar sites using hospital computer systems was often not permissible, which meant that integrating social networking platforms within the health care arena was problematic.”

Classified as probably not a substantial claim.

There was a common view that information governance regulations were not currently fit for purpose.

“This led to a common view that information governance regulations were not currently ‟fit for purpose.””

May combine two claims.

Providing self-monitoring devices as part of health promotion was deemed acceptable, but providing clinical data back to those with known health problems was perceived as a different proposition involving significant risk.

“Providing self-monitoring devices as part of health promotion was deemed acceptable but providing clinical data back to those with known health problems was perceived as a different proposition, involving significant risk.”

May combine two claims.

Recent social benefits reforms encouraged many individuals to improve their computer literacy by joining Lay and Digital champion programs and to engage with digital hubs so they could access their social welfare benefits.

“This encouraged many individuals to improve their computer literacy by joining ‟Lay” and ‟Digital champion” programs and to engage with digital hubs so they could access their social welfare benefits.”

May combine two claims.

As a result of digital enablement activities, disadvantaged groups potentially at risk of digital exclusion were provided with digital skills and educated about health technologies.

“As a result, disadvantaged groups potentially at risk of digital exclusion were provided with digital skills and educated about health technologies.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Recent social benefits reforms compelled people to sign on the Web.

“For example, the experience from digital enablement activities suggested that recent social benefits reforms compelled people ‟to sign on the Web.””

May combine two claims.

The time and effort required for codesign created real challenges, introduced delays, and reduced the time available to develop and deploy new solutions within the defined timescale of the program.

“However, the time and effort required for this created real challenges, as it introduced delays and consequently reduced the time available to develop and deploy new solutions within the defined timescale of the program.”

May combine two claims.

Each new implementation site had to go through its own deployment pain-barrier, with ad-hoc local solutions providing little insight on what to expect elsewhere.

“Practically this meant that each new implementation site had to go through its own deployment ‟pain-barrier,” with ad-hoc local solutions providing little insight on what to expect elsewhere.”

May combine two claims.

Workload pressures and lack of capacity was a recurring barrier to incorporating new technologies into everyday working practices, and incentivization had to be used to overcome this on occasion.

“Workload pressures and lack of capacity was a recurring barrier to incorporating new technologies into everyday working practices and on occasion incentivization had to be used to overcome this.”

May combine two claims.

A lack of senior management buy-in was cited as a barrier to organizational readiness for digital health, which could negatively affect implementation efforts.

“a lack of senior management buy-in was also cited as a barrier to organizational readiness for digital health, which could negatively affect implementation efforts.”

May combine two claims.

The target market ranged from younger consumers who were more adept, confident, and ready to use digital tools through to some older adults with little or no previous awareness or understanding of basic IT.

“The target market ranged from younger consumers—who were more adept, confident, and ready to use digital tools—through to some older adults with little or no previous awareness or understanding of basic IT.”

May combine two claims.

The issue of individual agency arose as people lead busy lives with competing career and family responsibilities, as well as financial and social demands, which were often prioritized ahead of health.

“The issue of individual ‟agency” arose as people lead busy lives with competing career and family responsibilities, as well as—for some—financial and social demands, which were often prioritized ahead of health.”

May combine two claims.

Smaller organizations such as consortia of primary care practices were able to be more flexible and more responsive to opportunities presented by dallas communities.

“Thus, smaller organizations such as consortia of primary care practices were able to be more flexible and more responsive to opportunities presented by dallas communities.”

May combine two claims.

Problems at the executive level within larger health trusts contrasted with readiness of smaller, more flexible organizations such as general practices, which seemed more receptive to adopting digital health services to large patient groups.

“Problems at the executive level within larger health trusts were contrasted with readiness of smaller, more flexible organizations such as general practices, which seemed more receptive to adopting digital health services to large patient groups.”

May combine two claims.

Despite reported growing use of digital tablets and smartphones among the general population, many people still lacked basic access to such mobile devices.

“Despite reported growing use of digital tablets and smartphones among the general population, many people still lacked basic access to such mobile devices.”

May combine two claims.

The main barrier was not a fear of role replacement, but lack of knowledge and skills in relation to digital health, which significantly affected how prepared some were to engage with the different technologies.

“However, the main barrier was not a fear of role replacement, but lack of knowledge and skills in relation to digital health which significantly affected how prepared some were to engage with the different technologies.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Some nurses worried that personal information may not be kept confidential and secure or could be sold or shared with commercial companies without the explicit consent or knowledge of patients.

“Some nurses worried that personal information may not be kept confidential and secure or could be sold or shared with commercial companies without the explicit consent or knowledge of patients.”

May combine two claims.

Interoperability problems limited the effectiveness of many new digital platforms and required workarounds by health care staff and end users to ensure benefits could still be exploited locally.

“This limited the effectiveness of many of the new digital platforms and required workarounds by health care staff and end users to ensure the benefits could still be exploited locally.”

May combine two claims.

Some health organizations had not yet fully developed their own digital strategies and/or had not yet bought in fully to the self-care agenda at the time of the program.

“Some health organizations had not yet fully developed their own digital strategies and/or had not yet bought in fully to the self-care agenda at the time of the program.”

Probably not a substantial claim: may not pass S3 (load-bearing).

There was a perception by those tasked with driving forward new initiatives that clinicians and others feared digital health as they believed it could be used to disempower and in some cases replace them as care providers.

“There was a perception by those tasked with driving forward new initiatives that clinicians and others feared digital health as they believed it could be used to disempower and in some cases replace them as care providers.”

May combine two claims.

Budget constraints were visible among some partners who struggled with manpower capacity and to provide mobile technology and other equipment for their health care staff to roll out digital health products and services, which was a major impediment to progressing the digital health consumer agenda.

“Budget constraints were clearly visible among some partners who struggled with manpower capacity and to provide mobile technology and other equipment for their health care staff to roll out different digital health products and services, which was a major impediment to progressing the digital health ‟consumer” agenda.”

May combine two claims.

Given that many digital health initiatives have come and gone over the years and failed to be integrated into routine care in the UK health service, some health professionals were skeptical about the likely longevity of the dallas program and were slow to engage with the technologies it was promoting.

“Also, given that many digital health initiatives have come and gone over the years and failed to be integrated into routine care in the UK health service, some health professionals were skeptical about the likely longevity of the dallas program and were slow to engage with the technologies it was promoting.”

May combine two claims.

The health service and other organizations frequently had to contend with major resource constraints during the current period of financial austerity, which affected ability to engage with various initiatives within dallas.

“The health service and other organizations frequently had to contend with major resource constraints during the current period of financial austerity, which affected ability to engage with various initiatives within dallas.”

May combine two claims.

The pace of technological change was noted as a problem as clinicians felt unable to keep up to date with new and emerging developments, especially the growth and lack of regulation within the mobile app market.

“The pace of technological change was also noted as a problem as clinicians felt unable to keep up to date with new and emerging developments, especially the growth and lack of regulation within the mobile app market.”

May combine two claims.

Technologies which required data entry and/or a change in daily routines affected the readiness of users to adopt these and proved another challenge for the preventive self-care agenda.

“Technologies which required data entry and/or a change in daily routines affected the ‟readiness” of users to adopt these and this proved another challenge for the preventive self-care agenda.”

May combine two claims.

The dallas program highlighted challenges but also potential solutions to the large scale implementation of digital health, for example, through the development of information governance recommendations for health care organizations and the use of digital champions to address skill deficiencies.

“The dallas program highlighted challenges but also potential solutions to the large scale implementation of digital health, for example, through the development of information governance recommendations for health care organizations [45] and the use of digital champions to address skill deficiencies.”

• cites [45] ChittyAHSJ201506222017-02-03Health Services JournalMake people the controllers of their data to help the NHS go digital http://www.hsj.co.uk/5086634.article#.VYkov2jF99dNot checked yet

  Not checked in the free run.

May combine two claims.

Our work suggests that the health care community would welcome better integration of health records, although persistent challenges are posed by the way current information governance rules are interpreted and enacted.

“Our work suggests that the health care community would welcome better integration of health records, although persistent challenges are posed by the way current information governance rules are interpreted and enacted.”

Classified as probably not a substantial claim. May combine two claims.

Although there is much rhetoric about the consumer push for digital health, our research raises outstanding issues relating to the readiness for digital health that need attention.

“Although there is much rhetoric about the consumer push for digital health, our research raises some outstanding issues relating to the readiness for digital health that need attention.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

The dallas program has shown that although such issues persist across a range of digital health initiatives, more user centered design techniques, intensive engagement, and support of users and incentivization of professionals can help increase interest in digital health.

“Importantly the dallas program has shown that although such issues persist across a range of digital health initiatives more user centered design techniques, intensive engagement, and support of users and incentivization of professionals can help increase interest in digital health.”

May combine two claims.

Uptake and adoption of digital health are slow overall, and sociotechnical, organizational, or cultural barriers slow the mainstreaming of digital health.

“Despite positive evidence from localized implementations, new technologies have proven slow to become accepted, integrated, and routinized at scale.”

• Also stated in: § Introduction · hedged

  “Previous research, however, has shown that uptake and adoption are slow for digital health overall and that there may be many sociotechnical, organizational, or cultural barriers that are slowing the mainstreaming of digital health [1,2].”

• cites [1] Jennett, PA, 2001, Comput Methods Programs Biomed sourceNot checked yet

  Not checked in the free run.

• cites [2] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Existing evidence on eHealth readiness mainly relates to organizational readiness within the health service or the readiness of health professionals, patients, or carers for a specific single digital health service.

“However, the existing evidence mainly relates to organizational readiness within the health service [17,18] or the readiness of health professionals [19-21], patients or carers for a specific single digital health service [22].”

• cites [17] May, C, 2003, Soc Sci Med sourceNot checked yet

  Not checked in the free run.

• cites [18] sourceNot checked yet

  Not checked in the free run.

• cites [19] sourceNot checked yet

  Not checked in the free run.

• cites [20] sourceNot checked yet

  Not checked in the free run.

• cites [21] Greenhalgh, T, 2014, Health Services and Delivery Research sourceNot checked yet

  Not checked in the free run.

• cites [22] Greenhalgh, T, 2010, Br Med J sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Prior literature identifies barriers to digital health implementation including liability concerns, interoperability issues, costs, usability, misaligned incentives, policy problems, and acceptability to patients and professionals.

“Issues that have been identified as barriers to implementation range from liability concerns, interoperability issues, costs, usability, misaligned incentives, and policy problems through to acceptability to patients and professionals [4,8].”

• cites [4] sourceNot checked yet

  Not checked in the free run.

• cites [8] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Although there is receptiveness to digital health, barriers to mainstreaming remain.

“Although there is receptiveness to digital health, barriers to mainstreaming remain.”

• AssumptionRests on an assumption: The dallas program and its findings are sufficiently representative of national digital health innovation ecosystems to support lessons about readiness for digital health at scale beyond the program.
• AssumptionRests on an assumption: The barriers and facilitators identified in the dallas program are relevant to digital health implementation internationally.

Probably not a substantial claim: may not pass S3 (load-bearing).

Anonymity was protected by use of generic descriptors throughout.

“Anonymity was protected by use of generic descriptors throughout.”

Classified as probably not a substantial claim. May combine two claims.

Care was taken to include representatives from all types of organizations involved (private, public, voluntary) and stakeholders at strategic, project management, and service design and delivery levels.

“Care was taken to include representatives from all types of organizations involved (private, public, voluntary) and to include stakeholders involved at the strategic level, the project management level, and the service design and delivery level.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Eight researchers with multidisciplinary experience were involved at coding clinics, with one or two senior academics involved in all coding sessions.

“A total of 8 researchers with multidisciplinary experience in health services research, clinical research, informatics, and social science were involved at coding clinics with 1 or 2 senior academics (FSM, MML) involved in all coding sessions.”

Probably not a substantial claim: may not pass S3 (load-bearing).

University ethical approvals were granted for the collection of all qualitative data reported.

“University ethical approvals were granted for the collection of all qualitative data reported”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The e-HIT tool was developed in previous research by members of the team and is designed to help promote understanding of digital health implementation issues.

“This tool was developed in previous research by members of the team [28] and is designed to help promote understanding of digital health implementation issues.”

• cites [28] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Informed consent to participation was gained from all participants.

“Informed consent to participation was gained from all participants.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The work was overseen by an external advisory group.

“Our work was overseen by an external advisory group.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Coding clinics were undertaken to ensure consistency of coding and shared understanding of coding constructs.

“Coding clinics were undertaken to ensure consistency of coding and shared understanding of coding constructs.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Care was taken to expand, collapse, and rename codes so that all were confident the coding book was a fair and accurate reflection of the data.

“Care was taken to expand, collapse, and rename codes so that all were confident that the coding book was a fair and accurate reflection of the data.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Interviews and focus groups were audio-recorded with participant consent, transcribed verbatim, and anonymized.

“Interviews and focus groups were audio-recorded with participant consent, transcribed verbatim, and anonymized.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The efforts required to implement and manage changes are less complex and costly for smaller organizations.

“Naturally, the efforts required to implement and manage changes are also less complex and costly for smaller organizations”

May combine two claims.

The UK government has recognized the digital skills issue and vowed to make fast broadband available to every home, but our findings suggest this will still leave much to do.

“The UK government has recognized this issue and vowed to make fast broadband available to every home [39] but our findings suggest this will still leave much to do.”

• cites [39] Department of CultureGov.UK20132017-02-03Broadband Delivery UK https://www.gov.uk/guidance/broadband-delivery-ukNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Interoperability is a key aim in the United Kingdom as it strives to implement digital standards and achieve system wide interoperability, but others have reported interoperability as a barrier to implementation and large scale deployments of mHealth and global health interventions.

“Interoperability is a key aim in the United Kingdom as it strives to implement digital standards and achieve system wide interoperability but others have reported interoperability as a barrier to implementation and large scale deployments of mHealth and global health interventions [40,41].”

• cites [40] sourceNot checked yet

  Not checked in the free run.

• cites [41] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

A robust sociological theory-NPT-underpinned the evaluation as recommended good practice.

“A robust, sociological theory—NPT—underpinned the evaluation as recommended as good practice [32,33].”

• cites [32] sourceNot checked yet

  Not checked in the free run.

• cites [33] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Consumer adoption of digital health is seen as a great market opportunity with numerous policy drivers and yet penetration of this large potential market remains relatively poor.

“Consumer adoption of digital health is seen as a great market opportunity with numerous policy drivers and yet penetration of this large potential market remains relatively poor [37].”

• cites [37] Gov.UK2015092017-02-03Digital Health in the UK: An industry study for the Office of Life Sciences https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/461479/BIS-15-544-digital-health-in-the-uk-an-industry-study-for-the-Office-of-Life-Sciences.pdfNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Digital skill and infrastructure deficiencies have been noted by a recent select committee report on digital skills for 2014-2015 that examined challenges for a digital economy more broadly.

“Our findings resonate with reports and data from other sources, for example, digital skill and infrastructure deficiencies have been noted by a recent select committee report on digital skills for 2014-2015 [38] that examined challenges for a ‟digital economy” more broadly.”

• cites [38] Publications.parliament20152017-02-03Make or Break the UKs Digital Future https://www.publications.parliament.uk/pa/ld201415/ldselect/lddigital/111/111.pdfNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Clinical endorsement of digital health products and services including systems for regulation and accreditation of technology and data enabled services is required and has been suggested previously, and a recent systematic review has suggested this is a key issue.

“Clinical endorsement of digital health products and services including systems for regulation and accreditation of technology and data enabled services is required and has been suggested previously [42], and a recent systematic review of the international literature on barriers and facilitators to patient and public involvement with digital health has suggested this is a key issue [43].”

• cites [42] sourceNot checked yet

  Not checked in the free run.

• cites [43] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The research was located in 2 countries within the United Kingdom which operate a NHS system where health care is free at the point of access and there are funding constraints, which is important when considering the implications of this work.

“Our research was located in 2 countries within the United Kingdom which operate a NHS system where health care is free at the point of access and there are funding constraints which is important to note when considering the implications of this work.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Clinical endorsement and validation was seen as one way to address trust concerns and promote uptake and utilization among consumer groups.

“Clinical endorsement and validation was seen as one way to address this and promote uptake and utilization among consumer groups.”

May combine two claims.

The study rigorously collected and analyzed process data from a wide range of stakeholders involved in the implementation, identifying not only potential barriers, but also why these occur and how to address them in future.

“We have rigorously collected and analyzed process data from a wide range of stakeholders involved in the implementation, identifying not only potential barriers, but also why these occur and how to address them in future (Table 4).”

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

These readiness issues are not insurmountable challenges but their existence does need to be acknowledged and addressed if deployment at scale to the widest population is to be realized.

“These issues are not insurmountable challenges but their existence does need to be acknowledged and addressed if deployment at scale to the widest population is to be realized.”

• AssumptionRests on an assumption: There is a need to deploy digital health at scale to the widest population.

May combine two claims.

Our whole system view of the digital health ecosystem suggests that interoperability is not a technical issue but rather due to industry inertia and to multiple organizations operating within the health system in the United Kingdom.

“Our whole system view of the digital health ecosystem provides potential explanations, suggesting that interoperability is not a technical issue but rather due to industry inertia and to multiple organizations operating within the health system in the United Kingdom.”

May combine two claims.

Qualitative data collection was largely limited to those engaging with the dallas program; the study did not undertake work with individuals or organizations that were unable or unwilling to engage, which could have provided different perspectives and possibly shed light on digital exclusion or nonparticipation.

“Our qualitative data collection was largely limited to those engaging with the dallas program and we did not undertake work with individuals or organizations that were unable or unwilling to engage with the program, which could have provided different perspectives and possibly shed light on digital exclusion or nonparticipation.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The authors were rigorous in looking for data that fell outside the NPT framework, despite the risk of inappropriate shoe horning of data.

“This could result in inappropriate ‟shoe horning” of data, however, we would argue that we were rigorous in looking for data that also fell outside the framework.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The recent European Union (EU) Task Force on eHealth acknowledged the need to develop EU-wide standards on interoperability and data sharing.

“The recent European Union (EU) Task Force on eHealth also acknowledged the need to develop EU-wide standards on interoperability and data sharing [26].”

• cites [26] Ec.europa20142017-02-03LuxembourgEU Task Force on eHealthEU Task Force on eHealth: Redesigning health in Europe for 2020 https://ec.europa.eu/digital-single-market/en/news/eu-task-force-ehealth-redesigning-health-europe-2020Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Previous research relating to the use of personal electronic health records has demonstrated less public and professional appetite than anticipated.

“Previous research relating to the use of personal electronic health records has demonstrated less public and professional appetite than anticipated [22,44].”

• cites [22] Greenhalgh, T, 2010, Br Med J sourceNot checked yet

  Not checked in the free run.

• cites [44] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable).

If the 10 key recommendations are addressed, they will promote the right market and environment to permit the routine deployment and true scaling-up of digital health and wellness technologies and services.

“If addressed, these recommendations will promote the right market and environment to permit the routine deployment and true scaling-up of digital health and wellness technologies and services.”

• AssumptionRests on an assumption: There is a need to deploy digital health at scale to the widest population.