Spread, Scale-up, and Sustainability of Video Consulting in Health Care: Systematic Review and Synthesis Guided by the NASSS Framework

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Complexity was mentioned in a few articles, although not as a key organizing framework.

“Complexity was mentioned in a few articles, although not as a key organizing framework.”

May combine two claims.

The most common opportunities included availability of clinical and/or nonclinical telehealth champions or coordinators (n=8), provider acceptance (n=4), absence of billing or licensure restrictions (n=3), adequate funding (n=4), and strong interorganizational communications (n=4).

“The most common opportunities included the availability of clinical and/or nonclinical telehealth champions or coordinators (n=8), provider acceptance (n=4), absence of billing or licensure restrictions (n=3), adequate funding (n=4), and strong interorganizational communications (n=4).”

May combine two claims.

Of these studies, 6 paid limited attention to how clinical characteristics played a role in successful spread of video consulting services, although there was mention of provider concerns around using video consulting in specific conditions.

“Of these studies, 6 paid limited attention to how clinical characteristics played a role in the successful spread of video consulting services [32-35,37,42], although there was mention of provider concerns around using video consulting in specific conditions (eg, “for patients who are at high risk of suicide or who have psychotic symptoms”).”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

One study noted that for patients with agoraphobia, the video medium removed the need to travel to the clinic and allowed patients to more easily receive mental health support.

“One study noted that for patients with agoraphobia, the video medium removed the need to travel to the clinic and allowed patients to more easily receive mental health support [42].”

• cites [42] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Review using CASP and MMAT indicated included studies generally fulfilled relevant quality standards, although in a few articles data collection or analysis methods were unclear, sample sizes were small, or sampling frame information was inadequate; they remained highly descriptive or illustrative.

“A review of the included studies using the CASP and MMAT appraisal tools indicated that they generally fulfilled relevant quality standards, although in a few articles it was not clear how data collection or analysis methods were applied, sample sizes were small, or there was inadequate information on sampling frame; furthermore, they remained at a highly descriptive or even purely illustrative”

May combine two claims.

Not all articles explicitly reported opportunities and challenges: 10 (77%) reported both, 2 (15%) reported challenges exclusively, and 1 (8%) focused more on opportunities.

“Not all articles explicitly reported opportunities and challenges: 10 (77%) reported both challenges and opportunities [2,33-35,37-41,43], 2 (15%) reported challenges exclusively [32,36] and 1 (8%) article focused more on opportunities and how these could be translated to other settings [42].”

May combine two claims.

Another two articles employed frameworks to support spread and scale-up efforts, although they did not draw on relevant approaches such as the Going to Full Scale framework, 3S infrastructure, or the Dynamic Sustainability framework.

“Another two articles employed frameworks to support spread and scale-up efforts, although they did not draw on relevant approaches such as the Going to Full Scale framework, 3S infrastructure, or the Dynamic Sustainability framework [58-60].”

• cites [58] sourceNot checked yet

  Not checked in the free run.

• cites [59] sourceNot checked yet

  Not checked in the free run.

• cites [60] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

In Norway, infrastructural differences between regions challenged spread of telemedicine and meant the national eHealth strategy could not be implemented consistently.

“In Norway, infrastructural differences between the different regions have challenged the spread of telemedicine and meant that the national eHealth strategy could not be implemented consistently [36].”

• cites [36] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

The 13 papers referred to various video consultation programs but focused on their implementation process rather than providing details of the video conferencing technologies used.

“The 13 papers referred to various video consultation programs but focused on their implementation process rather than providing details of the video conferencing technologies used.”

May combine two claims.

Services covered rehabilitation, geriatrics, cancer surgery, diabetes, mental health, general specialist care, and primary care.

“Services covered various clinical specializations including rehabilitation [35,39], geriatrics [39], cancer surgery [2,38], diabetes [2,38], and mental health [34,37,40,42], in addition to general specialist care and primary care [32,33,36,41,43].”

• Also stated in: § Results · plain

  “Services covered rehabilitation, geriatrics, cancer surgery, diabetes, and mental health, as well as general specialist care and primary care.”

May combine two claims.

Papers described 10 different video consultation services in 6 regions: Australia, Cabo Verde, England, Nepal, Norway, and the United States.

“Papers described 10 different video consultation services in 6 regions: Australia [39], Cabo Verde [41], England [2,38], Nepal [32], Norway [36], and the United States (Multimedia Appendix 3).”

Probably not a substantial claim: may not pass S3 (load-bearing).

The included studies employed diverse research designs, including interpretive case studies, qualitative interviews, mixed methods, action research, prospective implementation studies, and retrospective case analysis.

“Diverse research designs were employed in the included studies, including interpretive case studies [32], structured or semistructured qualitative interviews [33-35], mixed methods combining qualitative and quantitative data [2,36-38], action research and deliberative methods [39], prospective implementation studies including quantitative activity and performance indicators [40-42], and”

May combine two claims.

The most common challenges were lack of technical telehealth-specific support for clinical staff (n=6), need for redevelopment of workflows and organizational routines (n=6), financial pressures (n=5), and lack of training (n=5).

“The most common challenges were lack of technical telehealth-specific support for clinical staff (n=6), need for redevelopment of workflows and organizational routines (n=6), financial pressures (n=5), and lack of training (n=5).”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Initial searches identified 4484 unique articles published between January 2010 and December 2018, with 193 articles for full-text screening and 12 initially included; the updated search in March 2020 returned 185 new articles, 5 full-text, one new relevant, bringing the total to 13.

“Initial searches identified 4484 unique articles published between January 2010 and December 2018, with 193 articles identified for full-text screening and 12 initially included in the final review. An updated search in March 2020 returned 185 new articles for screening, of which 5 were identified for full-text screening. One new relevant article was identified, bringing the total records included”

May combine two claims.

Social science theory was employed rarely in a highly theoretical mode.

“Social science theory was also employed, although rarely in a highly theoretical mode.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Preoperative cancer surgery was deemed too complex for video consulting given necessity of a physical exam and likely no pre-existing relationship between patient and clinical team, but postoperative follow-up within the same clinic was deemed more appropriate.

“Preoperative cancer surgery was deemed too complex for video consulting given the necessity of a physical exam combined with there likely being no pre-existing relationship between the patient and clinical team, but postoperative follow-up within the same clinic was deemed more appropriate [2,38].”

• cites [2] Shaw, S, 2018, Advantages and limitations of virtual online consultations in a NHS acute trust: the VOCAL mixed-methods studyNot checked yet

  Not checked in the free run.

• cites [38] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

There was ambiguity in the use and variations of the term telehealth; three studies gave no definition and others defined variably; 11 articles (85%) used the terms "video telehealth," "telehealth," "telemedicine," or "telecare," while two used "video consultations".

“There was ambiguity in the use and variations of the term telehealth, with three studies giving no definition and others defining telehealth variably depending on study focus (Multimedia Appendix 3). The terms telehealth, telemedicine, and telemental health were common and often used interchangeably: 11 articles (85%) used the terms “video telehealth,” “telehealth,” “telemedicine,” or “telecare,””

• cites [2] Shaw, S, 2018, Advantages and limitations of virtual online consultations in a NHS acute trust: the VOCAL mixed-methods studyNot checked yet

  Not checked in the free run.

• cites [38] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Reported patient and caregiver challenges/opportunities included patient trust and acceptance of video consulting, patient's ability to transport themselves to a hub site, and patients requiring appropriate technology to participate.

“Reported patient and caregiver challenges to and opportunities for widespread implementation included patient trust and acceptance of video consulting [2,33,38], the patient’s ability to transport themselves to a “hub” site (for hub-and-spoke video consulting [33]), and patients requiring appropriate technology to participate in video consultations [34].”

Probably not a substantial claim: may not pass S3 (load-bearing).

Martinez et al reported some providers encountered initial hesitance from individuals with spinal cord injuries or disorder, mostly attributed to patients' uncertain feelings about new technology.

“For example, Martinez et al [35] reported that “some providers encountered initial hesitance from individuals with Spinal Cord Injuries or Disorder which they mostly attributed to patients’ uncertain feelings about the new technology.””

• cites [35] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Providers were slow or reluctant to buy in because they did not trust the technology or consider it clinically applicable; did not have experience; did not have adequate time; did not have enough evidence; or lacked training and technological literacy or ongoing support.

“Providers were slow or reluctant to buy in because they did not trust the technology or consider it clinically applicable [33,34,42]; did not have experience with the technology [34,40]; did not have adequate time to devote to the new technology [2,38]; did not have enough evidence to support video consulting use, resulting in resistance to changes in care models, as they found existing care”

May combine two claims.

Potential enablers of spread and scale-up included embedded leadership and the presence of a telehealth champion, appropriate reimbursement mechanisms, user-friendly technology, pre-existing staff relationships, and adaptation over time.

“Potential enablers of spread and scale-up included embedded leadership and the presence of a telehealth champion, appropriate reimbursement mechanisms, user-friendly technology, pre-existing staff relationships, and adaptation (of technology and services) over time.”

May combine two claims.

Challenges tended to be related to service development, such as absence of a long-term strategic plan, resistance to change, cost and reimbursement issues, and technical experience of staff.

“Challenges tended to be related to service development, such as the absence of a long-term strategic plan, resistance to change, cost and reimbursement issues, and the technical experience of staff.”

May combine two claims.

For mental health care in the United States, patients reported liking convenience and privacy of video consulting; however, these responses were collected from patients with repeated prior use, possibly excluding important challenges of patients with little or no experience.

“For mental health care in the United States, patients reported liking the convenience and privacy of video consulting; however, these responses were collected from patients with repeated prior use of video consulting services, possibly excluding important challenges of patients with little or no experience with video consultations [42].”

• cites [42] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Upstream supply of technology was reported as a facilitator by Greenhalgh et al, who explained scale-up in a Skype-based service was facilitated partially by clear benefit for both technology supplier and patient.

“The upstream supply of technology (ie, the relationship between the organization and the supplier or developer) was reported as a facilitator by Greenhalgh et al [38], who explained that scale-up in a Skype-based service was facilitated partially by “clear benefit for both the technology supplier and the patient.””

• cites [38] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

In Nepal, challenges for patients included cultural barriers (feelings of inferiority, women and older patients not speaking for themselves, lack of confidence) and literacy concerns, which made it difficult and time consuming to use video consults for such patients.

“In Nepal, challenges for patients were numerous and included cultural barriers (eg, patients’ feelings of inferiority to health care workers; women and older patients not speaking for themselves; lack of confidence) and literacy concerns, which were reported to make “it difficult and time consuming to use [video consults] for such patients” [32].”

• cites [32] Bhatta, R, 2015, J Nepal Health Res CouncNot checked yet

  Not checked in the free run.

May combine two claims.

Challenges reported in 5 reviewed articles included the labor-intensive process of scheduling video consultation appointments within and across organizations and the need to redesign organizational processes around the technology.

“Challenges reported in 5 articles included the labor-intensive process of scheduling video consultation appointments within and across organizations and the need to redesign organizational processes around the technology [2,36-38,40].”

• cites [2] Shaw, S, 2018, Advantages and limitations of virtual online consultations in a NHS acute trust: the VOCAL mixed-methods studyNot checked yet

  Not checked in the free run.

• cites [36] sourceNot checked yet

  Not checked in the free run.

• cites [37] sourceNot checked yet

  Not checked in the free run.

• cites [38] sourceNot checked yet

  Not checked in the free run.

• cites [40] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Centralized funding for video consulting in the Veterans Health Administration in the United States meant that the availability of national support infrastructure and the absence of billing for services were distinct advantages.

“In contrast, centralized funding for video consulting in the Veterans Health Administration in the United States meant that the “availability of national support infrastructure and the absence of billing for services were distinct advantages” [37].”

• cites [37] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

National strategies in Norway did not always emphasize the potential for scale-up and spread against other priorities; national approaches were mainly focused on messaging services and electronic exchange, with less attention to telemedicine services, especially videoconferencing.

“Even when the political environment was more stable, papers reported that national strategies did not always emphasize the potential for scale-up and spread against other priorities. Take the following report from an evaluation in Norway, which suggests that national approaches “were mainly focused on messaging services and electronic exchange, while less attention was paid towards telemedicine”

• cites [36] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

In a scale-up of 4 video consulting services, the challenges and opportunities for each service were connected across the domains; simply assessing a patient as having a theoretically appropriate condition for video consulting and having a supportive national policy did not guarantee success.

“For example, in a scale-up of 4 video consulting services, the challenges and opportunities for each service were connected across the domains; simply assessing a patient as having a theoretically appropriate condition for video consulting (micro) and having a supportive national policy (macro) did not guarantee success.”

• cites [2] Shaw, S, 2018, Advantages and limitations of virtual online consultations in a NHS acute trust: the VOCAL mixed-methods studyNot checked yet

  Not checked in the free run.

• cites [38] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

In a UK study, although lack of clarity on remuneration was often raised as a key barrier by implementing teams, published policy documents rarely explained how reimbursement for virtual consultations would be implemented.

“In a UK study, the authors reported that, although lack of clarity on remuneration was often raised as a key barrier by implementing teams, published policy documents rarely explained how “reimbursement for virtual consultations would be implemented” [38].”

• cites [38] sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

The absence of consistent theorization was demonstrated in this review by the absence of consideration of how challenges and opportunities might interact with each other and influence the process.

“The absence of consistent theorization was demonstrated in this review by the absence of consideration of how challenges and opportunities might interact with each other and influence the process.”

May combine two claims.

There are gaps in the existing evidence on the spread and scale-up of video consulting, particularly in relation to interorganizational networking, which is critical in enabling and sustaining innovation and yet is not explored in-depth.

“There are gaps in the existing evidence on the spread and scale-up of video consulting. This was particularly the case in relation to interorganizational networking, which is critical in enabling and sustaining innovation [24,30,64], and yet is not explored in-depth.”

• cites [24] sourceNot checked yet

  Not checked in the free run.

• cites [30] sourceNot checked yet

  Not checked in the free run.

• cites [64] sourceNot checked yet

  Not checked in the free run.

• AssumptionRests on an assumption: The absence of mention of specific interorganizational initiatives in the reviewed papers indicates that such initiatives were not present or studied in the included studies.

May combine two claims.

Findings from the review raised challenges including technically challenged staff, resistance to change, cost, reimbursement, patient characteristics, project management, patient recruitment, leadership involvement, funding, absence of a long-term strategic plan, workflow changes, lack of resources, and liability.

“Findings also raised a number of challenges including the following: technically challenged staff, resistance to change, cost, reimbursement, and patient characteristics [24], as well as project management, patient recruitment, leadership involvement, funding, absence of a long-term strategic plan, resistance to change, workflow changes, lack of resources, and liability [64-66].”

• cites [24] sourceNot checked yet

  Not checked in the free run.

• cites [64] sourceNot checked yet

  Not checked in the free run.

• cites [65] Bradford, N, 2016, Rural Remote HealthNot checked yet

  Not checked in the free run.

• cites [66] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Findings from the review pointed to potential enablers of spread and scale-up of video consulting services, notably the presence of a telehealth champion, especially during initial spread, as well as leadership at multiple levels, appropriate reimbursement mechanisms, user-friendly technology, pre-existing relationships between staff, and adaptation of technology and services over time.

“Findings from the review pointed to potential enablers of the spread and scale-up of video consulting services, notably regarding the influence of the presence of a telehealth champion, especially during initial spread [63], as well as leadership at multiple levels, appropriate reimbursement mechanisms, user-friendly technology, pre-existing relationships between staff [64], and adaptation (of”

• cites [63] sourceNot checked yet

  Not checked in the free run.

• cites [64] sourceNot checked yet

  Not checked in the free run.

• cites [65] Bradford, N, 2016, Rural Remote HealthNot checked yet

  Not checked in the free run.

May combine two claims.

There is limited data on the material properties and design of different video consulting platforms, and on the value proposition in relation to spread and scale-up when a technology is free in the place where it was initially developed or implemented but potentially costly in other settings.

“There is also limited data on the material properties and design of different video consulting platforms, and on the value proposition in relation to spread and scale-up when a technology is free (eg, part of a research study) in the place where it was initially developed or implemented, but potentially costly in other settings.”

May combine two claims.

The review reveals an absence of concrete operationalization and theorization of scale-up and spread, exemplified by the absence of analysis and concrete definitions and terminology, as well as a lack of appreciation of complexity, particularly in relation to the clinical condition and organizational implementation.

“The review also reveals an absence of concrete operationalization and theorization of scale-up and spread, exemplified by the absence of analysis and concrete definitions and terminology, as well as a lack of appreciation of complexity (particularly, but not only, in relation to the clinical condition and organizational implementation).”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Measuring and assessing outcomes of implementation and sustainability and sharing those results with stakeholders increases motivation and momentum for practice change and enables sites to respond to challenges in real time.

“Furthermore, the authors emphasized how measuring and assessing outcomes of implementation and sustainability then sharing those results with stakeholders “increases motivation and momentum for practice change and enables site to respond to challenges in real time” [42].”

• cites [42] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

In one VHA health center implementation project, resiliency and sustainability were enabled by introducing video consulting into general practice rather than training specific providers and using a flexible implementation approach that can be modified for different health system contexts.

“In this implementation project, resiliency and sustainability were enabled by introducing video consulting into general practice rather than training specific providers and using a flexible implementation approach that can be modified for different health system contexts.”

• cites [42] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

The trials that have been undertaken on video consulting have provided few or no data on the organizational complexities of implementing a technology-based service, and do not address the question of how video consultation services become embedded in real-world settings.

“The trials that have been undertaken on video consulting have provided few or no data on the organizational complexities of implementing a technology-based service, and do not address the question of how video consultation services become embedded in real-world settings.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

The studies we reviewed are broadly positive about video consulting.

“The studies we reviewed are broadly positive about video consulting.”

Classified as probably not a substantial claim.

The application of the NASSS framework surfaced complexity in a number of domains, notably characterizing the importance of organizational support and the wider system in the scale-up and spread of video consulting services.

“The application of the NASSS framework surfaced complexity in a number of domains, notably characterizing the importance of organizational support and the wider system in the scale-up and spread of video consulting services.”

May combine two claims.

Other frameworks do exist that might support spread and scale-up but they have yet to be taken up in the context of expanding video consulting.

“Other frameworks do exist that might support spread and scale-up (see [24] for an overview) but they have yet to be taken up in the context of expanding video consulting.”

• cites [24] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Studies have drawn on specific implementation frameworks to support and theorize spread and scale-up of video consulting, with a mechanistic logic of change being dominant.

“Studies have drawn on specific implementation frameworks to support and theorize spread and scale-up of video consulting, with a mechanistic logic of change being dominant.”

May combine two claims.

To date, there have been only a handful of rigorous and theoretically grounded qualitative or mixed methods studies that explore the emergence of video consultation services naturalistically, and such studies have yet to systematically study spread and scale-up beyond the initial implementation.

“To date, there have been only a handful of rigorous and theoretically grounded qualitative or mixed methods studies that explore the emergence of video consultation services naturalistically. Such studies have yet to systematically study spread and scale-up beyond the initial implementation.”

May combine two claims.

Prior reviews have focused exclusively on the technical aspects of video consulting technology; considered technologies at any stage of implementation rather than technologies with widespread adoption; or restricted the types of articles included to randomized control trials.

“Prior reviews have also focused exclusively on the technical aspects of video consulting technology [7]; considered technologies at any stage of implementation, rather than technologies with widespread adoption [24]; or restricted the types of articles included in the review to randomized control trials [78].”

• cites [7] sourceNot checked yet

  Not checked in the free run.

• cites [24] sourceNot checked yet

  Not checked in the free run.

• cites [78] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Reviews have focused on telemedicine or eHealth in general without focusing on video consulting, with which our findings are consistent.

“Additionally, reviews have focused on telemedicine or eHealth in general without focusing on video consulting [24,77], with which our findings are consistent.”

• cites [24] sourceNot checked yet

  Not checked in the free run.

• cites [77] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Studies on video consultations tend to be small scale, focus on initial adoption in a research context, adopt a technology-centric approach, and use trial methodology.

“Studies tend to be small scale and focus on initial adoption in a research context [2,5,18]; adopt a technology-centric approach (in which the technology is the primary focus, rather than the service or organization by which the technology is being used); and use trial methodology to study whether video consultation technology works or not.”

• cites [2] Shaw, S, 2018, Advantages and limitations of virtual online consultations in a NHS acute trust: the VOCAL mixed-methods studyNot checked yet

  Not checked in the free run.

• cites [5] sourceNot checked yet

  Not checked in the free run.

• cites [18] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The COVID-19 pandemic and the need for self-isolation and social distancing have prompted rapid and widespread adoption of video consultations.

“The COVID-19 pandemic, and the requisite need for self-isolation and social distancing, has prompted rapid and widespread adoption of video consultations [4].”

• cites [4] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

There is a rapidly growing literature on the feasibility, safety, acceptability, and effectiveness of video consultations across clinical areas.

“There is a rapidly growing literature on the feasibility, safety, acceptability, and effectiveness of video consultations across clinical areas [5] including diabetes [6,7], rehabilitation [8,9], mental health and addiction [10], cancer [11], palliative care [12,13], long-term care [14], geriatrics [15], postpartum support [16], and primary care [17].”

• cites [5] sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

The synthesis worked inductively to surface challenges and opportunities, then deductively using the NASSS framework to ensure all 7 domains were covered.

“We worked inductively to surface the challenges to and opportunities for the spread and scale-up of video consulting across studies. Sensitized by the Nonadoption, Abandonment, and challenges to the Scale-up, Spread, and Sustainability (NASSS) of health and care technologies framework [23], we then worked deductively to ensure we had not missed any of the 7 domains identified as relevant to the”

• cites [23] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The synthesis process was piloted and refined on one paper and then extended across the rest of the data set.

“We piloted and refined this process on one paper and then extended the process across the rest of our data set.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The initial search strategy was referenced against two published systematic reviews on telegenetics and implementation science to identify supplementary terms.

“We referenced our initial search strategy against two published systematic reviews on telegenetics [28] and implementation science [29] to identify supplementary terms.”

• cites [28] sourceNot checked yet

  Not checked in the free run.

• cites [29] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Systematic searches were conducted in PubMed, CINAHL via EBSCOhost, and Web of Science in December 2018, updated in March 2020.

“In December 2018, we systematically searched 3 databases: PubMed, CINAHL via EBSCOhost, and Web of Science. The search was updated in March 2020.”

Classified as probably not a substantial claim. May combine two claims.

Included articles were peer-reviewed and reported on synchronous, video-supported consultations scaled or spread to more than one setting beyond the initial pilot or feasibility stage.

“Included articles were peer-reviewed and reported on synchronous, video-supported consultations that had been scaled or spread to more than one setting beyond the initial pilot or feasibility stage (either within the same organization or to other organizations or geographic settings).”

Classified as probably not a substantial claim. May combine two claims.

Content analysis was used to compile a descriptive overview of opportunities and challenges, including frequency distributions, and NASSS was applied to synthesize and interpret findings.

“Using content analysis, we compiled a descriptive overview of the opportunities and challenges identified in the papers, including frequency distributions. We then applied a more analytical lens to synthesize and interpret our findings using the NASSS framework [23].”

• cites [23] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Search strategies were developed with a research librarian and used a mix of keywords, MeSH, and MH.

“Our search strategies, developed with the help of a research librarian, used a mix of keywords, Medical Subject Headings (MeSH), and Major Headings (MH), as provided in Multimedia Appendix 2.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Each record was first screened by title and abstract by HJ, then verified by JW and SS; remaining articles were screened by full texts by HJ (100%) and SS (15% sample), with disputes resolved through discussion and consensus with JW.

“Each record was first screened by title and abstract (initially by HJ, then verified by JW and SS). Remaining articles were screened by reading the full texts (HJ screening 100%, SS a 15% sample), with any disputes resolved through discussion and consensus with JW.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

PRISMA-P was used to draft the protocol and PRISMA guidelines were consulted throughout the review.

“PRISMA-P (Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols) was used to draft the protocol for this review (unpublished) and PRISMA guidelines were consulted throughout the review as a guide.”

Classified as probably not a substantial claim.

Spread, scale-up, and sustainability are often used as interchangeable terms without a standard definition or adequate theorization.

“Spread, scale-up, and sustainability are often used as interchangeable terms without a standard definition or adequate theorization (see Multimedia Appendix 1).”

• cites Multimedia Appendix 1Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

Articles exclusively reporting use, feasibility, acceptance, or pilot implementation with no evaluation of the implementation process were excluded.

“Articles exclusively reporting use, feasibility, acceptance, or pilot implementation with no evaluation of the implementation process were excluded.”

May combine two claims.

The same things that interfere with implementation at one site are also seen as interfering with spread and scale-up, although this may be an artifact of how spread is commonly studied as the sum of multiple implementations.

“They also indicate that the same things that interfere with implementation at one site [eg, 48] are also seen as interfering with spread and scale-up [5,7,17,24], although this may be an artifact of how spread is commonly studied as the sum of multiple implementations.”

• cites [5] sourceNot checked yet

  Not checked in the free run.

• cites [7] sourceNot checked yet

  Not checked in the free run.

• cites [17] sourceNot checked yet

  Not checked in the free run.

• cites [24] sourceNot checked yet

  Not checked in the free run.

• cites [48] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The strengths of this review include the comprehensive searching methods used to identify a breadth of published research, albeit limited in numbers and scope.

“The strengths of this review include the comprehensive searching methods used to identify a breadth of published research (albeit limited in numbers and scope).”

May combine two claims.

Publication bias is a long-standing and recognized phenomenon in health research and likely impacts the broadly positive results across studies included in this review.

“Publication bias is a long-standing and recognized phenomenon in health research [71] and likely impacts the broadly positive results across studies included in this review.”

• cites [71] sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

Given the lack of theorization in the included studies, it is hard to know how broadly the factors that may have acted as challenges or opportunities were considered in the evaluations.

“Given the lack of theorization in the included studies, it is hard to know how broadly the factors that may have acted as challenges or opportunities were considered in the evaluations.”

Classified as probably not a substantial claim. May combine two claims.

We cannot claim that this evidence provides a conclusive list of the challenges to scale-up and spread generally, nor even specifically to the studies included in this review.

“The consequence of this is that we cannot claim that this evidence provides a conclusive list of the challenges to scale-up and spread generally, nor even specifically to the studies included in this review.”

May combine two claims.

This review is limited by likely bias within the articles, bias inherent in the reviewing process, and the lack of theorization in the included studies.

“This review is limited by likely bias within the articles, bias inherent in the reviewing process, and the lack of theorization in the included studies.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The wider context of the COVID-19 pandemic might similarly enable widespread adoption of video consulting services.

“This suggests that the wider context of the COVID-19 pandemic might similarly enable widespread adoption of video consulting services.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The analysis and synthesis of the listed challenges has been framed by a comprehensive theoretical framework.

“The analysis and synthesis of the listed challenges has also been framed by a comprehensive theoretical framework.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The final domain, embedding and adaptation over time, focuses on the scope for adaptation and the resilience of the organization in the face of implementation and potential spread and scale-up.

“The final domain, embedding and adaptation over time, focuses on the scope for adaptation and the resilience of the organization in the face of implementation and potential spread and scale-up [23].”

• cites [23] sourceNot checked yet

  Not checked in the free run.