CoChat
Sign Up
Cited by 181Telemedicine and Telehealth ImplementationHealthcare Systems and TechnologyHealthcare Systems and Practices

Planning and Evaluating Remote Consultation Services: A New Conceptual Framework Incorporating Complexity and Practical Ethics

Open access
Greenhalgh, Trisha, Rosen, Rebecca, Shaw, Sara E., Byng, Richard, Faulkner, Stuart, Finlay, Teresa, Grundy, Emily, Husain, Laiba +10 more·2021·Frontiers in Digital Health
DOI 10.3389/fdgth.2021.726095PMC PMC8521880
Open this paper in CoChat — free accountOpen PDF

Abstract

Establishing and running remote consultation services is challenging politically (interest groups may gain or lose), organizationally (remote consulting requires implementation work and new roles and workflows), economically (costs and benefits are unevenly distributed across the system), technically (excellent care needs dependable links and high-quality audio and images), relationally (interpersonal interactions are altered), and clinically (patients are unique, some examinations require contact, and clinicians have deeply-held habits, dispositions and norms). Many of these challenges have an under-examined ethical dimension. In this paper, we present a novel framework, Planning and Evaluating Remote Consultation Services (PERCS), built from a literature review and ongoing research. PERCS has 7 domains—the reason for consulting, the patient, the clinical relationship, the home and family, technologies, staff, the healthcare organization, and the wider system—and considers how these domains interact and evolve over time as a complex system. It focuses attention on the organization's digital maturity and digital inclusion efforts. We have found that both during and beyond the pandemic, policymakers envisaged an efficient, safe and accessible remote consultation service delivered through state-of-the art digital technologies and implemented via rational allocation criteria and quality standards. In contrast, our empirical data reveal that strategic decisions about establishing remote consultation services, allocation decisions for appointment type (phone, video, e-, face-to-face), and clinical decisions when consulting remotely are fraught with contradictions and tensions—for example, between demand management and patient choice—leading to both large- and small-scale ethical dilemmas for managers, support staff, and clinicians. These dilemmas cannot be resolved by standard operating procedures or algorithms. Rather, they must be managed by attending to here-and-now practicalities and emergent narratives, drawing on guiding principles applied with contextual judgement. We complement the PERCS framework with a set of principles for informing its application in practice, including education of professionals and patients.

AI snapshot

Written from the paper’s full text.

TL;DR

This paper presents PERCS, a complexity-informed framework for planning and evaluating remote consultation services, adapted from NASSS and refined using UK mixed-method studies and a Delphi exercise. It argues that remote consultation decisions are ethically and practically fraught, requiring contextual judgement and guiding principles rather than algorithms.

Key findings

  • PERCS was built from a literature review and ongoing research and adapted from NASSS; it covers domains including the reason for consulting, the patient, the clinical relationship, the home and family, technologies, staff, the healthcare organization, and the wider system, follows their interaction over time, and adds side panels on organizational digital maturity and digital inclusion.
  • Policymakers envisaged an efficient, safe, accessible remote service delivered through state-of-the-art digital technologies and implemented via rational allocation criteria and quality standards; empirical data showed strategic, allocation, and clinical decisions were fraught with contradictions and tensions (e.g., demand management vs patient choice), producing ethical dilemmas not resolvable by standard operating procedures or algorithms.
  • A four-round Delphi produced 26 guiding principles; in round 2 all but one of 25 revised principles were supported by ≥80%, in round 3 all but one of 26 were supported by ≥80% (final item 71%), and after a fourth round there was high agreement on all 26. Final agreement ranged from 80% (patient training in digital skills) to 100% (safeguarding; balancing patient autonomy with support from carers/friends; remote physical examinations; information continuity and action points).
  • The digital maturity scale has five levels from Level 1 Traditional (reactive) to Level 5 System-oriented (extending and spreading); digital inclusion was conceptualized as requiring diversity of provision, digital access support, and non-digital alternatives. Despite policy emphasis on video, telephone was the dominant technology used.

Methods

  • Multiple UK mixed-methods studies, predominantly qualitative, using interviews, ethnography, documentary analysis, and descriptive quantitative data. Key datasets included a Scottish Government evaluation with 60 h of ethnographic observation, 223 interviews, and quantitative analysis of over 69,000 consultations (including over 18,000 patient assessments); Remote by Default with over 100 interviews, four locality case studies, four online focus groups with 19 participants, four cross-sector workshops with ~160 stakeholders, and a four-round Delphi; and the Health Foundation Video Consulting study with a survey of over 800 NHS staff, 40 follow-up interviews (20 repeated longitudinally), 10 patient interviews, two group discussions with 15 patient/public representatives, and 7 locality case studies.
  • Framework development involved coding transcripts, field notes, and workshop write-ups (NVivo or Excel), descriptive statistics for quantitative data, narrative synthesis, and iterative team discussion; NASSS was initially used and then adapted into PERCS as it proved inadequate for some findings.
  • The Delphi recruited 69 participants, of whom 19 did not attempt any items; 50 attempted at least one item in round 1, 40 in round 2, and 37 in rounds 3 and 4. It began with 60 draft principles organized into 30 partially contradictory pairs under four categories, then moved to a simple list of 25 principles in round 2 and 26 in round 3, with a fourth round on one remaining item; analysis used descriptive statistics and thematic analysis.

Limitations

  • Stated: all empirical studies were UK-based, so PERCS and the guiding principles may not transfer to very different contexts without adaptation; pandemic restrictions made ethnography impossible for later months, and although ethics approval was obtained to video and audiotape clinical consultations, this proved impossible in practice, so insights are based more on what people said was happening than on direct observation; the Delphi sample was relatively small and should be replicated and tested prospectively; PERCS is explanatory, not deterministic or predictive.
  • Evident from the text: the Delphi had substantial attrition (69 recruited, 19 attempted no items, 37 in final rounds) and later rounds included 11 patients or carers alongside mostly professional participants; the planned statistical analysis of Delphi quantitative data was abandoned after presenting principles in deliberately contradictory pairs caused confusion and irritation; SurveyMonkey was used because pandemic home-working precluded more specialized tools.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.3389/fdgth.2021.726095

Cites

111 of 111 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 137 cited sources checked · 0 supported · 0 flagged

  • Findingcitation

    Remote consulting is cognitively burdensome.

    “Our findings affirmed published claims that remote consulting is cognitively burdensome (54), excludes the physical touch on which the therapeutic relationship has traditionally depended (55), threatens continuity of care (7), brings greater clinical uncertainty, and makes the personal care afforded by the extended, face-to-face “long consultation” more difficult (8, 56, 57).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [54] Ambrose, 2020, Remote consulting: recognising the cognitive load, Br J Gen Pract. source

      Not checked in the free run.

  • Findingown data

    NASSS did not fully explain all of our empirical data-especially findings around the clinical relationship and the ethics of allocating appointment type.

    “But whilst NASSS was our theoretical starting point, we quickly discovered that it did not fully explain all of our empirical data—especially findings around the clinical relationship and the ethics of allocating appointment type.”
    Quote in the paperSubstantial claim
  • Findingown data

    Clinical risk management and safety-netting, especially in acutely unwell patients, was a theme in the in-pandemic datasets.

    “- Clinical risk management and safety-netting, especially in acutely unwell patients (e.g., deciding which patients with suspected COVID-19 to see in person or send to hospital);”
    Quote in the paperSubstantial claim
  • Findingown data

    Technical issues, including technical infrastructure; interoperability and in patients' homes, and the wasted time and stress caused by unreliable internet access and technologies, was a theme in the in-pandemic datasets.

    “- Technical issues, including technical infrastructure; interoperability (especially interfacing with the electronic patient record) and in patients' homes (e.g., linking with multiple devices), and the wasted time and stress caused by unreliable internet access and technologies;”
    Quote in the paperNot a substantial claim
  • Findingown data

    The patient's home and family context, notably concerns about privacy and safety, was a theme in the in-pandemic datasets.

    “- The patient's home and family context, notably concerns about privacy and safety (e.g., whether patients are safe from eavesdropping or coercion);”
    Quote in the paperNot a substantial claim
  • May combine two claims. 57
  • Classified as probably not a substantial claim. 55

  • May combine two claims.

    Findingown data

    Two prominent themes in our data were organizations' digital maturity in providing remote consultations and the need for proactive measures to improve digital inclusion.

    “Two prominent themes in our data were organizations' digital maturity in providing remote consultations and the need for proactive measures to improve digital inclusion.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    There were tensions between quality and efficiency, between demand management and patient choice, and between the needs and preferences of some patients and those of other patients in the context of limited staffing and material constraints.

    “There were tensions, for example, between quality and efficiency, between demand management and patient choice, and between the needs and preferences of some patients and those of other patients in the context of limited staffing and material constraints (e.g., availability of safe waiting areas which limited face-to-face appointments during the pandemic).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    The only available space for one (female) GP to consult from was the kitchen table, necessitating complex negotiations with other family members about access to the food supply.

    “The only available space for one (female) GP to consult from was the kitchen table, necessitating complex negotiations with other family members about access to the food supply.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    They talked about the need to be alert to the possibility of the "doorknob phenomenon"-that people sometimes seek a consultation for one reason but bring in the real reason late in the encounter, hence withholding key information at the triage stage.

    “They talked about the need to be alert to the possibility of the “doorknob phenomenon” —that people sometimes seek a consultation for one reason but bring in the real reason late in the encounter, hence withholding key information at the triage stage.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Policymakers envisaged an efficient, safe and accessible remote consultation service delivered through state-of-the art digital technologies and implemented via rational allocation criteria and quality standards.

    “We have found that both during and beyond the pandemic, policymakers envisaged an efficient, safe and accessible remote consultation service delivered through state-of-the art digital technologies and implemented via rational allocation criteria and quality standards.”
    • Also stated in: § Developing the Guiding Principles · hedged
      “Evidence from policy announcements and our elite policymaker interviews revealed a relatively confident vision of an efficient, safe and accessible remote consultation service delivered mostly through state-of-the art digital technologies and implemented via rational allocation criteria and quality standards.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingnone given

    Access to care, especially patients' difficulty getting the kind of appointment they wished and conflict with support staff associated with this, was a theme in the in-pandemic datasets.

    “- Access to care, especially patients' difficulty getting the kind of appointment they wished and conflict with support staff associated with this;”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    An initial plan to analyse quantitative data statistically was abandoned when it became clear that presenting our draft principles in deliberately contradictory pairs had led to confusion and irritation.

    “An initial plan to analyse quantitative data statistically was abandoned when it became clear that presenting our draft principles in deliberately contradictory pairs had led to confusion and irritation (which we discuss briefly in the Findings section).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation

    Clinicians described over-riding such systems because they had a gut feeling that a seemingly minor aspect of the history could indicate serious illness or because they had safety-netting concerns that a particular patient should be checked more frequently than the recommended interval.

    “Clinicians described over-riding such systems (for example when triaging e-consultations) because they had a gut feeling that a seemingly minor aspect of the history could indicate serious illness (45) or because they had safety-netting concerns that a particular patient should be checked more frequently than the recommended interval.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [45] Smith, 2020, Understanding the role of GPs' gut feelings in diagnosing cancer in primary care: a systematic review and meta-analysis of existing evidence, Br J Gen Pract. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    Many staff-rightly or wrongly-still considered face-to-face consulting to be the "gold standard" professional norm.

    “Many staff—rightly or wrongly—still considered face-to-face consulting to be the “gold standard” professional norm:”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    During the pandemic, some locality-based clinical commissioning groups became tied into commercial contracts with video software providers whose products were approved at speed at the height of the crisis and offered "free" or low-cost for the first year.

    “During the pandemic, some locality-based clinical commissioning groups became tied into commercial contracts with video software providers whose products were approved at speed at the height of the crisis and offered “free” or low-cost for the first year.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Several examples from our dataset illustrated the subtleties of the reason for consulting and why rigid algorithms or allocation criteria may prove too brittle to guide practice.

    “Several examples from our dataset illustrated the subtleties of the reason for consulting and why rigid algorithms or allocation criteria may prove too brittle to guide practice.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    One GP expressed concern that a patient was consulting from the lavatory because this was the only quiet and private space available. Another described connecting to a teenager's bedroom where a parent may have been present but off camera.

    “One GP expressed concern that a patient was consulting from the lavatory because this was the only quiet and private space available. Another described connecting to a teenager's bedroom where a parent may have been present but off camera.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Learning from models of good practice elsewhere tended to be a positive experience, whereas public release of performance data was viewed negatively by our participants.

    “Learning from models of good practice elsewhere tended to be a positive experience, whereas public release of performance data—in which each organization or department is compared competitively with others (for example in the proportion of consultations undertaken remotely)—was viewed negatively by our participants.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Whilst clinicians work comfortably with contradictory principles in their day to day work, they experienced strong cognitive dissonance when asked to confront them in a desktop exercise without a real, practical situation in mind.

    “This had been our intention, but it was clear that whilst clinicians work comfortably with contradictory principles in their day to day work, they experienced strong cognitive dissonance when asked to confront them in a desktop exercise without a real, practical situation in mind.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation

    Additional ethical tensions related to staff well-being and redistribution of the work of caring.

    “Additional ethical tensions related to staff well-being and redistribution of the work of caring (often shifting to the patient or lay carer).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Our interviewees depicted these changes positively-indeed, they considered that the changes would have been impossible or much delayed without them-but they also expressed concern about what would happen when organizations returned to business as usual, since the longer-term trade-offs between convenience and regulatory control are unclear.

    “Our interviewees depicted these changes positively—indeed, they considered that the changes would have been impossible or much delayed without them—but they also expressed concern about what would happen when organizations returned to business as usual, since the longer-term trade-offs between convenience and regulatory control (e.g., data privacy) are unclear.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Our dataset included numerous examples of capacity constraints which limited organizations' abilities to undertake co-design work, provide a high degree of flexibility or provide sufficient face-to-face slots to accommodate all requests.

    “However, our dataset included numerous examples of capacity constraints which limited organizations' abilities to undertake co-design work, provide a high degree of flexibility or provide sufficient face-to-face slots to accommodate all requests.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Findingown data

    The theme of digital inclusion recurred in our data.

    “The theme of digital inclusion recurred in our data.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Examples of digital inclusion efforts in our dataset included flexibility to accommodate patient preferences, signposting to local digital literacy programmes, and making it clear to patients that face-to-face appointments were available on request.

    “Examples of digital inclusion efforts in our dataset included flexibility to accommodate patient preferences (some teenagers, for example, preferred to consult with the video switched off; some parents preferred to video their child doing a task and send the video to the therapist rather than the child perform live to camera), signposting to local digital literacy programmes, and making it clear”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    All but one of the 25 revised principles in round 2 were supported in principle by 80% or more respondents, but many had suggestions for changes in wording.

    “All but one of the 25 revised principles in round 2 were supported in principle by 80% or more respondents, but many had suggestions for changes in wording (e.g., be less paternalistic, reduce jargon, split a maxim into two).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    All but one of the 26 revised principles in round 3 were supported by 80% or more participants with only minor suggestions for further revisions to wording; the final item was supported by 71%.

    “All but one of the 26 revised principles in round 3 were supported by 80% or more participants with only minor suggestions for further revisions to wording; the final item was supported by 71%, with several respondents all suggesting omission of a particular phrase.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Our empirical data revealed that strategic decisions about setting up remote consultation services, allocation decisions for appointment type, and clinical decisions when consulting remotely were fraught with contradictions and tensions, leading to both large- and small-scale ethical dilemmas that were either unique to the remote modality or greatly exacerbated by it.

    “In contrast, our empirical data revealed that strategic decisions about setting up remote consultation services, allocation decisions for appointment type (phone, video, e-, face-to-face), and clinical decisions when consulting remotely were fraught with contradictions and tensions, leading to both large- and small-scale ethical dilemmas that were either unique to the remote modality or greatly”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    Establishing and running remote consultation services is challenging politically, organizationally, economically, technically, relationally, and clinically.

    “Establishing and running remote consultation services is challenging politically (interest groups may gain or lose), organizationally (remote consulting requires implementation work and new roles and workflows), economically (costs and benefits are unevenly distributed across the system), technically (excellent care needs dependable links and high-quality audio and images)”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    Clinical allocation criteria, perhaps built into algorithms, may suggest (though not determine) a type of appointment.

    “Clinical allocation criteria, perhaps built into algorithms, may suggest (though not determine) a type of appointment.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    Costs of technologies are typically unevenly distributed across the healthcare system.

    “Costs of technologies are typically unevenly distributed across the healthcare system.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    On 30th July 2020, it was announced that all healthcare consultations in the UK should henceforth be "remote by default," not just during the pandemic but indefinitely.

    “On 30th July 2020, it was announced that all healthcare consultations in the UK should henceforth be “remote by default,” not just during the pandemic but indefinitely (1).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [1] Hancock, 2020

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    As Hunter (39) has argued, clinical decisions are governed not by hard and fast rules but by shared rules of thumb or guiding principles which she calls maxims, some of which are contradictory.

    “As Hunter (39) has argued, clinical decisions are governed not by hard and fast rules but by shared rules of thumb or guiding principles which she calls maxims, some of which are contradictory (e.g., “ignore the anecdotal” but “listen to the patient's story”).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [39] Hunter, 1996, “Don't think zebras”: uncertainty, interpretation, and the place of paradox in clinical education, Theor Med. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationreasoning

    Whilst Hunter focused narrowly on clinical maxims, case-based ethical reasoning more broadly operates along similar lines.

    “Whilst Hunter focused narrowly on clinical maxims, case-based ethical reasoning more broadly—whether, for example, to invoke child protection measures or ask a social prescriber to get involved in someone's life—operates along similar lines.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    The situational application of such guiding principles has an important time dimension.

    “The situational application of such guiding principles has an important time dimension.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationnone given

    Both NASSS and PERCS are explanatory frameworks to guide a holistic interpretation of a complex and evolving phenomenon; they do not offer predictive certainty and are not intended to be applied formulaically.

    “Importantly, both NASSS and the adaptation described here (PERCS) are explanatory frameworks to guide a holistic interpretation of a complex and evolving phenomenon. They do not offer predictive certainty and are not intended to be applied formulaically.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    Maxims encapsulate shared understanding and wisdom; they suggest a potential way forward but are high-level enough to be flexibly applied.

    “Maxims, which tend to be passed on orally from more to less experienced practitioners, encapsulate shared understanding and wisdom; they suggest a potential way forward but are high-level enough to be flexibly applied.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [39] Hunter, 1996, “Don't think zebras”: uncertainty, interpretation, and the place of paradox in clinical education, Theor Med. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The Delphi method is a well-established semi-structured approach to working toward consensus among experts.

    “To develop and refine some ethical principles to guide application of the PERCS framework, we used the Delphi method—a well-established semi-structured approach to working toward consensus among experts (41).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [41] Boulkedid, 2011, Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review, PLoS ONE. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    With time, one constellation of symptoms, signs, and contextual influences will evolve to a subtly different constellation, generating clues as to the nature of the illness and its likely regression or further progression.

    “With time, one constellation of symptoms, signs, and contextual influences will evolve to a subtly different constellation, generating clues as to the nature of the illness and its likely regression or further progression.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisenone given

    Some software products (e.g., accuRx) potentially allow the clinician to instantly convert a phone consultation to video in real time, thus avoiding double-handling, but products which lack this functionality require a new appointment.

    “Some software products (e.g., accuRx) potentially allow the clinician to instantly convert a phone consultation to video in real time, thus avoiding double-handling, but products which lack this functionality require a new appointment.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Telephone advice services, underpinned by advanced computer decision support systems, can be costly to set up and maintain and require new work skills to operate; their introduction may generate supply-induced demand.

    “Telephone advice services, underpinned by advanced computer decision support systems, can be costly to set up and maintain and require new work skills to operate (32); their introduction may generate supply-induced demand (53).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [32] Turnbull, 2014, The work, workforce, technology and organisational implications of the ‘111' single point of access telephone number for urgent (non-emergency) care: a mixed-methods case study, Health Serv Delivery Res. source

      Not checked in the free run.

    • cites [53] Pope, 2017, Has the NHS 111 urgent care telephone service been a success? Case study and secondary data analysis in England, BMJ Open. source

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data

    Consulting behavior patterns previously learnt in the context of the clinic must now be transposed to the home space, which may produce dissonance.

    “More prosaically, consulting behavior patterns previously learnt in the context of the clinic must now be transposed to the home space, which may produce dissonance, as the following quote illustrates:”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    All these factors may help explain why the patient seeks a particular mode of consultation-and also why a clinician or support staff member may decide that it is in the patient's best interests (or not) to be seen face-to-face.

    “All these factors may help explain why the patient seeks a particular mode of consultation—and also why a clinician or support staff member may decide that it is in the patient's best interests (or not) to be seen face-to-face, as the following quote illustrates:”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    We recruited and obtained emailed consent from an initial sample of 69, though 19 of these did not attempt any items (mostly because they found the exercise too difficult).

    “We recruited and obtained emailed consent from an initial sample of 69, though 19 of these did not attempt any items (mostly because they found the exercise too difficult).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisenone given

    Telephone has less functionality video but is more dependable and familiar.

    “Telephone, for example, has less functionality video but is more dependable and familiar.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given

    Some reasons for consulting are clinician- or system-driven (e.g., invitations for screening or long-term condition monitoring).

    “Some reasons for consulting are clinician- or system-driven (e.g., invitations for screening or long-term condition monitoring).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Advantages of the Delphi method include practicality, anonymity, and iteration.

    “Advantages of this method include practicality (it can be done online, asynchronously, without specialist tools), anonymity (participants know the average group score but not individuals' scores), and iteration (through feedback, outliers are prompted to either defend their response to the group or change it) (42, 43).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [42] Khodyakov, 2020, Response changes in Delphi processes: why is it important to provide high-quality feedback to delphi participants?, J Clin Epidemiol source

      Not checked in the free run.

    • cites [43] Belton, 2019, Improving the practical application of the Delphi method in group-based judgment: a six-step prescription for a well-founded and defensible process, Technol Forecast Soc Change. source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    Maxims and other high-level guiding principles require an understanding of the circumstances in which the rule should be applied rather than formulaic replication.

    “Maxims and other high-level guiding principles require an understanding of the circumstances in which the rule should be applied rather than formulaic replication.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given

    Arguments against remote forms of care tended to be patient-focused, highlighting possible disbenefits such digital exclusion.

    “Arguments against remote forms of care tended to be patient-focused, highlighting possible disbenefits such digital exclusion.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    Staff attitudes are grounded in professional norms and values including deeply-held concerns about quality and safety of care.

    “Staff attitudes are grounded in professional norms and values including deeply-held concerns about quality and safety of care.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    The NASSS framework has been widely used across a range of settings for evaluating and explaining the fortunes of various kinds of health technology projects and draws on complexity theory.

    “The NASSS framework has been widely used across a range of settings for evaluating and explaining the fortunes of various kinds of health technology projects [see for example (18–20)] and draws on complexity theory (21).”
    Quote in the paperNot a substantial claim0 of 4 sources checked
    • cites [18] Greenhalgh, 2018, Analysing the role of complexity in explaining the fortunes of technology programmes: empirical application of the NASSS framework, BMC Med. source

      Not checked in the free run.

    • cites [19] Abimbola, 2019, The NASSS framework for ex post theorisation of technology-supported change in healthcare: worked example of the TORPEDO programme, BMC Med. source

      Not checked in the free run.

    • cites [20] Dyb, 2021, Adopt, adapt, or abandon technology-supported person-centred care initiatives: healthcare providers' beliefs matter, BMC Health Serv Res. source

      Not checked in the free run.

    • cites [21] Greenhalgh, 2018, Studying complexity in health services research: desperately seeking an overdue paradigm shift, BioMed source

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    NASSS explores the dynamic interaction between multiple domains in a complex system and how these domains and their interdependencies have evolved-and are likely to evolve further-over time.

    “NASSS explores the dynamic interaction between multiple domains (the technology, the people and so on) in a complex system and how these domains and their interdependencies have evolved—and are likely to evolve further—over time.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [17] Greenhalgh, 2017, Beyond adoption: a new framework for theorizing and evaluating nonadoption, abandonment, and challenges to the scale-up, spread, and sustainability of health and care technologies, J Med Internet Res. source

      Not checked in the free run.

    • cites [21] Greenhalgh, 2018, Studying complexity in health services research: desperately seeking an overdue paradigm shift, BioMed source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationnone given

    Many of these challenges have an under-examined ethical dimension.

    “Many of these challenges have an under-examined ethical dimension.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Digital inclusion is high on the policy agenda in the UK and elsewhere.

    “Digital inclusion is high on the policy agenda in the UK and elsewhere (79, 80).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [79] 2020

      Not checked in the free run.

    • cites [80] NHS, 2020

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    These themes resonated with writing by feminist philosophers on the ethics of care, including the hidden work and emotional labor of low-status workers such as receptionists and unpaid carers.

    “These themes resonated with writing by feminist philosophers (89–92) on the ethics of care, including the hidden work and emotional labor of low-status workers such as receptionists and unpaid carers.”
    Quote in the paperNot a substantial claim0 of 4 sources checked
    • cites [89] Tronto, 1993

      Not checked in the free run.

    • cites [90] Fisher, 1990, Toward a feminist theory of caring

      Not checked in the free run.

    • cites [91] Held, 2006

      Not checked in the free run.

    • cites [92] Pols, 2015, Towards an empirical ethics in care: Relations with technologies in health care, Med Health Care Philos. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Burden of treatment also includes the effort needed from the patient to access services, which technologies (especially e-consultations) could potentially alleviate.

    “Burden of treatment also includes the effort needed from the patient to access services (31, 93), which technologies (especially e-consultations) could potentially alleviate.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [31] Turnbull, 2019, A conceptual model of urgent care sense-making and help-seeking: a qualitative interview study of urgent care users in England, BMC Health Serv Res. source

      Not checked in the free run.

    • cites [93] May, 2014, Rethinking the patient: using Burden of Treatment Theory to understand the changing dynamics of illness, BMC Health Serv Res. source

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    Infrastructure also includes human elements, shared understandings and historical path-dependencies, and the regulatory and professional standards.

    “Infrastructure also includes human elements (e.g., organizational roles, routines, and relationships), shared understandings and historical path-dependencies (such as legacy technologies and commercial contracts), and the regulatory and professional standards noted above (27, 37).”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [27] Greenhalgh, 2019, Infrastructure revisited: an ethnographic case study of how health information infrastructure shapes and constrains technological innovation, J Med Internet Res. source

      Not checked in the free run.

    • cites [37] Star, 1999, The ethnography of infrastructure, Am Behav Sci. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    May et al. have called burden of treatment-the additional burden placed on the patient when they are asked to "self-manage" -with the sickest and most vulnerable carrying the greatest burden.

    “A related concept is what May et al. (93) have called burden of treatment—the additional burden placed on the patient when they are asked to “self-manage” —with the sickest and most vulnerable carrying the greatest burden.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [93] May, 2014, Rethinking the patient: using Burden of Treatment Theory to understand the changing dynamics of illness, BMC Health Serv Res. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    The evidence base for achieving greener health services by means of remote care is limited and contested, but one study suggests that a substantial reduction in carbon footprint could be made.

    “The evidence base for achieving greener health services by means of remote care is limited and contested, but one study suggests that a substantial reduction in carbon footprint could be made (66).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [66] Holmner, 2014, Carbon footprint of telemedicine solutions-unexplored opportunity for reducing carbon emissions in the health sector, PLoS ONE. source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    Pols has developed an empirical ethics of [technology-supported] care which considers technologies as relational actors which can enhance and support the personal ethical commitments and warm care relations between people.

    “Pols (92) has developed what she calls an empirical ethics of [technology-supported] care which considers technologies not as inanimate tools which make the clinical interaction more or less efficient (and as “cold” artifacts that contrast with “warm” human care) but as relational actors which, if creatively selected and “tinkered with” to fit specific situations, can enhance and support the”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [92] Pols, 2015, Towards an empirical ethics in care: Relations with technologies in health care, Med Health Care Philos. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Innovative organizations tend to be large, competently led and with clear strategic vision, functionally differentiated, non-hierarchical and with adequate slack.

    “Innovative organizations tend to be large, competently led and with clear strategic vision, functionally differentiated, non-hierarchical and with adequate slack (people and resources that can be channeled into new projects to get them up and running).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [62] Greenhalgh, 2004, Diffusion of innovations in service organizations: systematic review and recommendations, Milbank Q. source

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    During the pandemic a sense of urgency, relaxing of regulations and policy investment in technological solutions produced many new technological components but there was not time to strengthen either technical or human infrastructure, yet successful embedding and use of these novel solutions will be contingent upon the openness, distributedness, recombinability, re-programmability, andaccess

    “As Gkeredakis et al. (4) have pointed out, during the pandemic a sense of urgency, relaxing of regulations and policy investment in technological solutions produced many new technological components but there was not time to strengthen either technical or human infrastructure, yet successful embedding and use of these novel solutions will be “contingent upon the openness, distributedness, recombin”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Gkeredakis et al. (4) Gkeredakis, 2021, Crisis as opportunity, disruption and exposure: exploring emergent responses to crisis through digital technology, Inform Organ. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Tudor Hart's inverse care law states that people most in need of health care are least likely to seek it or receive it; the law reflects two mutually-reinforcing phenomena-worse health in deprived localities and barriers to accessing healthcare in those same localities.

    “Tudor Hart's (71) inverse care law states that people most in need of health care are least likely to seek it or receive it; the law reflects two mutually-reinforcing phenomena—worse health in deprived localities and barriers to accessing healthcare in those same localities (47, 48, 72, 73).”
    Quote in the paperNot a substantial claim0 of 5 sources checked
    • cites [71] Hart, 1971, The inverse care law, Lancet. source

      Not checked in the free run.

    • cites [47] Jansen, 2020, Socioeconomic inequalities in out-of-hours primary care use: an electronic health records linkage study, Eur J Public Health. source

      Not checked in the free run.

    • cites [48] Marmot, 2018, An inverse care law for our time, BMJ. source

      Not checked in the free run.

    • cites [72] Mercer, 2012, Multimorbidity and the inverse care law in primary care, BMJ. source

      Not checked in the free run.

    • cites [73] Rolewicz, 2020, Are the needs of people with multiple long-term conditions being met? Evidence from the 2018 General Practice Patient Survey, BMJ Open. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    NHS England recently announced a vision for a "net zero [carbon] NHS," including a major contribution of new modalities for delivering care.

    “NHS England, for example, recently announced a vision for a “net zero [carbon] NHS,” including a major contribution of new modalities for delivering care (65).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [65] NHS, 2021

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Held's taxonomy divides the ethics of care into personal, political and global.

    “Held's taxonomy (91) divides the ethics of care into personal (focusing on an individual's commitment and accountability to the person they are caring for), political (focusing on various kinds of inequality in caring—such as fairness, human rights, and inclusivity) and global (caring for the planet and its sustainability—and hence caring for future generations).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [91] Held, 2006

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Beauchamp and Childress later added four further principles based on behavioral norms-veracity, privacy, confidentiality, and fidelity.

    “Beauchamp and Childress (87) later added four further principles based on behavioral norms—veracity (tell the truth), privacy (ensure no intruders or eavesdroppers), confidentiality, and fidelity (avoid conflicts of interest such as the potential for personal profit).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [87] Beauchamp, 2019, Principles of biomedical ethics: marking its fortieth anniversary, Am J Bioethics source

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    Normalization includes supporting staff to make sense of the new technology in the context of their work; engaging them to participate; coordinating their implementation efforts; and monitoring benefits and costs.

    “Normalization includes supporting staff to make sense of the new technology in the context of their work; engaging them to participate; coordinating their implementation efforts; and monitoring benefits and costs (23).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [23] May, 2009, Development of a theory of implementation and integration: normalization process theory, Implement Sci. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    These principles were originally proposed by Beauchamp and Childress, and extended to consider scope of application by Gillon.

    “These principles were originally proposed by Beauchamp (who leant toward a consequentialist or utilitarian position, focusing on outcomes of decisions for the patient and others) and Childress (a deontologist focusing more on the clinician's duties) (85), and extended to consider scope of application by Gillon (86).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [85] Beauchamp, 1979

      Not checked in the free run.

    • cites [86] Gillon, 1994, Medical ethics: four principles plus attention to scope, BMJ. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    These barriers are part of a wider digital shift whereby many aspects of life are increasingly presented as "digital first," thereby excluding those unable to access them in this way.

    “These barriers are part of a wider digital shift whereby many aspects of life—insurance, banking, local government, education, travel, and holidays, and many more—are increasingly presented to the citizen, client or customer as “digital first,” thereby excluding (partially or wholly) those unable to access them in this way (75).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [75] Schou, 2019, Digital by default? A qualitative study of exclusion in digitalised welfare, Soc Policy Administr. source

      Not checked in the free run.

  • May combine two claims.

    Study design and conductnone given

    Although research ethics approval was obtained to video and audiotape clinical consultations, this aspect of the work proved impossible in practice.

    “whilst we obtained research ethics approval to video and audiotape clinical consultations, this aspect of the work has so far proved impossible in practice”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Patients randomized to video did no worse clinically and were no less satisfied than those randomized to usual care, and costs where measured were similar.

    “Overall, patients randomized to video did no worse clinically and were no less satisfied than those randomized to usual care, and that costs (where measured) were similar”
    Quote in the paperNot a substantial claim0 of 6 sources checked
    • cites [109] Chongmelaxme, 2019, The effects of telemedicine on asthma control and patients' quality of life in adults: a systematic review and meta-analysis, J Allergy Clin Immunol Pract. source

      Not checked in the free run.

    • cites [110] Gentry, 2019, Geriatric telepsychiatry: systematic review and policy considerations, Am J Geriatr Psychiatry. source

      Not checked in the free run.

    • cites [111] Hong, 2019, Effectiveness of tele-monitoring by patient severity and intervention type in chronic obstructive pulmonary disease patients: a systematic review and meta-analysis, Int J Nurs Stud. source

      Not checked in the free run.

    • cites [112] Kelson, 2019, Internet-delivered acceptance and commitment therapy for anxiety treatment: systematic review, J Med Internet Res. source

      Not checked in the free run.

    • cites [113] McFarland, 2019, The effect of telehealth versus usual care for home-care patients with long-term conditions: a systematic review, meta-analysis and qualitative synthesis, J Telemed Telecare. source

      Not checked in the free run.

    • cites [114] Thiyagarajan, 2020, Exploring patients' and clinicians' experiences of video consultations in primary care: a systematic scoping review, BJGP Open. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    The few detailed real-world studies suggest that efforts to implement remote consultation services even without an ongoing pandemic are logistically complex, resource-intensive and often stymied by technical and regulatory challenges.

    “The few detailed real-world studies in the literature suggest that efforts to implement remote consultation services even without an ongoing pandemic are logistically complex, resource-intensive and often stymied by technical and regulatory challenges (18, 26, 27, 115, 117, 118)”
    Quote in the paperSubstantial claim0 of 6 sources checked
    • cites [18] Greenhalgh, 2018, Analysing the role of complexity in explaining the fortunes of technology programmes: empirical application of the NASSS framework, BMC Med. source

      Not checked in the free run.

    • cites [26] Greenhalgh, 2018, Real-world implementation of video outpatient consultations at macro, meso, and micro levels: mixed-method study, J Med Internet Res. source

      Not checked in the free run.

    • cites [27] Greenhalgh, 2019, Infrastructure revisited: an ethnographic case study of how health information infrastructure shapes and constrains technological innovation, J Med Internet Res. source

      Not checked in the free run.

    • cites [115] Donaghy, 2019, Acceptability, benefits, and challenges of video consulting: a qualitative study in primary care, Br J Gen Pract. source

      Not checked in the free run.

    • cites [117] Odendaal, 2020, Health workers' perceptions and experiences of using mHealth technologies to deliver primary healthcare services: a qualitative evidence synthesis, Cochrane Database Syst Rev source

      Not checked in the free run.

    • cites [118] Wherton, 2020, Guidance on the introduction and use of video consultations during COVID-19: important lessons from qualitative research, BMJ Leader. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Several studies have suggested that double-handling may reduce efficiency.

    “several studies have suggested that double-handling may reduce efficiency (32, 98–102)”
    Quote in the paperNot a substantial claim0 of 6 sources checked
    • cites [32] Turnbull, 2014, The work, workforce, technology and organisational implications of the ‘111' single point of access telephone number for urgent (non-emergency) care: a mixed-methods case study, Health Serv Delivery Res. source

      Not checked in the free run.

    • cites [98] Thompson-Coon, 2013, Telephone consultations in place of face to face out-patient consultations for patients discharged from hospital following surgery: a systematic review, BMC Health Serv Res. source

      Not checked in the free run.

    • cites [99] Downes, 2017, Telephone consultations for general practice: a systematic review, Syst Rev. source

      Not checked in the free run.

    • cites [100] Campbell, 2014, Telephone triage for management of same-day consultation requests in general practice (the ESTEEM trial): a cluster-randomised controlled trial and cost–consequence analysis, Lancet J. source

      Not checked in the free run.

    • cites [101] Newbould, 2019, A ‘telephone first' approach to demand management in English general practice: a multimethod evaluation, Health Serv Deliv Res. source

      Not checked in the free run.

    • cites [102] Salisbury, 2020, The impact of digital-first consultations on workload in general practice: modeling study, J Med Internet Res. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Almost all primary studies of video versus face-to-face appointments were underpowered and had highly-selected samples.

    “almost all primary studies were underpowered and had highly-selected samples”
    Quote in the paperSubstantial claim0 of 6 sources checked
    • cites [109] Chongmelaxme, 2019, The effects of telemedicine on asthma control and patients' quality of life in adults: a systematic review and meta-analysis, J Allergy Clin Immunol Pract. source

      Not checked in the free run.

    • cites [110] Gentry, 2019, Geriatric telepsychiatry: systematic review and policy considerations, Am J Geriatr Psychiatry. source

      Not checked in the free run.

    • cites [111] Hong, 2019, Effectiveness of tele-monitoring by patient severity and intervention type in chronic obstructive pulmonary disease patients: a systematic review and meta-analysis, Int J Nurs Stud. source

      Not checked in the free run.

    • cites [112] Kelson, 2019, Internet-delivered acceptance and commitment therapy for anxiety treatment: systematic review, J Med Internet Res. source

      Not checked in the free run.

    • cites [113] McFarland, 2019, The effect of telehealth versus usual care for home-care patients with long-term conditions: a systematic review, meta-analysis and qualitative synthesis, J Telemed Telecare. source

      Not checked in the free run.

    • cites [114] Thiyagarajan, 2020, Exploring patients' and clinicians' experiences of video consultations in primary care: a systematic scoping review, BJGP Open. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Qualitative studies had indicated high patient and staff satisfaction but frequent technical problems.

    “Qualitative studies had indicated high patient and staff satisfaction but frequent technical problems (113–115)”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [113] McFarland, 2019, The effect of telehealth versus usual care for home-care patients with long-term conditions: a systematic review, meta-analysis and qualitative synthesis, J Telemed Telecare. source

      Not checked in the free run.

    • cites [114] Thiyagarajan, 2020, Exploring patients' and clinicians' experiences of video consultations in primary care: a systematic scoping review, BJGP Open. source

      Not checked in the free run.

    • cites [115] Donaghy, 2019, Acceptability, benefits, and challenges of video consulting: a qualitative study in primary care, Br J Gen Pract. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Observational studies documented a sharp increase in remote consultations (telephone, e- and video, in that order) during the first wave in the UK.

    “Observational studies documented a sharp increase in remote consultations (telephone, e- and video, in that order) during the first wave in the UK (14, 15)”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [14] Murphy, 2021, Implementation of remote consulting in UK primary care following the COVID-19 pandemic: a mixed-methods longitudinal study, Br J Gen Pract. source

      Not checked in the free run.

    • cites [15] Joy, 2020, Reorganisation of primary care for older adults during COVID-19: a cross-sectional database study in the UK, Br J Gen Pract. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    There was very little research on e-consultations prior to the pandemic, and findings were limited.

    “There was very little research on e-consultations prior to the pandemic, and findings were limited (12, 103–107)”
    Quote in the paperNot a substantial claim0 of 6 sources checked
    • cites [12] Brant, 2016, Using alternatives to face-to-face consultations: a survey of prevalence and attitudes in general practice, Br J Gen Pract. source

      Not checked in the free run.

    • cites [103] Huxley, 2015, Digital communication between clinician and patient and the impact on marginalised groups: a realist review in general practice, Br J Gen Pract. source

      Not checked in the free run.

    • cites [104] Banks, 2018, Use of an electronic consultation system in primary care: a qualitative interview study, Br J Gen Pract. source

      Not checked in the free run.

    • cites [105] Huygens, 2018, Understanding the use of email consultation in primary care using a retrospective observational study with data of Dutch electronic health records, BMJ Open. source

      Not checked in the free run.

    • cites [106] Edwards, 2017, Use of a primary care online consultation system, by whom, when and why: evaluation of a pilot observational study in 36 general practices in South West England, BMJ Open. source

      Not checked in the free run.

    • cites [107] Marshall, 2018, Online consulting in general practice: making the move from disruptive innovation to mainstream service, BMJ. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The Delphi exercise to develop guiding principles for applying the PERCS framework was undertaken on a relatively small sample.

    “The Delphi exercise to develop guiding principles for applying the PERCS framework was undertaken on a relatively small sample”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationnone given

    PERCS is an explanatory framework, not a deterministic or predictive tool.

    “PERCS is an explanatory framework, not a deterministic or predictive tool”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Study design and conductown data

    The combined empirical dataset used to refine and extend the frameworks was large and diverse, comprising several pre-pandemic studies, a government-funded evaluation and two in-pandemic research studies, with primary data from surveys, ethnographic observations, interviews, focus groups, cross-sectoral workshops, and a Delphi panel.

    “the combined empirical dataset which we used to refine and extend those frameworks was large and diverse; it included several pre-pandemic studies as well as a government-funded evaluation and two in-pandemic research studies, incorporating extensive primary data from surveys, ethnographic observations, interviews, focus groups, cross-sectoral workshops, and a Delphi panel”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The empirical datasets used (surveys, ethnographic observations, interviews, focus groups, cross-sectoral workshops, and a Delphi panel) adequately capture the phenomena needed to develop and refine the PERCS framework.

  • May combine two claims.

    Interpretationreasoning

    Findings from the authors' in-pandemic research on remote consultation services contrast sharply with pre-pandemic research on such services, which was incomplete, skewed and lacking granularity.

    “Findings from our in-pandemic research on remote consultation services contrast sharply with research on such services undertaken before the pandemic which—in retrospect—was incomplete, skewed and lacking granularity”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisenone given

    The previous frameworks for organizational and technological innovation have been widely used and have stood the test of time.

    “these have been widely-used and stood the test of time”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [17] Greenhalgh, 2017, Beyond adoption: a new framework for theorizing and evaluating nonadoption, abandonment, and challenges to the scale-up, spread, and sustainability of health and care technologies, J Med Internet Res. source

      Not checked in the free run.

    • cites [62] Greenhalgh, 2004, Diffusion of innovations in service organizations: systematic review and recommendations, Milbank Q. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    One quantitative survey followed up 756 e-consultations in general practice and found that most generated either a telephone (32%) or face-to-face (38%) consultation.

    “One quantitative survey followed up 756 e-consultations in general practice and found that most generated either a telephone (32%) or face-to-face (38%) consultation (108)”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [108] Farr, 2018, Implementing online consultations in primary care: a mixed-method evaluation extending normalisation process theory through service co-production, BMJ Open. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Previous research on telephone consultations is sparse and supports no firm conclusions.

    “Previous research on telephone consultations is surprisingly sparse and supports no firm conclusions”
    Quote in the paperSubstantial claim0 of 6 sources checked
    • cites [32] Turnbull, 2014, The work, workforce, technology and organisational implications of the ‘111' single point of access telephone number for urgent (non-emergency) care: a mixed-methods case study, Health Serv Delivery Res. source

      Not checked in the free run.

    • cites [98] Thompson-Coon, 2013, Telephone consultations in place of face to face out-patient consultations for patients discharged from hospital following surgery: a systematic review, BMC Health Serv Res. source

      Not checked in the free run.

    • cites [99] Downes, 2017, Telephone consultations for general practice: a systematic review, Syst Rev. source

      Not checked in the free run.

    • cites [100] Campbell, 2014, Telephone triage for management of same-day consultation requests in general practice (the ESTEEM trial): a cluster-randomised controlled trial and cost–consequence analysis, Lancet J. source

      Not checked in the free run.

    • cites [101] Newbould, 2019, A ‘telephone first' approach to demand management in English general practice: a multimethod evaluation, Health Serv Deliv Res. source

      Not checked in the free run.

    • cites [102] Salisbury, 2020, The impact of digital-first consultations on workload in general practice: modeling study, J Med Internet Res. source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    The studies involving human participants were reviewed and approved by NHS East Midlands Leicester Central Research Ethics Committee (REC ref 20/EM0128; IRAS ID: 283196 and subsequent amendments).

    “The studies involving human participants were reviewed and approved by NHS East Midlands Leicester Central Research Ethics Committee (REC ref 20/EM0128; IRAS ID: 283196 and subsequent amendments)”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Staff later reported feeling tired and under pressure as well as concerned about the loss of in-person care, threats to the therapeutic relationship, potential for missed diagnoses especially in deprived and vulnerable groups, and a rising backlog of unmet need.

    “they later reported feeling tired and under pressure as well as concerned about the loss of in-person care, threats to the therapeutic relationship, potential for missed diagnoses especially in deprived and vulnerable groups, and a rising backlog of unmet need”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [14] Murphy, 2021, Implementation of remote consulting in UK primary care following the COVID-19 pandemic: a mixed-methods longitudinal study, Br J Gen Pract. source

      Not checked in the free run.

    • cites [15] Joy, 2020, Reorganisation of primary care for older adults during COVID-19: a cross-sectional database study in the UK, Br J Gen Pract. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

    Interpretationreasoning

    Many interacting factors need to be taken into account, and the fortunes of remote consultation programs will unfold differently in different circumstances.

    “Many interacting factors need to be taken into account, and the fortunes of remote consultation programs will unfold differently in different circumstances”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Other studies have described problems of increased antimicrobial prescribing, reduced ordering of diagnostic investigations and delayed diagnosis of cancer attributed to the pandemic-related shift to remote consultations.

    “Other studies have described problems of increased antimicrobial prescribing (120), reduced ordering of diagnostic investigations (121) and delayed diagnosis of cancer (122, 123) attributed to the pandemic-related shift to remote consultations”
    Quote in the paperNot a substantial claim0 of 4 sources checked
    • cites [120] Han, 2020, Impact of remote consultations on antibiotic prescribing in primary health care: systematic review, J Med Internet Res. source

      Not checked in the free run.

    • cites [121] Crawford, 2021, Requests from primary care for chest X-ray and CA125 measurements during the COVID-19 emergency: an observational study, Clin Med. source

      Not checked in the free run.

    • cites [122] Hiom, 2020

      Not checked in the free run.

    • cites [123] Helsper, 2020, Cancer has not gone away: a primary care perspective to support a balanced approach for timely cancer diagnosis during COVID-19, Eur J Cancer Care source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    In surveys of staff and patients, most described their experience of technology-mediated care as positive but also questioned whether remote clinical care was as good as face-to-face.

    “In surveys of staff and patients, most described their experience of technology-mediated care as “positive” but also questioned whether remote clinical care was as good as face-to-face (3)”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [3] Horton, 2021

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    The best kind of consultation for a particular patient at a particular time, taking account of the needs of other patients and staff, cannot be decided formulaically.

    “The best kind of consultation for a particular patient at a particular time, taking account of the needs of other patients and staff, cannot be decided formulaically”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Staff experienced the pandemic shift to remote consultation as organizationally and professionally challenging but considered it justified for safety reasons.

    “that staff experienced the shift as organizationally and professionally challenging but considered it justified for safety reasons”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [14] Murphy, 2021, Implementation of remote consulting in UK primary care following the COVID-19 pandemic: a mixed-methods longitudinal study, Br J Gen Pract. source

      Not checked in the free run.

    • cites [15] Joy, 2020, Reorganisation of primary care for older adults during COVID-19: a cross-sectional database study in the UK, Br J Gen Pract. source

      Not checked in the free run.

85 of 85

  • The authors state this as an assumption.

    Each of several philosophical ethical lenses (utilitarian, rights-based, fairness/justice, virtue ethics, common good) may prove useful in different scenarios.

    “We were aware of the many different ways in which ethical dilemmas in healthcare may be theorized—including utilitarian, rights-based, fairness or justice, virtue ethics, and the common good (40), and took the view that each of these philosophical lenses might prove useful in different scenarios.”

  • The paper relies on this without support.

    Readiness for a particular innovation generally involves both top and middle-management support, absence of opponents, and a formal assessment of innovation-system fit.

    “Readiness generally involves both top and middle-management support, absence of opponents, and a formal assessment of innovation-system fit (e.g., a business case), though such requirements were often relaxed during the pandemic.”

  • The paper relies on this without support.

    Profit-oriented providers offering only a limited range of remote services and targeted primarily at low-risk patients such as young professionals were part of the landscape before the pandemic, competing with local GP practices.

    “Profit-oriented providers offering only a limited range of remote services and targeted primarily at low-risk patients such as young professionals were part of the landscape before the pandemic, competing with local GP practices.”

  • The paper relies on this without support.

    Policymakers envisaged an efficient, safe, and accessible remote consultation service delivered through state-of-the art digital technologies and implemented via rational allocation criteria and quality standards.

    “Both during and beyond the pandemic, policymakers envisaged an efficient, safe, and accessible remote consultation service delivered through state-of-the art digital technologies and implemented via rational allocation criteria and quality standards.”

  • The paper relies on this without support.

    The empirical datasets used (surveys, ethnographic observations, interviews, focus groups, cross-sectoral workshops, and a Delphi panel) adequately capture the phenomena needed to develop and refine the PERCS framework.

    “the combined empirical dataset which we used to refine and extend those frameworks was large and diverse”
5 of 5

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Evaluation dates extend beyond the stated study period

    Minor

    The study is described as running from June to October 2020, yet the same sentence says its evaluation covered the pandemic through August 2021.

  • PERCS is said to have seven domains, but eight are listed

    Minor

    The list names eight domains, contradicting the stated count of seven. The same count and list recur in the Discussion.

2 of 2

Cites

Flagged first, then by citations

Research works printed but not linked: 17

  1. Hancock, 2020, The Future of Healthcare (Speech, 30th July)
  2. NHS, 2020, Advice on How to Establish a Remote ‘Total Triage' Model in General Practice Using Online Consultations
  3. Wherton, 2021, Evaluation of the Near Me video Consulting Service in Scotland During COVID-19, 2020
  4. 2014, The Forward View Into Action: Planning for 2015/16
  5. 2015, A Framework for Ethical Decision Making
  6. NHS, 2020, Advice on Using Video Consultation Systems
  7. NHS, 2021, Delivering a Net Zero NHS
  8. NHS, 2014, Five Year Forward View
  9. NHS, 2019, NHS Long Term Plan
  10. UK, 2019, The Digital Transformation Portfolio
  11. 2019, Exploring the UK's Digital Divide
  12. 2020, Implementing Phase 3 of the NHS Response to the COVID-19 Pandemic
  13. NHS, 2020, How We Can Support Digital Inclusion
  14. Stone, 2020, Digital Inclusion in Health and Care: Lessons Learned from the NHS Widening Digital Participation Programme
  15. Fisher, 1990, Toward a feminist theory of caring, 35
  16. 2001, Crossing the Quality Chasm: A New Health System for the 21st Century
  17. Hiom, 2020, How Coronavirus Is Impacting Cancer Services in the UK

Cited by

Sorted by citations

Related

Authors
Greenhalgh, T., Rosen, R., Shaw, S. E., Byng, R., Faulkner, S., Finlay, T., Grundy, E., Husain, L., Hughes, G., Leone, C., Moore, L., Papoutsi, C., Pope, C., Rybczynska-Bunt, S., Rushforth, A., Wherton, J., Wieringa, S., & Wood, G. W.
Affiliations
  • University of Oxford (GB)
  • Nuffield Trust (GB)
  • University of Plymouth (GB)
Corresponding author
Greenhalgh, Trisha
ORCID
Venue
Frontiers in Digital Health
Publisher
Frontiers Media SA
Published
2021-08-13
Volume
3
Pages
726095-726095
ISSN
2673-253X (ISSN-L)
DOI
10.3389/fdgth.2021.726095
PMC
PMC8521880
OpenAlex
W3193547041
Cited by
173
References
111
Citations by year
2026: 25 · 2025: 34 · 2024: 33 · 2023: 34 · 2022: 42 · 2021: 5
FWCI
24.70
Type
journal article
Language
en
Open access
Licence
Locations
  • Frontiers in Digital Health (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, other-oa)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • PEARL (University of Plymouth) (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • PEARL (University of Plymouth) (submittedVersion)
Funders
  • Wellcome Trust (WT104830MA)
  • UK Research and Innovation (ES/V010069/1)
  • National Institute for Health and Care Research (BRC-1215–20008, NIHR202396)
  • Economic and Social Research Council (ES/V010069/1)
Topics
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Telemedicine and Telehealth Implementation
  • Social Sciences › Business, Management and Accounting › Organizational Behavior and Human Resource Management › Healthcare Systems and Technology
  • Health Sciences › Health Professions › General Health Professions › Healthcare Systems and Practices
Keywords
teleconsultation, telemedicine, clinical ethics, applied ethics, complex systems, digital maturity, digital inclusion, demand management, clinical decision-making, therapeutic relationship, healthcare organizations, COVID-19 pandemic, professional education, patient education

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-06.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-06.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review