CoChat
Sign Up
Cited by 48Electronic Health Records SystemsArtificial Intelligence in Healthcare and EducationClinical Reasoning and Diagnostic Skills

‘Too much, too late’: mixed methods multi-channel video recording study of computerized decision support systems and GP prescribing

Open access
Hayward, James, Thomson, Fionagh, Milne, Heather, Buckingham, Susan, Sheikh, Aziz, Fernando, Bernard, Cresswell, Kathrin, Williams, Robin +1 more·2013·Journal of the American Medical Informatics Association
DOI 10.1136/amiajnl-2012-001484PMC PMC3715350
Open this paper in CoChat — free accountOpen PDF

Abstract

OBJECTIVE: Computerized decision support systems (CDSS) are commonly deployed to support prescribing, although over-riding of alerts by prescribers remains a concern. We aimed to understand how general practitioners (GPs) interact with prescribing CDSS in order to inform deliberation on how better to support prescribing decisions in primary care. MATERIALS AND METHODS: Quantitative and qualitative analysis of interactions between GPs, patients, and computer systems using multi-channel video recordings of 112 primary care consultations with eight GPs in three UK practices. RESULTS: 132 prescriptions were issued in the course of 73 of the consultations, of which 81 (61%) attracted at least one alert. Of the total of 117 alerts, only three resulted in the GP checking, but not altering, the prescription. CDSS provided information and safety alerts at the point of generating a prescription. This was 'too much, too late' as the majority of the 'work' of prescribing occurred prior to using the computer. By the time an alert appeared, the GP had formulated the problem(s), potentially spent several minutes considering, explaining, negotiating, and reaching agreement with the patient about the proposed treatment, and had possibly given instructions and printed an information leaflet. DISCUSSION: CDSS alerts do not coincide with the prescribing workflow throughout the whole GP consultation. Current systems interrupt to correct decisions that have already been taken, rather than assisting formulation of the management plan. CONCLUSIONS: CDSS are likely to be more acceptable and effective if the prescribing support is provided much earlier in the process of generating a prescription.

AI snapshot

Written from the paper’s full text.

TL;DR

In UK primary care, computerized decision support alerts usually appeared after GPs had already formulated, negotiated, and agreed a prescribing decision, so they were rarely acted on. The authors argue CDSS should support prescribing much earlier in the consultation workflow rather than interrupting at the point of generating a prescription.

Key findings

  • Of 132 prescriptions issued across 73 of 112 recorded consultations, 81 (61%) attracted at least one alert; there were 117 alerts in total (55 safety alerts, 57 process warnings, 5 add-ins), and only 3 (2%) resulted in the GP checking—but not altering—the prescription.
  • Among 62 new acute prescriptions in 49 consultations, therapeutic talk accounted for 40% of all GP utterances, with 53% occurring before the computer was used to process the prescription and 72% before the prescription was printed; patient therapeutic talk was 16% of conversation, with 79% before printing.
  • Qualitative analysis identified four prescribing tasks—formulating the problem, negotiating a proposed medication, reaching agreement, and instructing the patient—that generally occurred before computer prescribing, making alerts “too much, too late.”

Methods

  • Mixed methods multi-channel video recording of 112 primary care consultations with 8 GPs in 3 UK practices using EMIS or INPS Vision; whole clinics were recorded during two site visits approximately 6 months apart.
  • Quantitative analysis used OBSWIN to time computer tasks, Roter Interaction Analysis System coding for therapeutic talk, and independent classification of alerts and GP responses; qualitative analysis coded verbatim transcripts of consultations containing new acute prescriptions.
  • Detailed analysis focused on 62 new acute prescriptions from 49 consultations.

Limitations

  • Stated by the paper: the eight participating GPs may not be representative; willingness to be videoed may reflect confidence in consulting and computer use; the study may not capture the full range of prescribing scenarios; and the study was UK-based, although alert fatigue is an international concern.
  • Evident from the text: the sample is small and limited to three practices and two clinical software systems, and the observational video method could affect consultation behavior, although the authors cite evidence that recording does not significantly alter the consultation process.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1136/amiajnl-2012-001484

Cites

50 of 50 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 23 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    The CDSS delivered information and safety alerts at the moment a prescription was being generated.

    “CDSS provided information and safety alerts at the point of generating a prescription.”
    Quote in the paperSubstantial claim
  • Findingown data

    Most of the work involved in prescribing took place before the GP used the computer.

    “This was 'too much, too late' as the majority of the 'work' of prescribing occurred prior to using the computer.”
    Quote in the paperSubstantial claim
  • Findingown data

    New acute copied prescriptions had 11 interruptions (69%).

    “| Number of prescriptions interrupted by an alert or warning, n | 31 (50) | 11 (69) | 3 (21) | 4 (14) | 0 | 49 |”
    Quote in the paperSubstantial claim
  • Findingown data

    Background information about known allergies, interactions, or adverse events was presented for every new prescription.

    “In addition, background information about the presence/absence of known allergies, interactions, or adverse events was presented for all new prescriptions.”
    Quote in the paperNot a substantial claim
  • Findingown data

    Re-authorize repeat prescriptions had 4 interruptions (14%).

    “| Number of prescriptions interrupted by an alert or warning, n | 31 (50) | 11 (69) | 3 (21) | 4 (14) | 0 | 49 |”
    Quote in the paperNot a substantial claim
  • May combine two claims. 20
  • Classified as probably not a substantial claim. 19

  • May combine two claims.

    Findingown data

    When an alert appeared, the GP had already formulated the problem, and may have spent time considering, explaining, negotiating, and agreeing the treatment with the patient, and possibly given instructions and printed an information leaflet.

    “By the time an alert appeared, the GP had formulated the problem(s), potentially spent several minutes considering, explaining, negotiating, and reaching agreement with the patient about the proposed treatment, and had possibly given instructions and printed an information leaflet.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Of the 117 alerts recorded, only three (2%) led the GP to check prescribing information, and in none of these cases did the GP change the prescription.

    “Only three (2%) of the alerts resulted in apparent action on the part of the GP: in each case the GP checked the prescribing information (in a formulary or computer database) but did not subsequently alter the prescription.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    In the study, 132 prescriptions were issued during 73 of the 112 consultations, and 81 of those prescriptions (61%) triggered at least one alert.

    “Overall, 132 prescriptions were issued in the course of 73 of the consultations, of which 81 (61%) attracted at least one alert.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Four tasks were identified as forming the work of prescribing: formulating the problem so a prescription was justified, negotiating a proposed medication, reaching agreement on treatment, and instructing the patient on using the medication.

    “we identified four tasks which formed the 'work' of prescribing: (i) formulating the problem such that a prescription was justified; (ii) negotiating a proposed medication; (iii) reaching agreement on the treatment; and (iv) instructing the patient on using the medication.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingown data

    The subsequent analysis focused on 62 new acute prescriptions from 49 consultations.

    “Our subsequent analysis focuses on the 62 new acute prescriptions observed in the course of 49 consultations.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning

    By the time an alert appears, the GP has already formulated the problem, may have spent several minutes considering, explaining, negotiating and reaching agreement with the patient, possibly given instructions, and printed an information leaflet about the proposed treatment.

    “By the time an alert appears the GP will have formulated a problem, potentially spent several minutes considering, explaining, negotiating, and reaching agreement with the patient, possibly given instructions, and printed an information leaflet about the proposed treatment.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Of the 117 alerts, 55 were safety alerts, 57 were process warnings, and five were generated by ScriptSwitch.

    “Of the total of 117 alerts, 55 were safety alerts and 57 were process warnings, with an further five generated by 'ScriptSwitch' (an additional system funded by the local healthcare trust to advise on cost-effective prescribing) in one of the practices.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    There were 5 add-ins.

    “| Add-ins | 3 | 0 | 2 | 0 | 0 | 5 |”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Multiple alerts occurred in 29 prescriptions (22%), and five prescriptions were interrupted by three or more alerts.

    “Multiple alerts occurred in 29 (22%) prescriptions, with five interrupted by three or more alerts”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Therapeutic talk made up 16% of the patient's conversation, and 79% of it occurred before the prescription was printed.

    “Similarly, therapeutic talk contributed 16% of the patient's conversation, with 79% occurring before the prescription was printed.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Three GPs in one practice used INPS Vision software, and five GPs in two practices used EMIS.

    “Three GPs in one practice (Seaside) used INPS Vision software and five GPs in two practices (Church and Hills) used EMIS”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    Exploring previous episodes is often part of history taking.

    “exploration of which is often part of history taking.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Therapeutic talk made up 40% of all GP utterances in consultations that resulted in a new acute prescription; 53% of this talk occurred before the computer was used to process the prescription, and 72% occurred before the prescription was printed.

    “Therapeutic talk accounted for 40% of all GP utterances in consultations resulting in a new acute prescription, 53% of which occurred before the computer was used to process the prescription and 72% before the prescription was printed.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    These four tasks were present, implicitly or explicitly, in every consultation involving a new acute prescription, though they varied in duration, order, and could be interspersed with other tasks.

    “These tasks were present (implicitly or explicitly) in all the consultations involving a new acute prescription, although they varied in the time they took, did not occur in any particular order, and could be interspersed with other tasks within the consultation.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The study recorded 132 prescriptions.

    “| Number of prescriptions | 62 | 16 | 14 | 28 | 12 | 132 |”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    The CDSS alerts were too much and came too late in the prescribing process.

    “This was 'too much, too late' as the majority of the 'work' of prescribing occurred prior to using the computer.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Alerts that give the prescriber a choice and require a reason for deviating from protocol may be more effective than alerts that can simply be over-ridden.

    “Alerts that offer the prescriber choice, and require reasons for deviation from protocol may be more effective than those that can simply be over-ridden.20 26”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [20] van der Sijs, 2006, Overriding of drug safety alerts in computerized physician order entry, J Am Med Inform Assoc source

      Not checked in the free run.

    • cites [26] Hasley, 2011, Decision support and patient safety: the time has come, Am J Obstet Gynecol source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    The challenge of implementing effective decision support for prescribing, and specifically alert fatigue, is international.

    “Our study was UK-based, although the challenge of implementing effective decision support for prescribing, and specifically the problem of alert fatigue, is international.12 24 50”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [12] Kuperman, 2007, Medication-related clinical decision support in computerized provider order entry systems: a review, J Am Med Inform Assoc source

      Not checked in the free run.

    • cites [24] Phansalkar, 2010, A review of human factors principles for the design and implementation of medication safety alerts in clinical information systems, J Am Med Inform Assoc source

      Not checked in the free run.

    • cites [50] Ludwick, 2009, Adopting electronic medical records in primary care: Lessons learned from health information systems implementation experience in seven countries, Int J Med Inform source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    CDSS is used universally in UK primary care and now includes warnings about real and potential adverse drug effects.

    “CDSS is universal in UK primary care and has broadened to include warnings of real and potential adverse drug effects.5 8”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [5] Duerden, 2011

      Not checked in the free run.

    • cites [8] Kaushal, 2003, Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review, Arch Intern Med source

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductnone given

    All consultations were transcribed verbatim, and transcripts containing new acute prescriptions were examined for all prescribing-related conversation.

    “All consultations were transcribed verbatim and transcripts of consultations which included one or more new acute prescriptions were examined for all conversation relating to prescribing.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Study design and conductnone given

    The GPs' willingness to be videoed may reflect relatively high confidence in their consulting skills and computer use.

    “Their willingness to be videoed may reflect relative confidence in their consulting skills and computer use, although the particular task of interest (computer prescribing) is routine within primary care.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationown data

    The study's data, from observation of real-life consultations, can help designers and users understand how to align CDSS functionality with the GP perspective on prescribing work.

    “Our data, from observation of real-life consultations, can help designers and users understand how to align CDSS functionality with the GP perspective on the work of prescribing.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    Problems such as inaccurate allergy coding, spurious alerts that frustrate users and prompt inappropriate clinical responses, and workarounds that bypass safety features have been identified.

    “inaccurate coding of allergies, 9 spurious alerts incurring user frustration and inappropriate clinical responses, 10 19 and workarounds which bypass intrinsic safety features 15 have been identified.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [9] Kaelber, 2007, Health information exchange and patient safety, J Biomed Inform source

      Not checked in the free run.

    • cites [10] Karnon, 2007, A prospective hazard and improvement analytic approach to predicting the effectiveness of medication error interventions, Saf Sci source

      Not checked in the free run.

    • cites [19] Cresswell, 2008, Information technology-based approaches to reducing repeat drug exposure in patients with known drug allergies, J Allergy Clin Immunol source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    All prescribing-related dialogue was coded using a framework developed inductively by two researchers.

    “All dialogue related to prescribing was coded into a framework developed inductively by JH and HP (see online supplementary table S2).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    An alert appearing in the final seconds of generating the paper prescription is likely to be seen as intrusive and unwelcome, and raises the chance that it will be ignored.

    “An alert in the final seconds of the task of generating the paper prescription is likely to be regarded as intrusive and unwelcome, and increases the probability of it being ignored.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductcitation

    The paper provided descriptions of practices, clinicians, computer systems, patients, and the prescribing process using the two computer systems to allow assessment of transferability to other healthcare contexts.

    “To enable assessment of transferability to other healthcare contexts, 49 we have provided a description of the practices, clinicians, and computer systems (table 2) and characteristics of the patients (table 4) and the process of prescribing using the two computer systems (table 6).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [49] Murphy, 1998

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    Established methodology was used for multi-channel video recordings of consultations.

    “We used established methodology to undertake multi-channel video (MCV) recordings of consultations.[42][43][44]”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [42] Pinnock H Cresswell K Fernado B . Evaluation of the effect of IT on interactions between healthcare workers and patients (NHS CFHEP 010) Connecting for Health Evaluation Programme. 2011. http://www.birmingham.ac.uk/research/activity/mds/projects/HaPS/PHEB/CFHEP/reports/projects/010.aspx (accessed Sep 2012).

      Not checked in the free run.

    • cites [43] de Lusignan, 2008, The ALFA (Activity Log Files Aggregation) toolkit: a method for precise observation of the consultation, J Med Internet Res source

      Not checked in the free run.

    • cites [44] de Lusignan, 2011, Reporting observational studies of the use of information technology in the clinical consultation. A position statement from the IMIA Primary Health Care Informatics Working Group (IMIA PCI WG), . Yearb Med Inform

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    Secondary care studies have suggested that alerts should be presented at the point of selecting drugs rather than at the moment of printing.

    “Studies undertaken in a secondary care setting have suggested the importance of presenting alerts at the point of selecting drugs (as opposed to the moment of printing), 22”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [22] Miller, 2005, The anatomy of decision support during inpatient care provider order entry (CPOE): Empirical observations from a decade of CPOE experience at Vanderbilt, J Biomed Inform source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Emerging themes were discussed with the multidisciplinary team.

    “Emerging themes were discussed with the multidisciplinary team.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductcitation

    The Roter Interaction Analysis System (RIAS) is a validated and widely used tool that codes verbal utterances into mutually exclusive categories.

    “The validated Roter Interaction Analysis System (RIAS) 45 is a widely used tool which codes the verbal 'utterances' of both the patient and clinician into 41 mutually exclusive categories”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [45] Pflug, 2010, Measuring the impact of the computer on the consultation: an open source application to combine multiple observational outputs, Inform Health Soc Care source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The use of discrete cameras made the recording process as unobtrusive as possible.

    “our use of discrete cameras ensured that the process was as unobtrusive as possible”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    The work of prescribing is tacit, while processing the prescription involves explicit, codified rules and procedures.

    “The work of prescribing is tacit, while that required for processing the prescription involves explicit, codified rules and procedures.52”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [52] Wyatt, 2001, Management of explicit and tacit knowledge, J R Soc Med source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductown data

    Two researchers independently observed the number and type of alerts and the associated dialogue boxes for each prescription.

    “Two researchers ( JH and HP) independently observed the number, type of alert (see table 1 for definitions) and associated dialogue boxes presented by the system in connection with each prescription.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    GP practices were purposively sampled to ensure a spread of demography, size, and clinical software.

    “We purposively sampled GP practices to ensure a spread of demography, size, and clinical software, and invited up to three GPs in each practice to participate.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductown data

    The mixed-method analysis allowed the study both to quantify the issue and to use qualitative analysis to understand the prescribing process that CDSS aims to influence.

    “Our mixed method analysis enabled us both to quantify the issue and to use qualitative analysis to understand the process of prescribing which the CDSS seeks to influence.”
    Quote in the paperSubstantial claim
35 of 35

  • The paper relies on this without support.

    The findings from the eight GPs in three UK practices are representative of GP prescribing with CDSS in primary care more generally.

    “We observed a total of 112 GP consultations from the clinics of eight GPs in three practices (identified by pseudonym in this report) (table 2).”

  • The paper relies on this without support.

    When a GP over-rode an alert with no apparent attention, this indicates that the GP did not attend to the alert.

    “We classified the clinician as having responded to an alert (if they took action such as selecting an alternative drug, changing a dose, checking with the patient, or seeking information from a formulary) or dismissed an alert (if they over-rode it with no apparent attention).”

  • The paper relies on this without support.

    Computer prescribing is routine within primary care.

    “although the particular task of interest (computer prescribing) is routine within primary care.”

  • The paper relies on this without support.

    CDSS has become the norm for GP prescribing in the UK and many other countries.

    “CDSS has become the norm for GP prescribing in the UK and many other countries but is concentrated in the form of information and safety alerts issued at the point of prescribing.”
4 of 4

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Conflicting counts of prescriptions with alerts

    Major

    The results say 81 of 132 prescriptions attracted an alert, but Table 3 reports 49 prescriptions interrupted by an alert or warning. The table distinguishes background information from alerts, so its count does not support the reported 81.

  • Patient age-band counts total 46, not 47

    Minor

    The seven age-band counts in Table 4 sum to 46, one fewer than the stated 47 patients.

    Reported 47 · recomputed 46

  • Percentage of alerts prompting action is rounded incorrectly

    Minor

    Three of 117 alerts is 2.56%, which rounds to 3% rather than the reported 2% at whole-percent precision.

    Reported 2 · recomputed 2.564103

3 of 3

Cites

Flagged first, then by citations

5 of the 55 references couldn’t be matched to a known paper.

Research works printed but not linked: 4

  1. Stroetmann, 2007, eHealth for safety. Impact of ICT on patient safety and risk management
  2. Safety and Quality Council, 2002, Second national report on patient safety, improving medication safety
  3. Duerden, 2011, The quality of GP prescribing. An inquiry into the quality of general practice in England
  4. Davies, 2011, English GP systems market

Cited by

Sorted by citations

Related

Authors
Hayward, J., Thomson, F., Milne, H., Buckingham, S., Sheikh, A., Fernando, B., Cresswell, K., Williams, R., & Pinnock, H.
Affiliations
University of Edinburgh (GB)
Corresponding author
Pinnock, Hilary
ORCID
Venue
Journal of the American Medical Informatics Association
Publisher
Oxford University Press (OUP)
Published
2013-03-07
Volume
20
Issue
e1
Pages
e76-e84
ISSN
  • 1067-5027 (ISSN-L)
  • 1527-974X
DOI
10.1136/amiajnl-2012-001484
PMC
PMC3715350
OpenAlex
W2120444843
Semantic Scholar
8f801d09ea4265ea71ec65ae99f3b09d8e8066cb
Cited by
48
References
55
Citations by year
2024: 2 · 2023: 4 · 2022: 1 · 2021: 4 · 2020: 9 · 2019: 5 · 2018: 1 · 2017: 7 · 2016: 1 · 2015: 8 · 2014: 5 · 2013: 1
FWCI
8.63
Type
journal article
Language
en
Open access
Locations
  • Journal of the American Medical Informatics Association (publishedVersion)
  • Europe PMC (PubMed Central) (publishedVersion)
  • PubMed Central (publishedVersion)
  • PubMed (publishedVersion)
  • HighWire Press Open Archive (submittedVersion)
  • Newcastle University ePrints (Newcastle Univesity)
Funders
Scottish Government
Topics
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
Keywords
computerized decision support systems, drug prescribing, general practitioners, clinical alerts, primary care visits, patient-clinician interaction, drug safety
MeSH terms
Drug Therapy, Computer-Assisted, Humans, Physician-Patient Relations, Practice Patterns, Physicians', Task Performance and Analysis, Video Recording, Decision Support Systems, Clinical, Electronic Prescribing

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-09.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-09.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review