Cited by 104Artificial Intelligence in Healthcare and EducationElectronic Health Records SystemsMachine Learning in Healthcare

How Chatbots and Large Language Model Artificial Intelligence Systems Will Reshape Modern Medicine: Fountain of Creativity or Pandora’s Box?

Open access
Li, Ron, Kumar, Andre, Chen, Jonathan H.·2023·JAMA Internal Medicine

Abstract

In an era of clinicians being burned out by electronic medical records and documentation burdens, we might all dream of having a personal scribe to draft progress notes, translate patient instructions, summarize the literature, complete insurance authorization paperwork, and respond to unending in-basket messages, as described in the Perspective in this issue of JAMA Internal Medicine. 1 This would have sounded like a fantasy just a few years ago, but the release of rapidly developing chatbots now demonstrates the potential of large language model artificial intelligence (AI) systems with surprisingly adept language manipulation and knowledge processing capabilities.The underlying foundation model technology rides atop the peak of inflated expectations, 2 reflecting a disruptive technology likely to change the way we work and live, even as we must be aware of substantial limitations.Good or bad, ready or not, Pandora's box has already been opened.One such large language model, ChatGPT, is the fastest-growing internet application in history with more than 100 million users.3 This has shifted access to sophisticated AI capabilities away from concentrated pockets of technical experts to the masses, where all types of otherwise unimaginable (and unintended) use cases are being discovered.To ensure that the adoption of such tools into health care practice is done effectively and responsibly, physicians must lean in to understand and drive this conversation.Large language models represent the underlying class of machine learning models trained in autocomplete tasks.Given the words "coronary artery," these models may predict the next word to be "disease," "bypass graft," or "calcification" based on statistical parameters learned from prior training data text on how often those words appear together.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1001/jamainternmed.2023.1835

Cites

6 of 6 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The effective deployment of AI agents is hindered by several technical barriers, including efficiency issues, quality of outputs, context length limitations, unexpected behaviors, and network reliability.

    “The effective deployment of AI agents is hindered by several technical barriers. Efficiency remains a significant issue, as some tasks require extended processing times, impeding timely decision-making in critical medical scenarios.130”

    From A foundational architecture for AI agents in healthcare (2025) · cites this paper as [130]

    The Abstract discusses fabricated or inaccurate chatbot responses, but it does not address slow processing or delayed decisions in critical medical scenarios. A search of our copy for efficiency, processing time, and latency found no passage supporting the claim attached to this citation. (LLM, conf=0.76)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Li, R., Kumar, A., & Chen, J. H.
Affiliations
Stanford Medicine (US)
Venue
JAMA Internal Medicine
Published
2023-06-01
Volume
183
Issue
6
Pages
596
ISSN
2168-6106 (ISSN-L)
OpenAlex
W4367311208
Cited by
104
References
6
Citations by year
2026: 11 · 2025: 30 · 2024: 34 · 2023: 29
FWCI
4.15
Type
journal article
Language
en
Open access
Locations
  • JAMA Internal Medicine (publishedVersion)
  • PubMed (publishedVersion)
  • PubMed Central (acceptedVersion)
Funders
  • American Heart Association (COVID-19)
  • Google
  • National Institutes of Health (COVID-19)
  • National Institute on Drug Abuse (UG1DA015815)
  • National Institute of Allergy and Infectious Diseases (R01AI178121)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
Keywords
chatbots, large language models, artificial intelligence in healthcare, ChatGPT, clinical documentation burden, electronic health records, physician burnout, AI adoption in healthcare, clinical workflow automation
MeSH terms
Artificial Intelligence, Humans

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review