CoChat
Sign Up
Cited by 3Artificial Intelligence in Healthcare and EducationPharmaceutical Practices and Patient OutcomesElectronic Health Records Systems

Comparative performance of large language models and Drugs.com versus Lexicomp for antiseizure medication drug–drug interactions: A cross-sectional study with iterative prompting analysis

Open access
Mohammed, Maazuddin, Khan, Mohammed Amer, Chaudhary, Vaibhav, Alsaleh, Hamzeh, Fatima, Syeda Bushra, Pal, Biplab·2026·Exploratory Research in Clinical and Social Pharmacy
DOI 10.1016/j.rcsop.2026.100733PMC PMC13081677
Open this paper in CoChat — free account

Abstract

Antiseizure medications (ASMs) are frequently co-prescribed and are associated with a high risk of clinically significant drug–drug interactions (DDIs). Large language models (LLMs) are increasingly used for clinical queries, yet their performance in detecting ASM-related DDIs compared with established drug interaction databases remains uncertain. A cross-sectional comparative study evaluated 186 ASM–comedication pairs (126 classified as major/moderate by Lexicomp) using ChatGPT-4, DeepSeek-V3, and Drugs.com , with Lexicomp as the reference standard. Interactions were assessed for presence, severity, mechanism, and management recommendations. Major and moderate interactions were classified as clinically relevant for performance analysis. A post hoc iterative prompting analysis was conducted exclusively on false positive cases to assess improvements in specificity. Performance metrics included sensitivity, specificity, predictive values, and overall accuracy. Two clinical pharmacists independently assessed outputs for accuracy, clarity, and completeness, with adjudication by a third pharmacist. Drugs.com demonstrated the best overall performance, with sensitivity 0.870, specificity 0.629, and accuracy 0.709. ChatGPT showed high sensitivity (0.842) but low specificity (0.358), reflecting frequent over prediction of interactions. Low specificity reflected frequent overclassification of non-interacting or minor pairs as clinically relevant, potentially leading to alert fatigue, unnecessary monitoring or therapeutic modification. DeepSeek achieved the highest sensitivity (0.877) but the lowest specificity (0.200) and the greatest number of false positives. Iterative prompting substantially improved specificity for ChatGPT (0.77) and DeepSeek (0.42), correcting many false positive classifications. ChatGPT and DeepSeek provided broad interaction overviews but demonstrated limited specificity under single-prompt conditions. Structured, evidence-constrained prompting improved specificity in false positive cases within this dataset. Drugs.com showed the most balanced performance compared with zero-shot LLM outputs. At present, LLMs may serve as supplementary explanatory tools, whereas structured drug interaction databases remain more reliable for primary DDI screening. • Drugs.com showed the highest accuracy among tested tools for AED interactions in primary analysis. • Initial, single-prompt queries with ChatGPT and DeepSeek produced a high rate of false positives due to low clinical specificity. • Iterative (multi-step) prompting significantly improved the specificity of both ChatGPT and DeepSeek, correcting many false positives. • Iterative prompting is key to improving clinical precision and reducing the over-prediction of interactions by large language models. • Evaluators rated Drugs.com highest in completeness and ChatGPT best in clarity. • AI tools are useful for support, but require iterative, evidence-focused prompting to increase precision and should not be relied upon for independent clinical use.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.rcsop.2026.100733

Cites

20 of 25 references verified4 citation errors · 1 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • In another study, ChatGPT-3.5 was less accurate when the required answers were not based on publicly available information.

    “whilst in another study, ChatGPT-3.5 was less accurate when the required answers were not based on publicly available information.91”

    From Large Language Models in Pharmacist-Led Direct Patient Care: A Rapid Review of Published Evaluations (2026) · cites this paper as [91]

    The Methods (Data collection procedure; page unavailable) say the authors chose ChatGPT-4 because it had higher sensitivity and specificity than ChatGPT-3.5. The paper does not say ChatGPT-3.5 was less accurate specifically when answers depended on information that was not publicly available; searches for that condition found no supporting passage. (LLM, conf=0.77)

    Experimental — accuracy not yet measured

  • ChatGPT-3.5 performed adequately in providing medication information but had poor performance in recognizing adverse drug reactions and on high-complexity tasks.

    “In other studies, ChatGPT-3.5 was reported to have performed adequately in providing medication information, 32,55 but had poor performance in J o u r n a l P r e -p r o o f recognizing adverse drug reactions and similar poor performance on high-complexity tasks.55,91,92”

    From Large Language Models in Pharmacist-Led Direct Patient Care: A Rapid Review of Published Evaluations (2026) · cites this paper as [91]

    The Methods (Study design and Data collection procedure; page unavailable) specify that the study evaluated ChatGPT-4, not ChatGPT-3.5. The Results report performance on antiseizure-medication drug interactions, but no passage found assesses ChatGPT-3.5’s provision of medication information, recognition of adverse drug reactions, or performance on high-complexity tasks. (LLM, conf=0.78)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. [11]Kluwer Wolters Lexicomp Online, Wolters Kluwer Clinical Drug Information, Hudson, OH https://www.lexicomp.com 2025 (accessed 11 June 2025)

Cited by

Sorted by citations

Related

Authors
Mohammed, M., Khan, M. A., Chaudhary, V., Alsaleh, H., Fatima, S. B., & Pal, B.
Affiliations
  • Lovely Professional University (IN)
  • Hamad Medical Corporation (QA)
Corresponding author
Pal, Biplab
ORCID
Venue
Exploratory Research in Clinical and Social Pharmacy
Publisher
Elsevier BV
Published
2026-06
Volume
22
Pages
100733
ISSN
2667-2766 (ISSN-L)
DOI
10.1016/j.rcsop.2026.100733
PMC
PMC13081677
OpenAlex
W7140309989
Cited by
2
References
18
Citations by year
2026: 2
FWCI
3.93
Type
journal article
Language
en
Open access
Licence
Locations
  • Exploratory Research in Clinical and Social Pharmacy (publishedVersion, cc-by-nc-nd)
  • PubMed Central (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Medicine › Geriatrics and Gerontology › Pharmaceutical Practices and Patient Outcomes
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
Keywords
antiepileptic drugs, drug-drug interactions, large language models, ChatGPT, DeepSeek, Drugs.com, Lexicomp, iterative prompting, false positives, specificity, alarm fatigue, drug interaction databases, co-prescription, clinical decision support, pharmacovigilance

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review