Comparing Artificial Intelligence (ChatGPT, Gemini, DeepSeek) and Oral Surgeons in Detecting Clinically Relevant Drug–Drug Interactions in Dental Therapy

May combine two claims.

Large language models such as ChatGPT, Gemini and DeepSeek could rapidly analyze medication lists and cross-reference known interactions, providing comprehensive and standardized DDI screening at high speed and potentially streamlining preoperative prescribing workflows.

“Large language models (LLMs) such as Chat-GPT, Gemini and specialized systems like DeepSeek could rapidly analyze medication lists and cross-reference known interactions, providing comprehensive and standardized DDI screening at high speed and potentially streamlining preoperative prescribing workflows [29][30][31][32].”

• cites [29] Wang, 2025, Accuracy of Large Language Models When Answering Clinical Research Questions: Systematic Review and Network Meta-Analysis, J. Med. Internet Res. sourcePartly supported

  Split into 2 atoms — 0 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 1 unverifiable. Expand the claim to see per-atom verdicts.

  Experimental — accuracy not yet measured
• cites [30] Zitu, M.M., Owen, D., Manne, A., Wei, P., and Li, L. (2025). Large Language Models for Adverse Drug Events: A Clinical Perspective. J. Clin. Med., 14. sourcePartly supported

  Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

  Experimental — accuracy not yet measured
• cites [31] Hung, 2024, Evaluating the capability of ChatGPT in predicting drug-drug interactions: Real-world evidence using hospitalized patient data, Br. J. Clin. Pharmacol. sourceDisputed

  Split into 2 atoms — 0 confirmed, 1 partially supported, 1 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

  Checked by a reasoning model — may be wrong.

  Experimental — accuracy not yet measured
• cites [32] Sicard, J., Montastruc, F., Achalme, C., Jonville-Bera, A.P., Songue, P., Babin, M., Soeiro, T., Schiro, P., de Canecaude, C., and Barus, R. (2025). Can large language models detect drug-drug interactions leading to adverse drug reactions?. Ther. Adv. Drug Saf., 16. sourcePartly supported

  Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S3 (load-bearing).

Ong et al. (2025) identified DDI detection as one of the most promising yet under-validated applications of generative AI for pharmacovigilance.

“Likewise, Ong et al. (2025) [39], in a scoping review of generative AI for pharmacovigilance, identified DDI detection as one of the most promising yet under-validated applications-a limitation our study begins to address within the oral-surgery context.”

• cites Ong et al. (2025) [39] sourcePartly supported

  The Discussion (page unavailable) identifies DDI identification and prediction as a key GenAI application and says few studies reported validation performance; the Results add that none tested models prospectively. However, the review lists DDI work and automation of pharmacovigilance as separate applications, rather than identifying DDI detection specifically as a pharmacovigilance application. (LLM, conf=0.83)

  Experimental — accuracy not yet measured

May combine two claims.

Dentists may lack access to patients' full medication histories, which can result in missed or underestimated DDIs and increase the risk of DDI-related harm.

“Compounding the challenge, dentists may lack access to patients' full medication histories, which can result in missed or underestimated DDIs and increase the risk of DDI-related harm [23][24][25].”

• cites [23] Johnson, 2018, A Pharmacist’s Role in a Dental Clinic: Establishing a Collaborative and Interprofessional Education Site, Innov. Pharm. sourcePartly supported

  The Background (page not provided) reports medication omissions in dental records and says pharmacists can identify potential interactions and adverse effects. The Evaluation (page not provided) records pharmacist interventions for adverse reactions or drug interactions. The paper does not directly establish that incomplete histories caused dentists to miss or underestimate DDIs, or that this increased DDI-related harm. (LLM, conf=0.82)

  Experimental — accuracy not yet measured
• cites [24] Laddha, R. (2025). Bridging the Gap: The Critical Role of Collaboration Between Dentistry and Pharmacy. J. Pharm. Bioallied Sci., 17. sourcePartly supported

  In the main text (page unavailable), the paper describes communication gaps between dentists and pharmacists, pharmacists’ expertise in drug interactions, and the potential to reduce adverse drug reactions. It does not say that dentists lack access to full medication histories or that this causes DDIs to be missed or underestimated. (LLM, conf=0.82)

  Experimental — accuracy not yet measured
• cites [25] Choi, 2017, Medication discrepancies in the dental record and impact of pharmacist-led intervention, Int. Dent. J. sourceSupported

May combine two claims.

Sicard et al. (2025) showed that general-purpose LLMs reached very high detection rates of DDIs but remained unable to reliably exclude non-interactions.

“As shown by Sicard et al. (2025) [32], general-purpose LLMs reached very high detection rates of DDIs but remained unable to reliably exclude non-interactions; the moderate AUC values and relatively low positive predictive values (16-20%) observed here align with that limitation, indicating that most alerts generated by even the best-performing models do not correspond to truly actionable DDIs.”

• cites Sicard et al. (2025) [32] sourceSupported

May combine two claims.

No study has directly compared AI with clinicians in dentistry or oral surgery, fields in which robust DDI data remain scarce.

“While AI-driven DDI checks have been investigated in general medicine (intensive care and pharmacovigilance), no study has directly compared AI with clinicians in dentistry or oral surgery, fields in which robust DDI data remain scarce [35].”

• cites [35] Huang, W., Wang, X., Chen, Y., Yu, C., and Zhang, S. (2025). Advancing drug-drug interactions research: Integrating AI-powered prediction, vulnerable populations, and regulatory insights. Front. Pharmacol., 16. sourceCould not verify

  Section 2.4 discusses AI analysis of pharmacovigilance data, and Section 8.1 discusses AI-enabled DDI prediction generally. Neither addresses dentistry or oral surgery, the availability of DDI data in those fields, or whether a study has directly compared AI with clinicians there. Searches for those terms and comparisons found no passage that verifies the claim. (LLM, conf=0.75)

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S3 (load-bearing).

The study was reported in accordance with the principles of the STARD 2015 guideline, adapted for simulation-based evaluation.

“This study was reported in accordance with the principles of the STARD 2015 guideline, adapted for simulation-based evaluation [36].”

• cites [36] Bossuyt, 2015, STARD2015: An updated list of essential items for reporting diagnostic accuracy studies, BMJ sourceSupported

Probably not a substantial claim: may not pass S3 (load-bearing).

Wang et al. (2025) reported that progressive LLM evolution is narrowing the gap between general and specialised architectures.

“This trend supports the conclusion of Wang et al. (2025) [29], who reported that progressive LLM evolution is narrowing the gap between general and specialised architectures.”

• cites Wang et al. (2025) [29] sourcePartly supported

  The Discussion (Principal Findings; page unavailable) describes rapid LLM development and compares specialised models with general LLMs. It does not report that the performance gap is narrowing; instead, it says newer versions do not necessarily outperform older ones and that some clinically fine-tuned models outperform general LLMs on biomedical tasks. (LLM, conf=0.84)

  Experimental — accuracy not yet measured

May combine two claims.

Studies comparing ChatGPT, DeepSeek and similar models with reference databases have reported only moderate agreement regarding interaction severity, along with frequent inaccuracies in AI-generated recommendations.

“Studies comparing ChatGPT, DeepSeek and similar models with reference databases have reported only moderate agreement regarding interaction severity, along with frequent inaccuracies in AI-generated recommendations [34].”

• cites [34] Lima, 2025, Capacity of ChatGPT, Deepseek, and Gemini in predicting major potential drug interactions in adults within the Intensive Care Unit, J. Hosp. Pharm. Health Serv. sourcePartly supported

  The Discussion (page not provided) reports low agreement with Drugs.com and moderate agreement with Micromedex on interaction severity. It also notes discrepancies in recommended management, but describes them as occurring in some information rather than establishing that inaccurate recommendations were frequent. (LLM, conf=0.82)

  Experimental — accuracy not yet measured

Classified as probably not a substantial claim.

Artificial intelligence has emerged as a promising clinical decision-support tool in dentistry.

“In this context, artificial intelligence (AI) has emerged as a promising clinical decision-support tool in dentistry [26][27][28].”

• cites [26] Samaranayake, 2025, The Transformative Role of Artificial Intelligence in Dentistry: A Comprehensive Overview. Part 1: Fundamentals of AI, and its Contemporary Applications in Dentistry, Int. Dent. J. sourceSupported
• cites [27] Schwendicke, 2020, Artificial Intelligence in Dentistry: Chances and Challenges, J. Dent. Res. sourceSupported
• cites [28] Sitaras, 2025, Applications of Artificial Intelligence in Dental Medicine: A Critical Review, Int. Dent. J. sourceSupported

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Other medication classes often encountered in oral surgery patients that may increase DDI risk include antidiabetic drugs such as GLP-1 agonists and SGLT2 inhibitors, statins, antiepileptics, and antiresorptive osteoporosis drugs associated with medication-related osteonecrosis of the jaw.

“Other medication classes often encountered in oral surgery patients that may increase the risk of DDI include antidiabetic drugs (such as GLP-1 agonists and SGLT2 inhibitors) [13], statins [14], antiepileptics [15,16] and antiresorptive osteoporosis drugs, which are associated with medication-related osteonecrosis of the jaw (MRONJ) [17][18][19][20].”

• cites [13] May, 2016, Clinically and pharmacologically relevant interactions of antidiabetic drugs, Ther. Adv. Endocrinol. Metab. sourcePartly supported

  The GLP-1 analogs section reports a possible additive hypoglycemic effect with sulfonylureas, and the SGLT-2 inhibitors section identifies a few interactions, so these antidiabetic drugs can present DDI risks. The paper also says no clinically relevant drug–drug interactions had been described for exenatide or liraglutide and that SGLT-2 inhibitors generally show none apart from specified exceptions. It does not establish that these drugs are often encountered in oral surgery patients; the statin, antiepileptic, and osteoporosis-drug clauses have separate citations and are not attributed to this source. (LLM, conf=0.85)

  Experimental — accuracy not yet measured
• cites [14] Spanakis, M., Alon-Ellenbogen, D., Ioannou, P., and Spernovasilis, N. (2023). Antibiotics and Lipid-Modifying Agents: Potential Drug-Drug Interactions and Their Clinical Implications. Pharmacy, 11. sourceSupported
• cites [15] Johannessen, 2010, Antiepileptic drug interactions—Principles and clinical implications, Curr. Neuropharmacol. sourceSupported
• cites [16] Kutt, 1984, Interactions between anticonvulsants and other commonly prescribed drugs, Epilepsia sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

Adding tramadol to SSRIs can precipitate serotonin syndrome.

“adding tramadol (an opioid) to SSRIs can precipitate serotonin syndrome [11,12].”

• cites [11] Zhou, 2020, Potentially Inappropriate Medication Combination with Opioids among Older Dental Patients: A Retrospective Review of Insurance Claims Data, Pharmacotherapy sourceCould not verify

  The Introduction (page unavailable) lists SSRIs and opioids as CNS-active drug classes of concern, and a table indexed as REFERENCES records tramadol prescriptions. Neither passage addresses serotonin syndrome or says that combining tramadol with SSRIs can precipitate it. Searches for “tramadol serotonin syndrome,” “serotonin syndrome,” and “tramadol” found no passage making that link. (LLM, conf=0.72)

  Experimental — accuracy not yet measured
• cites [12] Baldo, 2020, The anaesthetist, opioid analgesic drugs, and serotonin toxicity: A mechanistic and clinical review, Br. J. Anaesth. sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

Huang et al. (2025) noted that AI-powered DDI prediction must balance precision with clinical explainability through domain-specific calibration.

“This polarity mirrors the challenge outlined by Huang et al. (2025) [35], who noted that AI-powered DDI prediction must balance precision with clinical explainability through domain-specific calibration.”

• cites Huang et al. (2025) [35] sourcePartly supported

  Section 7.1 says DDI models can make precise predictions while remaining difficult to interpret, limiting clinical use; section 4.7 also calls explainable AI essential to clinical relevance. The paper supports the need to address both precision and explainability, but the cited passages do not identify domain-specific calibration as the way to do so. (LLM, conf=0.86)

  Experimental — accuracy not yet measured

May combine two claims.

The present study's design incorporated deterministic case generation, blinded expert adjudication, and standardized severity grading using the Lexicomp framework.

“A key strength of the present study lies in its rigorous and reproducible design, which incorporated deterministic case generation, blinded expert adjudication and standardized severity grading using the Lexicomp framework, an approach consistent with that used by Hughes et al. (2024) [6] in pharmacological safety research.”

• cites Hughes et al. (2024) [6] sourcePartly supported

  The Methods (Drug–Drug Interaction Exposure; page unavailable) describe applying a predefined list of interactions classified by severity, which supports a broad similarity in standardized grading. But Hughes et al. used the British National Formulary and Stockley’s Drug Interactions, not Lexicomp; the cited passages do not establish blinded expert adjudication or deterministic case generation. (LLM, conf=0.84)

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S3 (load-bearing).

Hakim et al. (2025) emphasized the need for robust guardrails and continuous monitoring when LLMs are deployed in safety-critical medical domains such as drug safety.

“At the same time, our observation of false-positive inflation aligns with the warning issued by Hakim et al. (2025) [40], who emphasized the need for robust guardrails and continuous monitoring when LLMs are deployed in safety-critical medical domains such as drug safety.”

• cites Hakim et al. (2025) [40] sourcePartly supported

  The Introduction describes a false-positive drug-safety concern that an LLM could generate, and the Discussion calls for a continuous need for robust guardrails in pharmacovigilance and other medical safety-critical settings. The paper also advocates human review of uncertain outputs, but the passages do not specifically call for continuous monitoring of deployed LLMs. (LLM, conf=0.84)

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S3 (load-bearing).

The inclusion of five distinct LLM architectures provides a comprehensive view rarely explored in dental informatics.

“The inclusion of five distinct LLM architectures (general-purpose and pharmacologically oriented) also provides a comprehensive view rarely explored in dental informatics, complementing the multidomain analyses discussed by Schwendicke et al. (2020) [27] and Sitaras et al. (2025) [28].”

• cites Schwendicke et al. (2020) [27] sourceSupported
• cites Sitaras et al. (2025) [28] sourceSupported

Classified as probably not a substantial claim. May combine two claims.

Huang et al. (2025) emphasised that AI-driven DDI research is making progress but still lacks domain-specific tuning and explainability.

“Moreover, Huang et al. (2025) [35] emphasised that AI-driven DDI research is making progress but still lacks domain-specific tuning and explainability, which our work begins to address.”

• cites Huang et al. (2025) [35] sourcePartly supported

  Sections 4.7 and 7.1 describe substantial progress in AI-based DDI prediction alongside persistent interpretability problems. Section 4.3 calls for more personalized, context-aware models, but the passages do not specifically identify a lack of domain-specific tuning. (LLM, conf=0.84)

  Experimental — accuracy not yet measured

May combine two claims.

The use of standardized scenarios ensured reproducibility but may not capture the variability of real-world prescribing.

“First, as in similar simulation-based studies (Zitu et al., 2025 [30]), the use of standardized scenarios ensured reproducibility but may not capture the variability of real-world prescribing.”

• cites Zitu et al., 2025 [30] sourceCould not verify

  Section 2.4 describes the source as a review that categorizes other studies, and Section 5.1 discusses generalizability across clinical data sources. Neither section addresses standardized prescribing scenarios or a reproducibility–real-world variability tradeoff. Searches for simulation, standardized cases, reproducibility, and prescribing found no supporting passage in our copy. (LLM, conf=0.76)

  Experimental — accuracy not yet measured

May combine two claims.

For the binary endpoint action required versus no action, all models showed very high negative predictive values above 91% but low positive predictive values of 16-20%.

“Considering the binary endpoint”

May combine two claims.

All 15 pairwise latency contrasts were significant after multiplicity correction, with adjusted p < 0.001 for every pair.

“A within-case rank-based analysis found all 15 pairwise contrasts significant after multiplicity correction (adjusted p < 0.001 for every pair).”

May combine two claims.

DeepSeek-Chat had 100% specificity and 0% Overcall but detected only 18% of clinically actionable interactions, with Safety-FN of 82%.

“DeepSeek-Chat adopted an extremely conservative behaviour, with 100% specificity and 0% Overcall, but at the cost of detecting only 18% of clinically actionable interactions (Safety-FN 82%).”

May combine two claims.

In the present dataset, ChatGPT-5 showed sensitivity close to 98% for D/X interactions, specificity around 57%, and Overcall above 40%.

“In the present dataset, ChatGPT-5 showed a comparable pattern, with sensitivity close to 98% for D/X interactions but specificity around 57% and Overcall above 40%, suggesting that high recall is again obtained at the cost of a substantial number of false-positive alerts.”

Probably not a substantial claim: may not pass S3 (load-bearing).

A total of 500 cases were included.

“A total of 500 cases were included.”

May combine two claims.

DeepSeek-Reasoner showed sensitivity of 88%, specificity of 55%, and an intermediate Safety-FN of 12%.

“DeepSeek-Reasoner showed a slightly more conservative profile, with sensitivity of 88% and specificity of 55%, and an intermediate Safety-FN of 12%.”

May combine two claims. The quote only partly states this claim.

Overall accuracy was highest for DeepSeek-Chat at 91.8% and lowest for Gemini-Pro at 47.8%.

“Overall accuracy was therefore highest for DeepSeek-Chat (91.8%), driven by the large proportion of non-actionable cases, and lowest for Gemini-2.5-Pro(47.8%) (Table 5, Figure 2).”

May combine two claims.

Median absolute severity error was 0 for DeepSeek-Chat and 1 for the remaining models, with an interquartile range of [0,1] for all.

“The median absolute error was 0 for DeepSeek-Chat and 1 for the remaining models, with an interquartile range of [0, 1] for all.”

May combine two claims.

ChatGPT-5 achieved very high sensitivity of 98% with moderate specificity of about 56%.

“ChatGPT-5 and Gemini-2.5-Flashachieved very high sensitivity (98% and 94%, respectively) with moderate specificity (about 56%), resulting in low Safety-FN rates (2-6%) but Overcall rates around 43-44%.”

May combine two claims.

Gemini-Flash achieved very high sensitivity of 94% with moderate specificity of about 56%.

“ChatGPT-5 and Gemini-2.5-Flashachieved very high sensitivity (98% and 94%, respectively) with moderate specificity (about 56%), resulting in low Safety-FN rates (2-6%) but Overcall rates around 43-44%.”

May combine two claims. The quote only partly states this claim.

Gemini-Pro maximised sensitivity at 100% with Safety-FN of 0%, while showing the lowest specificity of 42% and the highest Overcall of 58%.

“At the opposite extreme, Gemini-2.5-Promaximised sensitivity (100%, Safety-FN 0%) while showing the lowest specificity (42%) and the highest Overcall (58%).”

May combine two claims.

The present study observed moderate AUC values and relatively low positive predictive values (16-20%).

“As shown by Sicard et al. (2025) [32], general-purpose LLMs reached very high detection rates of DDIs but remained unable to reliably exclude non-interactions; the moderate AUC values and relatively low positive predictive values (16-20%) observed here align with that limitation, indicating that most alerts generated by even the best-performing models do not correspond to truly actionable DDIs.”

May combine two claims.

DeepSeek-Chat had perfect specificity but low sensitivity (18%).

“Conversely, the perfect specificity but low sensitivity of DeepSeek-Chat (18%) reflects an opposite safety profile with fewer false alarms but a greater risk of missed interactions.”

Classified as probably not a substantial claim.

Each of the 500 simulated oral-surgery cases was submitted exactly once to each model under fixed settings.

“Each of the 500 simulated oral-surgery cases was submitted exactly once to each model under these fixed settings.”

Classified as probably not a substantial claim. May combine two claims.

During the first round, surgeons were explicitly blinded to the identity of the models, to all model outputs and to the interim ratings provided by the other evaluators.

“During this first round, surgeons were explicitly blinded to the identity of the models, to all model outputs and to the interim ratings provided by the other evaluators.”

May combine two claims.

ChatGPT-5 may have shown an improvement over previous versions, reaching a level of performance comparable to domain-oriented reasoning models.

“ChatGPT-5 may have shown an improvement over previous versions, reaching a level of performance comparable to domain-oriented reasoning models.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Coadministration of NSAIDs with SSRIs more than doubles the odds of gastrointestinal bleeding.

“coadministration of NSAIDs with selective serotonin reuptake inhibitors (SSRIs) more than doubles the odds of gastrointestinal bleeding [9,10]”

• cites [9] Haghbin, 2023, Risk of Gastrointestinal Bleeding with Concurrent Use of NSAID and SSRI: A Systematic Review and Network Meta-Analysis, Dig. Dis. Sci. sourceNot checked — no full text of the source
• cites [10] Anglin, 2014, Risk of upper gastrointestinal bleeding with selective serotonin reuptake inhibitors with or without concurrent nonsteroidal anti-inflammatory use: A systematic review and meta-analysis, Am. J. Gastroenterol. sourceNot checked — no full text of the source

Classified as probably not a substantial claim. May combine two claims.

A prospective, simulation-based, parallel evaluation was conducted on isolated workstations.

“A prospective, simulation-based, parallel evaluation was conducted on isolated workstations.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Deterministic case generation with fixed random seeds, unique study IDs, immutable raw archives and timestamped logs were maintained to support reproducibility and version transparency.

“Deterministic case generation (fixed random seeds), unique study IDs, immutable raw archives and timestamped logs (model names, API endpoints, parameters as set in code, request/response times) were maintained to support reproducibility and version transparency.”

Probably not a substantial claim: may not pass S4 (based).

If the compendia gave different assessments, a conservative rule was applied: whenever a trusted source suggested a more serious interaction, the higher severity level was chosen.

“If the compendia gave different assessments, a conservative rule was applied: whenever a trusted source suggested a more serious interaction (for example, D instead of C), the higher severity level was chosen.”

May combine two claims.

None of the evaluated systems is explicitly fine-tuned on dental prescribing or oral-surgery DDIs, and none was provided with direct access to proprietary drug interaction databases during inference.

“None of these systems is explicitly fine-tuned on dental prescribing or oral-surgery DDIs, and none was provided with direct access to proprietary drug interaction databases (such as Lexicomp or Micromedex) during inference.”

Probably not a substantial claim: may not pass S3 (load-bearing).

When additional information was needed, panel members could consult three drug interaction compendia, Lexicomp, Micromedex and Stockley's Drug Interactions, as supporting knowledge sources.

“When additional information was needed, panel members could consult three drug interaction compendia (Lexicomp, Micromedex and Stockley's Drug Interactions) as supporting knowledge sources [37,38].”

• cites [37] Abbas, 2022, Determination of potential drug-drug interactions in prescription orders dispensed in a community pharmacy setting using Micromedex® and Lexicomp®: A retrospective observational study, Int. J. Clin. Pharm. sourceNot checked — no full text of the source
• cites [38] Patel, 2016, Evaluation of resources for analyzing drug interactions, J. Med. Libr. Assoc. sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Krishnan et al. (2024) observed that ChatGPT models could accurately detect most clinically relevant interactions but tended to overestimate their severity, leading to inflated risk profiles.

“Similarly, Krishnan et al. (2024) [31] observed that ChatGPT models could accurately detect most clinically relevant interactions but tended to overestimate their severity, leading to inflated risk profiles.”

• cites Krishnan et al. (2024) [31]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim. May combine two claims.

After independent review, all cases with at least one discrepant rating were re-examined in a dedicated consensus session, and disagreements were resolved by majority rule with two-out-of-three agreement.

“After an independent review, all cases with at least one discrepant rating were re-examined in a dedicated consensus session.In this session, the three surgeons discussed the clinical scenario and their initial assessments and resolved disagreements by majority rule (two-out-of-three agreement).”

Classified as probably not a substantial claim. May combine two claims.

Each surgeon independently reviewed all 500 cases and assessed presence of any clinically relevant DDI, suspected mechanism, severity grade on the A-B-C-D-X scale, and recommended management strategy.

“Each surgeon independently reviewed all 500 cases and assessed, for every case, the presence of any clinically relevant drug-drug interaction, the suspected mechanism, the clinical severity grade on the A-B-C-D-X scale and the recommended management strategy.”

Classified as probably not a substantial claim. May combine two claims.

No real patient data were processed; all records were fully simulated and non-identifiable.

“No real patient data were processed; all records were fully simulated and non-identifiable.”

Classified as probably not a substantial claim.

Multiplicity across the 10 pairwise tests was controlled using Holm's step-down adjustment.

“Multiplicity across the 10 pairwise tests was controlled using Holm's step-down adjustment; adjusted p-values are presented alongside raw values where applicable.”

Classified as probably not a substantial claim. May combine two claims.

The configuration ensured that each large language model received the same clinically formatted information and operated under stable, reproducible conditions throughout the simulation.

“This configuration ensured that each large language model received the same clinically formatted information and operated under stable, reproducible conditions throughout the simulation.”

Classified as probably not a substantial claim. May combine two claims.

These knowledge sources were used only to support the human reference standard and were not provided directly to the language models in the prompts.

“These knowledge sources were used only to support the human reference standard and were not provided directly to the language models in the prompts.”

May combine two claims.

Although model performance is clearly better than chance, it remains insufficient to serve as a stand-alone reference without additional safeguards.

“In this context, the observed quadratic-weighted κ and Gwet's AC values indicate only moderate agreement at best between the models and the human consensus, meaning that, although performance is clearly better than chance, it remains insufficient to serve as a stand-alone reference without additional safeguards.”

Classified as probably not a substantial claim.

Predefined plausibility checks were applied to all simulated cases to avoid unrealistic or clinically inconsistent scenarios.

“To avoid unrealistic or clinically inconsistent scenarios, predefined plausibility checks were applied to all simulated cases.”

Classified as probably not a substantial claim.

Cases showing implausible dose combinations or internal inconsistencies were identified and removed before analysis.

“Cases showing implausible dose combinations or internal inconsistencies-for example, duplicate prescriptions of the same active ingredient at incompatible doses, or regimens unlikely to be used in routine oral-surgery practice-were identified and removed before analysis.”

Classified as probably not a substantial claim.

The ordinal scale was dichotomized a priori as action required = {D,X} versus no action = {A,B,C}.

“The ordinal scale was dichotomized a priori as”

May combine two claims.

Current LLMs can deliver rapid, structured DDI screening in oral surgery but exhibit distinct safety trade-offs between missed critical interactions and alert overcalling.

“These findings indicate that current LLMs can deliver rapid, structured DDI screening in oral surgery but exhibit distinct safety trade-offs between missed critical interactions and alert overcalling.”

Classified as probably not a substantial claim. May combine two claims.

The final majority label was considered the reference standard for that case and used in all comparative analyses.

“The final majority label was considered the reference standard for that case and used in all comparative analyses.”

Classified as probably not a substantial claim.

The human comparator consisted of three board-certified oral surgeons each with more than 10 years of independent practice.

“The human comparator consisted of three board-certified oral surgeons (each with >10 years of independent practice) from the Oral Surgery Unit.”

May combine two claims.

In oral surgery, assessment of potential DDIs is a time-critical clinical judgment often made under pressure, frequently while patients are still in the chair and the surgical team must decide within minutes whether to proceed, substitute, adjust or defer treatment.

“In oral surgery, the assessment of potential drug-drug interactions is not a leisurely academic exercise but a time-critical clinical judgment often made under pressure, frequently while patients are still in the chair and the surgical team must decide within minutes whether to proceed, substitute, adjust or defer treatment [21,22].”

• cites [21] Hersh, 2004, Drug interactions in dentistry: The importance of knowing your CYPs, J. Am. Dent. Assoc. sourceNot checked — no full text of the source
• cites [22] Moore, 1999, Adverse drug interactions in dental practice. Professional and educational implications, J. Am. Dent. Assoc. sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

The analysis plan and rubric were finalized before data generation.

“The analysis plan and rubric were finalized before data generation.”

Probably not a substantial claim: may not pass S2 (checkable).

Integrating AI-driven DDI screening tools into chairside decision-support systems could enhance patient safety, particularly for polypharmacy or medically complex patients.

“Integrating such tools into chairside decision-support systems could enhance patient safety, particularly for polypharmacy or medically complex patients.”

May combine two claims.

Formal pre-consensus inter-rater agreement statistics were not calculated, so the reliability of individual surgeon ratings cannot be quantified separately from the consensus.

“Finally, formal pre-consensus inter-rater agreement statistics were not calculated, so the reliability of individual surgeon ratings cannot be quantified separately from the consensus; however, the use of a blinded, multi-expert panel and a structured consensus procedure supports the robustness of the final reference labels.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The lack of extreme severity categories (A or X) may have reduced sensitivity to rare but critical DDIs.

“Third, the lack of extreme severity categories (A or X) may have reduced sensitivity to rare but critical DDIs.”

May combine two claims.

DeepSeek-Chat's profile reflects an opposite safety profile with fewer false alarms but a greater risk of missed interactions.

“Conversely, the perfect specificity but low sensitivity of DeepSeek-Chat (18%) reflects an opposite safety profile with fewer false alarms but a greater risk of missed interactions.”

May combine two claims.

Large language models can assist clinicians in identifying drug-drug interactions in oral surgery by enabling rapid, standardized screening and reducing cognitive workload.

“Large language models (LLMs) can assist clinicians in identifying drug-drug interactions (DDIs) in oral surgery by enabling rapid, standardized screening and reducing cognitive workload, but they should be used strictly as auxiliary tools rather than replacements for clinical judgement.”

May combine two claims.

The models were tested with single prompts and no iterative refinement.

“Second, the models were tested with single prompts and no iterative refinement; interactive prompting could improve reasoning accuracy, as suggested by Radha Krishnan et al. (2024) [31], so the present results should be interpreted as a conservative estimate of best-case model performance.”

• cites Radha Krishnan et al. (2024) [31]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.