Cited by 30Artificial Intelligence in Healthcare and EducationHealth Literacy and Information AccessibilityInformation Retrieval and Search Behavior

Can Large Language Models Aid Caregivers of Pediatric Cancer Patients in Information Seeking? A Cross‐Sectional Investigation

Open access
Sezgin, Emre, Jackson, Daniel I., Kocaballi, A. Baki, Bibart, Mindy, Zupanec, Sue, Landier, Wendy, Audino, Anthony, Ranalli, Mark +1 more·2025·Cancer Medicine

Published version of a preprint

Abstract

Purpose: Caregivers in pediatric oncology need accurate and understandable information about their child's condition, treatment, and side effects. This study assesses the performance of publicly accessible large language model (LLM)‐supported tools in providing valuable and reliable information to caregivers of children with cancer. Methods: In this cross‐sectional study, we evaluated the performance of the four LLM‐supported tools—ChatGPT (GPT‐4), Google Bard (Gemini Pro), Microsoft Bing Chat, and Google SGE—against a set of frequently asked questions (FAQs) derived from the Children's Oncology Group Family Handbook and expert input (In total, 26 FAQs and 104 generated responses). Five pediatric oncology experts assessed the generated LLM responses using measures including accuracy, clarity, inclusivity, completeness, clinical utility, and overall rating. Additionally, the content quality was evaluated including readability, AI disclosure, source credibility, resource matching, and content originality. We used descriptive analysis and statistical tests including Shapiro–Wilk, Levene's, Kruskal–Wallis H ‐tests, and Dunn's post hoc tests for pairwise comparisons. Results: ChatGPT shows high overall performance when evaluated by the experts. Bard also performed well, especially in accuracy and clarity of the responses, whereas Bing Chat and Google SGE had lower overall scores. Regarding the disclosure of responses being generated by AI, it was observed less frequently in ChatGPT responses, which may have affected the clarity of responses, whereas Bard maintained a balance between AI disclosure and response clarity. Google SGE generated the most readable responses whereas ChatGPT answered with the most complexity. LLM tools varied significantly ( p < 0.001) across all expert evaluations except inclusivity. Through our thematic analysis of expert free‐text comments, emotional tone and empathy emerged as a unique theme with mixed feedback on expectations from AI to be empathetic. Conclusion: LLM‐supported tools can enhance caregivers' knowledge of pediatric oncology. Each model has unique strengths and areas for improvement, indicating the need for careful selection based on specific clinical contexts. Further research is required to explore their application in other medical specialties and patient demographics, assessing broader applicability and long‐term impacts.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1002/cam4.70554

Cites

69 of 70 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • LLMs have the potential to provide real-time, individually customized, and up-to-date medical information, with concomitant risks and benefits, to allow for informed, complex, and empathic decision-making among caregivers for oncology patients.

    “Of particular interest is the potential of LLMs as providers of real-time, individually customized, and up-to-date medical information, with their concomitant risks and benefits, to allow for informed, complex, and empathic decision-making among caregivers for oncology patients [30].”

    From Empathic AI for Patient-Centered Cancer Care: A Scoping Review of Patient Navigation, Support, and Clinical Practice (2026) · cites this paper as [30]

    Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 3

  1. [19]H. Zhou, F. Liu, B. Gu, et al., “A Survey of Large Language Models in Medicine: Progress, Application, and Challenge,” arXiv (2023).
  2. [26]S. Qi, Y. He, and Z. Yuan, “Can We Catch the Elephant? A Survey of the Evolvement of Hallucination Evaluation on Natural Language Generation,” arXiv (2024).
  3. [60]J. Anthis, K. Lum, M. Ekstrand, et al., “The Impossibility of Fair LLMs,” arXiv (2024).

Cited by

Sorted by citations

Authors
Sezgin, E., Jackson, D. I., Kocaballi, A. B., Bibart, M., Zupanec, S., Landier, W., Audino, A., Ranalli, M., & Skeens, M.
Affiliations
  • The Abigail Wexner Research Institute at Nationwide Children's Hospital Columbus Ohio USA
  • Nationwide Children's Hospital (US)
  • Macquarie University (AU)
  • Hospital for Sick Children (CA)
  • University of Alabama at Birmingham (US)
Venue
Cancer Medicine
Publisher
Wiley
Published
2025-01
Volume
14
Issue
1
Pages
e70554-e70554
ISSN
2045-7634 (ISSN-L)
OpenAlex
W4406147515
Cited by
30
References
68
Citations by year
2026: 19 · 2025: 10 · 2024: 1
FWCI
6.98
Type
journal article
Language
en
Open access
Locations
  • Cancer Medicine (publishedVersion, cc-by)
  • medRxiv (acceptedVersion, cc-by-nc-nd)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Health Professions › General Health Professions › Health Literacy and Information Accessibility
  • Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
Keywords
large language models, childhood cancer, ChatGPT, Google Bard, Bing Chat, AI-generated health information, information accuracy, AI disclosure, readability, frequently asked questions
MeSH terms
Child, Cross-Sectional Studies, Female, Humans, Language, Male, Neoplasms, Caregivers, Information Seeking Behavior

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review