Cited by 50Artificial Intelligence in Healthcare and EducationTopic ModelingInformation Retrieval and Search Behavior

GastroBot: a Chinese gastrointestinal disease chatbot based on the retrieval-augmented generation

Open access
Zhou, Qingqing, Liu, Can, Duan, Yuchen, Sun, Kaijie, Li, Yu, Kan, Hongxing, Gu, Zongyun, Shu, Jianhua +1 more·2024·Frontiers in Medicine

Abstract

Introduction: Large Language Models (LLMs) play a crucial role in clinical information processing, showcasing robust generalization across diverse language tasks. However, existing LLMs, despite their significance, lack optimization for clinical applications, presenting challenges in terms of illusions and interpretability. The Retrieval-Augmented Generation (RAG) model addresses these issues by providing sources for answer generation, thereby reducing errors. This study explores the application of RAG technology in clinical gastroenterology to enhance knowledge generation on gastrointestinal diseases. Methods: We fine-tuned the embedding model using a corpus consisting of 25 guidelines on gastrointestinal diseases. The fine-tuned model exhibited an 18% improvement in hit rate compared to its base model, gte-base-zh. Moreover, it outperformed OpenAI’s Embedding model by 20%. Employing the RAG framework with the llama-index, we developed a Chinese gastroenterology chatbot named “GastroBot,” which significantly improves answer accuracy and contextual relevance, minimizing errors and the risk of disseminating misleading information. Results: When evaluating GastroBot using the RAGAS framework, we observed a context recall rate of 95%. The faithfulness to the source, stands at 93.73%. The relevance of answers exhibits a strong correlation, reaching 92.28%. These findings highlight the effectiveness of GastroBot in providing accurate and contextually relevant information about gastrointestinal diseases. During manual assessment of GastroBot, in comparison with other models, our GastroBot model delivers a substantial amount of valuable knowledge while ensuring the completeness and consistency of the results. Discussion: Research findings suggest that incorporating the RAG method into clinical gastroenterology can enhance the accuracy and reliability of large language models. Serving as a practical implementation of this method, GastroBot has demonstrated significant enhancements in contextual comprehension and response quality. Continued exploration and refinement of the model are poised to drive forward clinical information processing and decision support in the gastroenterology field.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3389/fmed.2024.1392555

Cites

38 of 38 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Extensive literature consistently reports that domain-specific models such as BioGPT are prone to hallucination or generating content beyond the retrieved evidence unless explicitly fine-tuned for retrieval-grounded generation.

    “Moreover, extensive literature consistently reports that while domain-specific models such as BioGPT excel at producing semantically rich and lexically appropriate text, they are prone to hallucination or generating content beyond the retrieved evidence unless explicitly fine-tuned for retrieval-grounded generation [16,24,29,32]”

    From Enhancing Medical Question Answering with LLMs via a Hybrid Retrieval-Augmented Generation Framework (2026) · cites this paper as [32]

    The Discussion (‘Previous research background’; page unavailable) says hallucinations remain a challenge for LLMs in clinical applications and that retrieval-augmented generation can improve answer accuracy. It does not discuss BioGPT, text extending beyond retrieved evidence, or a requirement to fine-tune generation for retrieval grounding. The fine-tuning described in Materials and methods (‘Experiment’; page unavailable) applies to an embedding model, not the generating model. (LLM, conf=0.85)

    Experimental — accuracy not yet measured
  • RAGAS was selected for its decomposed metric design, efficiency in large-scale evaluation (with over 95% agreement with human annotations), and demonstrated suitability in healthcare applications such as GastroBot.

    “RAGAS was selected for its decomposed metric design, efficiency in large-scale evaluation (with over 95% agreement with human annotations), and demonstrated suitability in healthcare applications such as GastroBot [32,52].”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

3 of the 41 references couldn’t be matched to a known paper.

Research works printed but not linked: 1

  1. [31]Levkowitz H. Introduction to information retrieval (IR). (2008). Available at: https://api.semanticscholar.org/CorpusID:61517903

Cited by

Sorted by citations

Authors
Zhou, Q., Liu, C., Duan, Y., Sun, K., Li, Y., Kan, H., Gu, Z., Shu, J., & Hu, J.
Affiliations
Anhui University of Traditional Chinese Medicine (CN)
Corresponding author
Hu, Jili
Venue
Frontiers in Medicine
Published
2024-05-22
Volume
11
Pages
1392555-1392555
ISSN
2296-858X (ISSN-L)
OpenAlex
W4398218462
Cited by
44
References
41
Citations by year
2026: 17 · 2025: 20 · 2024: 7
FWCI
4.11
Type
journal article
Language
en
Open access
Locations
  • Frontiers in Medicine (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
Funders
  • Ministry of Education of the People's Republic of China (202101123001)
  • Anhui Provincial Department of Education
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
Keywords
retrieval-augmented generation, gastrointestinal diseases, Chinese language models, large language models, clinical decision support, clinical guidelines, RAGAS evaluation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review