Accuracy of information given by ChatGPT for patients with inflammatory bowel disease in relation to ECCO guidelines
Open accessAbstract
Background: As acceptance of artificial intelligence [AI] platforms increases, more patients will consider these tools as sources of information. The ChatGPT architecture utilizes a neural network to process natural language, thus generating responses based on the context of input text. The accuracy and completeness of ChatGPT3.5 in the context of inflammatory bowel disease [IBD] remains unclear. Methods: In this prospective study, 38 questions worded by IBD patients were inputted into ChatGPT3.5. The following topics were covered: [1] Crohn’s disease [CD], ulcerative colitis [UC], and malignancy; [2] maternal medicine; [3] infection and vaccination; and [4] complementary medicine. Responses given by ChatGPT were assessed for accuracy [1—completely incorrect to 5—completely correct] and completeness [3-point Likert scale; range 1—incomplete to 3—complete] by 14 expert gastroenterologists, in comparison with relevant ECCO guidelines. Results: In terms of accuracy, most replies [84.2%] had a median score of ≥4 (interquartile range [IQR]: 2) and a mean score of 3.87 [SD: ±0.6]. For completeness, 34.2% of the replies had a median score of 3 and 55.3% had a median score of between 2 and <3. Overall, the mean rating was 2.24 [SD: ±0.4, median: 2, IQR: 1]. Though groups 3 and 4 had a higher mean for both accuracy and completeness, there was no significant scoring variation between the four question groups [Kruskal–Wallis test p > 0.05]. However, statistical analysis for the different individual questions revealed a significant difference for both accuracy [p < 0.001] and completeness [p < 0.001]. The questions which rated the highest for both accuracy and completeness were related to smoking, while the lowest rating was related to screening for malignancy and vaccinations especially in the context of immunosuppression and family planning. Conclusion: This is the first study to demonstrate the capability of an AI-based system to provide accurate and comprehensive answers to real-world patient queries in IBD. AI systems may serve as a useful adjunct for patients, in addition to standard of care in clinics and validated patient information resources. However, responses in specialist areas may deviate from evidence-based guidance and the replies need to give more firm advice.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Previous work evaluated the performance of ChatGPT-3.5 in answering IBD-related questions based on ECCO guidelines.
“Previous work evaluated the performance of ChatGPT-3.5 in answering IBD-related questions based on the European Crohn’s and Colitis Organisation (ECCO) guidelines (Sciberras et al., 2024).”
From When AI speaks like a specialist: ChatGPT-4 in the management of inflammatory bowel disease (2025) · cites this paper as Sciberras et al., 2024
A recent study showed that an AI-based system could provide accurate and comprehensive answers to real-world patient questions related to IBD.
“A recent study showed that an AI-based system could provide accurate and comprehensive answers to real-world patient questions related to IBD (Sciberras et al., 2024).”
From When AI speaks like a specialist: ChatGPT-4 in the management of inflammatory bowel disease (2025) · cites this paper as Sciberras et al., 2024
That prior study demonstrated a generally high level of accuracy but reported limited completeness of responses, particularly in complex scenarios such as malignancy screening, vaccination, and family planning.
“While that study demonstrated a generally high level of accuracy, it also reported limited completeness of responses, particularly in complex scenarios such as malignancy screening, vaccination, and family planning.”
From When AI speaks like a specialist: ChatGPT-4 in the management of inflammatory bowel disease (2025) · cites this paper as Sciberras et al., 2024
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · see citing works
Kaugeranna, R., Kaugeranna, E., & Aion, (. S. 4. 6. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.giscience.2023.43 · Cited by 13,695 · see citing works
Sallam, M. (2023) · Healthcare · doi:10.3390/healthcare11060887 · Cited by 3,126 · see citing works
Gilson, A., Safranek, C. W., Huang, T., Socrates, V., Chi, L., Taylor, R. A., & Chartash, D. (2023) · JMIR Medical Education · doi:10.2196/45312 · Cited by 2,078 · see citing works
Eysenbach, G., Powell, J., Kuss, O., & Sa, E.-R. (2002) · JAMA · doi:10.1001/jama.287.20.2691 · Cited by 1,871 · see citing works
Public health and online misinformation: challenges and recommendations
Swire-Thompson, B., & Lazer, D. (2020) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-040119-094127 · Cited by 1,170 · see citing works
Role of Chat GPT in public health
Biswas, S. S. (2023) · Annals of Biomedical Engineering · doi:10.1007/s10439-023-03172-7 · Cited by 673 · see citing works
Yeo, Y. H., Samaan, J. S., Ng, W. H., Ting, P., Trivedi, H. D., Vipani, A., Ayoub, W. S., Yang, J. D., Liran, O., Spiegel, B. M. R., et al. (2023) · Clinical and Molecular Hepatology · doi:10.3350/cmh.2023.0089 · Cited by 655 · see citing works
Johnson, D., Goodman, R., Patrinely, J., Stone, C., Zimmerman, E., Donald, R., Chang, S., Berkowitz, S., Finn, A., Jahangir, E., et al. (2023) · doi:10.21203/rs.3.rs-2566942/v1 · Cited by 598 · see citing works
Sustersic, M., Gauchet, A., Foote, A., & Bosson, J. (2016) · Health Expectations · doi:10.1111/hex.12487 · Cited by 176 · see citing works
ChatGPT Answers Common Patient Questions About Colonoscopy
Lee, T.-C., Staller, K., Botoman, V., Pathipati, M. P., Varma, S., & Kuo, B. (2023) · Gastroenterology · doi:10.1053/j.gastro.2023.04.033 · Cited by 146 · see citing works
Accuracy of Information Provided by ChatGPT Regarding Liver Cancer Surveillance and Diagnosis
Cao, J. J., Kwon, D. H.-M., Ghaziani, T., Kwo, P. Y., Tse, G., Kesselman, A. J., Kamaya, A., & Tse, J. R. (2023) · American Journal of Roentgenology · doi:10.2214/ajr.23.29493 · Cited by 75 · see citing works
Victora, C. G., Barros, A. J. D., de França, G. V. A., Crochemore‐Silva, I., Carvajal-Vélez, L., & Amouzou, A. (2017) · The Lancet Global Health · doi:10.1016/s2214-109x(17)30077-3 · Cited by 71 · see citing works
Cankurtaran, R. E., Polat, Y. H., Aydemir, N. G., Umay, E., & Yürekli, Ö. T. (2023) · Cureus · doi:10.7759/cureus.46736 · Cited by 36 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Large language models in patient education: a scoping review of applications in medicine
Aydin, S., Karabacak, M., Vlachos, V., & Margetis, K. D. (2024) · Frontiers in Medicine · doi:10.3389/fmed.2024.1477898 · Cited by 333 · see citing works
Pal, A., Wangmo, T., Bharadia, T., Ahmed-Richards, M., Bhanderi, M., Kachhadiya, R., Allemann, S. S., & Elger, B. S. (2025) · Patient Preference and Adherence · doi:10.2147/ppa.s527922 · Cited by 53 · see citing works
He, F., Yang, M., Li, J., Gong, T., Ma, J., Yang, T., Zhao, D., Li, S., & Tian, D. (2025) · BMC Public Health · doi:10.1186/s12889-025-22963-2 · Cited by 46 · see citing works
Iacucci, M., Santacroce, G., Maeda, Y., & Ghosh, S. K. (2025) · Gastroenterology · doi:10.1053/j.gastro.2025.03.005 · Cited by 49 · see citing works
Self-citationUse of artificial intelligence chatbots in clinical management of immune-related adverse events
Burnette, H., Pabani, A., von Itzstein, M. S., Switzer, B., Fan, R., Ye, F., Puzanov, I. J., Naidoo, J., Ascierto, P. A., Gerber, D. E., et al. (2024) · Journal for ImmunoTherapy of Cancer · doi:10.1136/jitc-2023-008599 · Cited by 35 · see citing works
Caballero-Mateos, A. M., la Fuente, G. A. C., & Gros, B. (2025) · Journal of Clinical Medicine · doi:10.3390/jcm14051536 · Cited by 24 · see citing works
Utilizing large language models for gastroenterology research: a conceptual framework
Berry, P., Dhanakshirur, R. R., & Khanna, S. (2025) · Therapeutic Advances in Gastroenterology · doi:10.1177/17562848251328577 · Cited by 21 · see citing works
Garcia, L. G., Emile, S. H., Linkeshwaran, L., Wignakumar, A., & Wexner, S. D. (2025) · Surgery · doi:10.1016/j.surg.2025.109393 · Cited by 19 · see citing works
Zhang, Y., Wan, X.-H., Kong, Q., Liu, H., Liu, J., Guo, J., Yang, X., Zuo, X., & Li, Y. (2025) · World Journal of Gastroenterology · doi:10.3748/wjg.v31.i6.102090 · Cited by 17 · see citing works
Deo, D. D., Buono, A. D., Gabbiadini, R., Nardone, O. M., Ferreiro‐Iglesias, R., Privitera, G., Bonifacio, C., Acosta, M. B.-D., Bezzio, C., & Armuzzi, A. (2025) · Frontiers in Immunology · doi:10.3389/fimmu.2025.1637159 · Cited by 16 · see citing works
Omar, M., Nassar, S., Sharif, Κ., Glicksberg, B. S., Nadkarni, G. N., & Klang, E. (2025) · Frontiers in Medicine · doi:10.3389/fmed.2024.1512824 · Cited by 15 · see citing works
Samaan, J. S., Issokson, K., Feldman, E., Fasulo, C., Rajeev, N. D., Ng, W. H., Hollander, B. S., Yeo, Y. H., & Vasiliauskas, E. A. (2024) · Crohn s & Colitis 360 · doi:10.1093/crocol/otae077 · Cited by 14 · see citing works
Wang, T., Chen, R., Wang, B., Zou, C., Fan, N., Yuan, S., Wang, A., Xi, Y., & Zang, L. (2025) · Journal of Medical Internet Research · doi:10.2196/75567 · Cited by 10 · see citing works
Lusetti, F., Maimaris, S., Rosa, G. P. L., Scalvini, D., Schiepatti, A., Biagi, F., Bernardi, A. D., Manes, G., & Saibeni, S. (2025) · Digestive and Liver Disease · doi:10.1016/j.dld.2025.04.026 · Cited by 9 · see citing works
Mapping the landscape of AI and ML in vaccine innovation: A bibliometric study
Niu, J.-R., Deng, R., Dong, Z., Yang, X., Xing, Z., Yin, Y., & Kang, J. (2025) · Human Vaccines & Immunotherapeutics · doi:10.1080/21645515.2025.2501358 · Cited by 8 · see citing works
Gregorio, V., Maeda, Y., Kudo, S., Kawabata, Y., Kuroki, T., Santacroce, G., Puga‐Tejada, M., Takenaka, K., Takabayashi, K., Ohara, J., et al. (2025) · Digestive Endoscopy · doi:10.1111/den.15081 · Cited by 10 · see citing works
Self-citationZhang, L., Wang, T., Zheng, Y., Kong, X., Hong, G., & Zang, L. (2025) · Journal of Orthopaedic Surgery and Research · doi:10.1186/s13018-025-05831-y · Cited by 7 · see citing works
Öztürk, O., Ergül, M., Çağır, Y., Atay, A., Acun, K. C., Çoşkun, O., Tenlik, İ., Durak, M. B., & Yüksel, İ. (2025) · Journal of Clinical Medicine · doi:10.3390/jcm14134599 · Cited by 7 · see citing works
Ghehsareh, M. M., Asri, N., Looha, M. A., Sadeghi, A., Ciacci, C., & Rostami‐Nejad, M. (2025) · Scientific Reports · doi:10.1038/s41598-025-15898-6 · Cited by 8 · see citing works
Wang, X., Yin, C., He, H., Guo, J., Fu, X., & 白飞虎 (2026) · Frontiers in Public Health · doi:10.3389/fpubh.2026.1810358 · Cited by 7 · see citing works
Forte, G., Mauro, D., Raimondi, M., Pantano, I., Gandolfo, S., Cauli, A., Guggino, G., Lubrano, E., Guiducci, S., Chimenti, M. S., et al. (2025) · Annals of the Rheumatic Diseases · doi:10.1016/j.ard.2025.11.012 · Cited by 6 · see citing works
Zeng, S., Dong, C., Liu, C., Zhen, J., Pu, Y., Hu, J., & Dong, W. (2025) · Digital Health · doi:10.1177/20552076251326217 · Cited by 4 · see citing works
Üşen, A., & Küçülmez, Ö. (2025) · Diagnostics · doi:10.3390/diagnostics15121455 · Cited by 4 · see citing works
Saadah, O. I., AlAmeel, T., Sarkhy, A. A. A., Hasosah, M. Y., Al‐Hussaini, A. A., Almadi, M. A., Al‐Bawardy, B., Altuwaijri, T. A., Al–Edreesi, M., Bakkari, S., et al. (2024) · Saudi Journal of Gastroenterology · doi:10.4103/sjg.sjg_171_24 · Cited by 4 · see citing works
Samaan, J. S., Margolis, S., Srinivasan, N., Srinivasan, A., Yeo, Y. H., Anand, R., Samaan, F. S., Mirocha, J., Safavi‐Naini, S. A. A., Kurdi, B. E., et al. (2024) · medRxiv · doi:10.1101/2024.07.27.24310809 · Cited by 4 · see citing works
Evaluation of artificial intelligence-based patient education models for irritable bowel syndrome
Raghavendran, A. K., Musunuri, B., Rajpurohit, S., Pai, G. C., Shetty, S., Kumari, P., Shetty, R., Shetty, A., & Bhat, G. (2025) · Indian Journal of Gastroenterology · doi:10.1007/s12664-025-01872-7 · Cited by 3 · see citing works
He, Z., Liu, Y., Wang, Z., Zhou, X., Fan, X., He, M., Wang, C., Fu, H., Wang, Z., & Bai, Y. (2025) · Gastroenterology report · doi:10.1093/gastro/goaf087 · Cited by 3 · see citing works
Iacucci, M., Santacroce, G., Maeda, Y., Majumder, S., Hassan, C., Shung, D. L., Stidham, R. W., Bisschops, R., Naranjo, V., Grisan, E., et al. (2026) · Nature Reviews Gastroenterology & Hepatology · doi:10.1038/s41575-026-01190-z · Cited by 3 · see citing works
Self-citationTranslation of Artificial Intelligence in Colonoscopy
Ahmed, J. F., El-Sayed, A., & Kader, R. (2025) · Digestion · doi:10.1159/000548823 · Cited by 2 · see citing works
Yazarkan, Y., Sönmez, G., & Simsek, C. (2026) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2026.106413 · Cited by 2 · see citing works
Use of ChatGPT for patient education involving HPV-associated oropharyngeal cancer
Patel, T. A., Michaelson, G., Morton, Z., Harris, A. L., Smith, B. D., Bourguillon, R., Wu, E. L., Eguia, A. A., & Maxwell, J. H. (2025) · American Journal of Otolaryngology · doi:10.1016/j.amjoto.2025.104642 · Cited by 2 · see citing works
When AI speaks like a specialist: ChatGPT-4 in the management of inflammatory bowel disease
De Cristofaro, E., Zorzi, F., Abreu, M., Colella, A., Blanco, G. D. V., Fiorino, G., Lolli, E., Noor, N., Lopetuso, L. R., Pioche, M., et al. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1678320 · Cited by 1 · see citing works
“5 in answering IBD-related questions based on the European Crohn’s and Colitis Organisation (ECCO) guidelines (Sciberras et al., 2024).”
Self-citationSood, A., Moyer, A., Jahangiri, P., Mar, D. A., Nitichaikulvatana, P., Ramreddy, N., Stolyar, L., & Lin, J. (2025) · JCR Journal of Clinical Rheumatology · doi:10.1097/rhu.0000000000002198 · Cited by 1 · see citing works
Xu, L., Zou, J., Sun, C., Chen, G., & Gao, S. (2025) · Frontiers in Medicine · doi:10.3389/fmed.2025.1600291 · Cited by 1 · see citing works
Hughes, R., Bello, A. L., Liang, R. F.-H., Pugliano, C. L., Zammarchi, I., Ghosh, S., & Iacucci, M. (2026) · Chinese Medical Journal · doi:10.1097/cm9.0000000000004170 · Cited by 1 · see citing works
Self-citationAssessing the accuracy of ChatGPT responses to guideline-based inquiries: A cross-sectional study
Xiang, A. J., Zhou, E., Moayad, L., Crowther, M. A., Eshaghpour, A., Li, P. Y., Javidan, A. P., & Li, A. (2025) · Canadian Journal of General Internal Medicine · doi:10.3138/cjgim.2024.0017 · Cited by 1 · see citing works
The Promise of ChatGPT in Medical Education: a systematic review (Preprint)Preprint
Tang, P., Xiao, R., Cao, Y., Ying, Z., Xiao, W., Wen, T., & Li, Y. (2025) · doi:10.2196/preprints.70796
Evaluation of the Applicability of ChatGPT in Patient Education on Obstructive Sleep Apnea
Riolobos, C. L., Riestra‐Ayora, J., Moreno, B. R., Rubio, N. L., García, J. M. D., Vaduva, C., Rodríguez, I. A., Pérez, L. S., García-Prieto, F., Pardo, S. C., et al. (2025) · Journal of Respiration · doi:10.3390/jor5010003
Infante, A., Perna, A., Chiloiro, S., Marziali, G., Martucci, M., Demarchis, L., Merlino, B., Natale, L., & Gaudino, S. (2025) · Journal of Personalized Medicine · doi:10.3390/jpm15080363
Yuan, J., & Zhao, D. (2025) · The American Surgeon · doi:10.1177/00031348251413530
Zhao, X., Sun, J., & Zhan, Q. (2026) · Endoscopy · doi:10.1055/s-0046-1822532
Zhao, X.-Y., Shen, Y., He, J.-J., Zhou, Q.-Y., Jiang, L.-S., Zhou, Z.-R., An, F.-M., Zhan, Q., Sun, J., & Feng, W. (2026) · World Journal of Gastroenterology · doi:10.3748/wjg.118690
Chen, R., Pan, Y., Liang, M., Wang, A., Ma, Z., Xi, Y., Fan, N., Yuan, S., Du, P., Wang, T., et al. (2026) · Journal of Medical Internet Research · doi:10.2196/87816
Liu, Y., Hu, C., Song, X., Xiang, L., tan, W., Guo, H. Y., & Wang, H. (2026) · Digestion · doi:10.1159/dig/acmag018
Leonforte, F., Cavallaro, A. S., Nicosia, V., Scapellato, A., Gambera, A., Genovese, C., Mistretta, A., Zanghi, A., Basile, G., Veroux, M., et al. (2026) · AI · doi:10.3390/ai7100396
- Authors
- Sciberras, M., Farrugia, Y., Gordon, H., Furfaro, F., Allocca, M., Torres, J., Arebi, N., Fiorino, G., Iacucci, M., Verstockt, B., Magro, F., Katsanos, K., Busuttil, J., De Giovanni, K., Fenech, V. A., Chetcuti Zammit, S., & Ellul, P.
- Affiliations
- Mater Dei Hospital (MT)
- Barts Health NHS Trust (GB)
- IRCCS Ospedale San Raffaele (IT)
- University of Lisbon (PT)
- St Mark's Hospital (GB)
- Azienda Ospedaliera San Camillo-Forlanini (IT)
- University College Cork (IE)
- KU Leuven (BE)
- Universidade do Porto (PT)
- University of Ioannina (GR)
- Association for Crohn`s and Colitis , Malta
- Department of Medicine, Division of Gastroenterology, Mater Dei Hospital , Msida , Malta
- Corresponding author
- Sciberras, Martina
- ORCID
- Venue
- Journal of Crohn's and Colitis
- Publisher
- Oxford University Press (OUP)
- Published
- 2024-03-23
- Volume
- 18
- Issue
- 8
- Pages
- 1215-1221
- ISSN
- 1873-9946 (ISSN-L)
- 1876-4479
- OpenAlex
- W4393111455
- Cited by
- 45
- References
- 13
- Citations by year
- 2026: 9 · 2025: 31 · 2024: 5
- FWCI
- 4.19
- Type
- journal article
- Language
- en
- Open access
- green
- hdl.handle.net
- Locations
- Journal of Crohn s and Colitis (publishedVersion)
- PubMed (publishedVersion)
- Archivio Istituzionale della Ricerca - Università Vita-Salute San Raffaele (submittedVersion, other-oa)
- Lirias (KU Leuven) (submittedVersion, other-oa)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Health Sciences › Health Professions › General Health Professions › Health Literacy and Information Accessibility
- Keywords
- ChatGPT, large language models
- MeSH terms
- Complementary Therapies, Artificial Intelligence, Colitis, Ulcerative, Crohn Disease, Female, Humans, Natural Language Processing, Neoplasms, Patient Education as Topic, Prospective Studies, Vaccination, Inflammatory Bowel Diseases, Practice Guidelines as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
