Cited by 160Artificial Intelligence in Healthcare and EducationBiomedical Text Mining and OntologiesArtificial Intelligence in Education

The DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.

Analysis of large-language model versus human performance for genetics questions

Open access

Duong, Dat, Solomon, Benjamin D.·2024·European Journal of Human Genetics

DOI 10.1038/s41431-023-01396-8PMC PMC10999420

Published version of a preprint

Open this paper in CoChat — free account Open PDF

Abstract

Large-language models like ChatGPT have recently received a great deal of attention. One area of interest pertains to how these models could be used in biomedical contexts, including related to human genetics. To assess one facet of this, we compared the performance of ChatGPT versus human respondents (13,642 human responses) in answering 85 multiple-choice questions about aspects of human genetics. Overall, ChatGPT did not perform significantly differently (p = 0.8327) than human respondents; ChatGPT was 68.2% accurate, compared to 66.6% accuracy for human respondents. Both ChatGPT and humans performed better on memorization-type questions versus critical thinking questions (p < 0.0001). When asked the same question multiple times, ChatGPT frequently provided different answers (16% of initial responses), including for both initially correct and incorrect answers, and gave plausible explanations for both correct and incorrect answers. ChatGPT's performance was impressive, but currently demonstrates significant shortcomings for clinical or other high-stakes use. Addressing these limitations will be important to guide adoption in real-life situations.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1038/s41431-023-01396-8

Cites

13 of 13 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Highly accurate protein structure prediction with AlphaFold
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03819-2 · Cited by 43,764
Large Language Models Encode Clinical Knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2212.13138 · Cited by 4,083
Large language models encode clinical knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2023) · Nature · doi:10.1038/s41586-023-06291-2 · Cited by 4,083
Predicting splicing from primary sequence with deep learning
Jaganathan, K., Kyriazopoulou Panagiotopoulou, S., McRae, J. F., Darbandi, S. F., Knowles, D., Li, Y. I., Kosmicki, J. A., Arbelaez, J., Cui, W., Schwartz, G. B., et al. (2019) · Cell · doi:10.1016/j.cell.2018.12.015 · Cited by 2,853
A universal SNP and small-indel variant caller using deep neural networks
Poplin, R., Chang, P.-C., Alexander, D., Schwartz, S., Colthurst, T., Ku, A., Newburger, D., Dijamco, J., Nguyen, N., Afshar, P. T., et al. (2018) · Nature Biotechnology · doi:10.1038/nbt.4235 · Cited by 1,840
A large language model for electronic health records
Yang, X., Chen, A., PourNejatian, N., Shin, H. C., Smith, K. E., Parisien, C., Compas, C., Martin, C., Costa, A. B., Flores, M. G., et al. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00742-2 · Cited by 847
AI for radiographic COVID-19 detection selects shortcuts over signal
DeGrave, A. J., Janizek, J. D., & Lee, S.-I. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00338-7 · Cited by 631
Artificial intelligence in clinical and genomic diagnostics
Dias, R., & Torkamani, A. (2019) · Genome Medicine · doi:10.1186/s13073-019-0689-8 · Cited by 499
The use of machine learning in rare diseases: a scoping review
Schaefer, J., Lehne, M., Schepers, J., Prasser, F., & Thun, S. (2020) · Orphanet Journal of Rare Diseases · doi:10.1186/s13023-020-01424-6 · Cited by 213
Can artificial intelligence pass the Fellowship of the Royal College of Radiologists examination? Multi-reader diagnostic accuracy study
Shelmerdine, S. C., Martin, H. M., Shirodkar, K. K., Shamshuddin, S., & Weir-McCall, J. R. (2022) · BMJ · doi:10.1136/bmj-2022-072826 · Cited by 45

Show all 12 references

Sorted by citations

ChatGPT Utility in Healthcare Education, Research, and Practice: Systematic Review on the Promising Perspectives and Valid Concerns
Sallam, M. (2023) · Healthcare · doi:10.3390/healthcare11060887 · Cited by 3,126
A Survey on Evaluation of Large Language Models
Chang, Y., Wang, X., Wang, J., Wu, Y., Yang, L., Zhu, K., Chen, H., Yi, X., Wang, C., Wang, Y., et al. (2024) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3641289 · Cited by 4,254
“[19] / Duong and Solomon [36] / Hamidi and Roberts [58] / Holmes et al.”
Background
Toward expert-level medical question answering with large language models
Singhal, K., Tu, T., Gottweis, J., Sayres, R., Wulczyn, E., Amin, M., Hou, L., Clark, K., Pfohl, S. R., Cole-Lewis, H., et al. (2025) · Nature Medicine · doi:10.1038/s41591-024-03423-7 · Cited by 1,734
A Survey on Large Language Models: Applications, Challenges, Limitations, and Practical UsagePreprint
Hadi, M. U., Al-Tashi, Q., Qureshi, R., Shah, A., Muneer, A., Irfan, M., Zafar, A., Shaikh, M. B., Akhtar, N., Hassan, S. Z., et al. (2023) · doi:10.36227/techrxiv.23589741.v1 · Cited by 475
ChatGPT in healthcare: A taxonomy and systematic review
Li, J., Dada, A., Puladi, B., Kleesiek, J., & Egger, J. (2024) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2024.108013 · Cited by 452
Assessing GPT-4 for cell type annotation in single-cell RNA-seq analysis
Hou, W., & Ji, Z. (2024) · Nature Methods · doi:10.1038/s41592-024-02235-4 · Cited by 277
Leveraging Generative AI and Large Language Models: A Comprehensive Roadmap for Healthcare Integration
Yu, P., Xu, H., Hu, X., & Deng, C. (2023) · Healthcare · doi:10.3390/healthcare11202776 · Cited by 296
Assessing Health Students' Attitudes and Usage of ChatGPT in Jordan: Validation Study
Sallam, M., Salim, N. A., Barakat, M., Al-Mahzoum, K., Al-Tammemi, A. B., Malaeb, D., Hallit, R., & Hallit, S. (2023) · JMIR Medical Education · doi:10.2196/48254 · Cited by 234
Generative AI: A systematic review using topic modelling techniques
Gupta, P., Ding, B., Guan, C., & Ding, D. (2024) · Data and Information Management · doi:10.1016/j.dim.2024.100066 · Cited by 299
Evaluation of ChatGPT-generated medical responses: A systematic review and meta-analysis
Wei, Q., Yao, Z., Cui, Y., Wei, B., Jin, Z., & Xu, X. (2024) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2024.104620 · Cited by 233
Educational Applications of the ChatGPT AI System: A Systematic Review Research
İPEK, Z. H., Gözüm, A. İ. C., Papadakis, S., & Kalogiannakis, M. (2023) · Educational Process International Journal · doi:10.22521/edupij.2023.123.2 · Cited by 215
Foundation and large language models: fundamentals, challenges, opportunities, and social impacts
Myers, D., Mohawesh, R., Chellaboina, V. I., Sathvik, A. L., Venkatesh, P., Ho, Y.-H., Henshaw, H., Al-Hawawreh, M. S., Berdik, D., & Jararweh, Y. (2023) · Cluster Computing · doi:10.1007/s10586-023-04203-7 · Cited by 217
Large Language Models: A Comprehensive Survey of its Applications, Challenges, Limitations, and Future ProspectsPreprint
Hadi, M. U., Al-Tashi, Q., Qureshi, R., Shah, A., Muneer, A., Irfan, M., Zafar, A., Shaikh, M. B., Akhtar, N., Hassan, S. Z., et al. (2023) · doi:10.36227/techrxiv.23589741.v4 · Cited by 193
A survey of large language models for healthcare: from data, technology, and applications to accountability and ethics
He, K., Mao, R., Lin, Q., Ruan, Y., Lan, X., Feng, M., & Cambria, E. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.102963 · Cited by 147
MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoning
Tang, X., Zou, A., Zhang, Z., Li, Z., Zhao, Y., Zhang, X., Cohan, A., & Gerstein, M. (2024) · Findings of the Association for Computational Linguistics ACL 2024 · doi:10.18653/v1/2024.findings-acl.33 · Cited by 376
ChatGPT in action: Harnessing artificial intelligence potential and addressing ethical challenges in medicine, education, and scientific research
Jeyaraman, M., Ramasubramanian, S., Balaji, S., Jeyaraman, N., Nallakumarasamy, A., & Sharma, S. (2023) · World Journal of Methodology · doi:10.5662/wjm.v13.i4.170 · Cited by 125
Large language models: a survey of their development, capabilities, and applications
Yadagiri, A., & Pakray, P. (2024) · Knowledge and Information Systems · doi:10.1007/s10115-024-02310-4 · Cited by 155
Unlocking precision medicine: clinical applications of integrating health records, genetics, and immunology through artificial intelligence
Chen, Y., Hsiao, T., Lin, C., & Fann, Y. C. (2025) · Journal of Biomedical Science · doi:10.1186/s12929-024-01110-w · Cited by 159
Comparison of the Performance of GPT-3.5 and GPT-4 With That of Medical Students on the Written German Medical Licensing Examination: Observational Study
Meyer, A., Riese, J., & Streichert, T. (2024) · JMIR Medical Education · doi:10.2196/50965 · Cited by 110
A Systematic Review of the Limitations and Associated Opportunities of ChatGPT
Cong‐Lem, N., Soyoof, A., & Tsering, D. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2344142 · Cited by 140
Using ChatGPT for Clinical Practice and Medical Education: Cross-Sectional Survey of Medical Students’ and Physicians’ Perceptions
Tangadulrat, P., Sono, S., & Tangtrakulwanich, B. (2023) · JMIR Medical Education · doi:10.2196/50658 · Cited by 99
The promise and peril of using a large language model to obtain clinical information: ChatGPT performs strongly as a fertility counseling tool with limitations
Chervenak, J., Lieman, H., Blanco-Breindel, M., & Jindal, S. (2023) · Fertility and Sterility · doi:10.1016/j.fertnstert.2023.05.151 · Cited by 87
Large Language Models: A Comprehensive Survey of its Applications, Challenges, Limitations, and Future ProspectsPreprint
Hadi, M. U., Al-Tashi, Q., Qureshi, R., Shah, A., Muneer, A., Irfan, M., Zafar, A., Shaikh, M. B., Akhtar, N., Wu, J., et al. (2023) · doi:10.36227/techrxiv.23589741 · Cited by 111
ChatGPT in medicine: A cross-disciplinary systematic review of ChatGPT’s (artificial intelligence) role in research, clinical practice, education, and patient interaction
Fatima, A., Shafique, M. A., Alam, K., Ahmad, T. K. F., & Mustafa, M. S. (2024) · Medicine · doi:10.1097/md.0000000000039250 · Cited by 95
Evaluating large language models and agents in healthcare: key challenges in clinical applications
Chen, X., Xiang, J., Lu, S., Liu, Y., He, M., & Shi, D. (2025) · Intelligent Medicine · doi:10.1016/j.imed.2025.03.002 · Cited by 97
ChatGPT Knowledge Evaluation in Basic and Clinical Medical Sciences: Multiple Choice Question Examination-Based Performance
Meo, S. A., Al-Masri, A. A., Alotaibi, M., Meo, M. Z. S., & Meo, M. O. S. (2023) · Healthcare · doi:10.3390/healthcare11142046 · Cited by 74
ChatGPT performance in the medical specialty exam: An observational study
Oztermeli, A. D., & Öztermeli, A. (2023) · Medicine · doi:10.1097/md.0000000000034673 · Cited by 70
ChatGPT- versus human-generated answers to frequently asked questions about diabetes: A Turing test-inspired survey among employees of a Danish diabetes center
Hulmán, Á., Dollerup, O. L., Mortensen, J. F., Fenech, M. E., Norman, K., Støvring, H., & Hansen, T. K. (2023) · PLoS ONE · doi:10.1371/journal.pone.0290773 · Cited by 75
A Survey of Large Language Models for Healthcare: From Data, Technology, and Applications to Accountability and Ethics
He, K., Mao, R., Lin, Q., Ruan, Y., Lan, X., Feng, M., & Cambria, E. (2024) · doi:10.2139/ssrn.4809363 · Cited by 69
Application of large language models in disease diagnosis and treatment
Yang, X., Li, T., Su, Q., Liu, Y., Kang, C., Lyu, Y., Zhao, L., Nie, Y., & Pan, Y. (2024) · Chinese Medical Journal · doi:10.1097/cm9.0000000000003456 · Cited by 76
A systematic review and meta-analysis on ChatGPT and its utilization in medical and dental research
Bagde, H. S., Dhopte, A., Alam, M. K., & Basri, R. (2023) · Heliyon · doi:10.1016/j.heliyon.2023.e23050 · Cited by 74
Can ChatGPT-3.5 Pass a Medical Exam? A Systematic Review of ChatGPT’s Performance in Academic Testing
Sumbal, A., Sumbal, R., & Amir, A. (2024) · Journal of Medical Education and Curricular Development · doi:10.1177/23821205241238641 · Cited by 63
Large Language Models in Medical Diagnostics: Scoping Review With Bibliometric Analysis
Su, H., Sun, Y., Li, R., Zhang, A., Yang, Y., Xiao, F., Duan, Z., Chen, J., Hu, Q., Yang, T., et al. (2025) · Journal of Medical Internet Research · doi:10.2196/72062 · Cited by 69
Large Language Models in Biomedical and Health Informatics: A Review with Bibliometric Analysis
Yu, H., Fan, L., Li, L., Zhou, J., Ma, Z., Xian, L., Hua, W., He, S., Jin, M., Zhang, Y., et al. (2024) · Journal of Healthcare Informatics Research · doi:10.1007/s41666-024-00171-8 · Cited by 82
Assessing the utility of large language models for phenotype-driven gene prioritization in the diagnosis of rare genetic disease
Kim, J., Wang, K., Weng, C., & Liu, C. (2024) · The American Journal of Human Genetics · doi:10.1016/j.ajhg.2024.08.010 · Cited by 53
Large Language Models: A Comprehensive Survey of its Applications, Challenges, Limitations, and Future ProspectsPreprint
Hadi, M. U., Al-Tashi, Q., Qureshi, R., Shah, A., Muneer, A., Irfan, M., Zafar, A., Shaikh, M. B., Akhtar, N., Hassan, S. Z., et al. (2023) · doi:10.36227/techrxiv.23589741.v3 · Cited by 71
The model student: GPT-4 performance on graduate biomedical science exams
Stribling, D., Xia, Y., Amer, M. K., Graim, K. S., Mulligan, C. J., & Renne, R. (2024) · Scientific Reports · doi:10.1038/s41598-024-55568-7 · Cited by 60
Artificial intelligence in clinical genetics
Duong, D., & Solomon, B. D. (2025) · European Journal of Human Genetics · doi:10.1038/s41431-024-01782-w · Cited by 60
Self-citation
Breaking barriers: can ChatGPT compete with a shoulder and elbow specialist in diagnosis and management?
Daher, M., Koa, J., Boufadel, P., Singh, J., Fares, M. Y., & Abboud, J. A. (2023) · JSES International · doi:10.1016/j.jseint.2023.07.018 · Cited by 49
Assessment of the capacity of ChatGPT as a self-learning tool in medical pharmacology: a study using MCQs
Choi, W. (2023) · BMC Medical Education · doi:10.1186/s12909-023-04832-x · Cited by 54
Large Language Models: A Comprehensive Survey of its Applications, Challenges, Limitations, and Future ProspectsPreprint
Hadi, M. U., Al-Tashi, Q., Qureshi, R., Shah, A., Muneer, A., Irfan, M., Zafar, A., Shaikh, M. B., Akhtar, N., Hassan, S. Z., et al. (2023) · doi:10.36227/techrxiv.23589741.v2 · Cited by 54
The Scientific Knowledge of Bard and ChatGPT in Endocrinology, Diabetes, and Diabetes Technology: Multiple-Choice Questions Examination-Based Performance
Meo, S. A., Al-Khlaiwi, T., AbuKhalaf, A. A., Meo, A. S., & Klonoff, D. C. (2025) · Journal of Diabetes Science and Technology · doi:10.1177/19322968231203987 · Cited by 40
A comparative evaluation of ChatGPT 3.5 and ChatGPT 4 in responses to selected genetics questions
McGrath, S., Kozel, B. A., Graceffo, S., Sutherland, N., Danford, C. J., & Walton, N. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocae128 · Cited by 40
Deciphering Public Voices in the Digital Era
Fu, X., Sanchez, T. W., Li, C., & Junqueira, J. R. (2024) · Journal of the American Planning Association · doi:10.1080/01944363.2024.2309259 · Cited by 40
Diagnostic Accuracy of a Custom Large Language Model on Rare Pediatric Disease Case Reports
Young, C. C., Enichen, E., Rivera, C., Auger, C. A., Grant, N., Rao, A. S., & Succi, M. D. (2024) · American Journal of Medical Genetics Part A · doi:10.1002/ajmg.a.63878 · Cited by 37
Applying fine-tuned LLMs for reducing data needs in load profile analysis
Hu, Y., Kim, H., Ye, K., & Lü, N. (2024) · Applied Energy · doi:10.1016/j.apenergy.2024.124666 · Cited by 34
Evaluating search engines and large language models for answering health questions
Fernández-Pichel, M., Pichel, J. C., & Losada, D. E. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01546-w · Cited by 33
Survey of Multimodal Medical Question Answering
Demirhan, H., & Zadrozny, W. (2023) · BioMedInformatics · doi:10.3390/biomedinformatics4010004 · Cited by 32
Bioinformatics and biomedical informatics with ChatGPT: Year one review
Wang, J., Cheng, Z., Yao, Q., Liu, L., Xu, D. H., & Hu, G. (2024) · Quantitative Biology · doi:10.1002/qub2.67 · Cited by 37
Code Interpreter for Bioinformatics: Are We There Yet?
Wang, L., Ge, X., Liu, L., & Hu, G. (2023) · Annals of Biomedical Engineering · doi:10.1007/s10439-023-03324-9 · Cited by 31
Artificial Intelligence-Based ChatGPT Responses for Patient Questions on Optic Disc Drusen
Potapenko, I. O., Malmqvist, L., Subhi, Y., & Hamann, S. (2023) · Ophthalmology and Therapy · doi:10.1007/s40123-023-00800-2 · Cited by 29
AI ChatBots’ solutions to mathematical problems in interactive e-textbooks: Affordances and constraints from the eyes of students and teachers
Ergene, Ö., & ERGENE, B. Ç. (2024) · Education and Information Technologies · doi:10.1007/s10639-024-13121-z · Cited by 41
Generative artificial intelligence: implications for biomedical and health professions education
Hersh, W. (2025) · Annual Review of Biomedical Data Science · doi:10.1146/annurev-biodatasci-103123-094756 · Cited by 28
Evaluating large language models on medical, lay-language, and self-reported descriptions of genetic conditions
Flaharty, K. A., Hu, P., Hanchard, S. L., Ripper, M. E., Duong, D., Waikel, R. L., & Solomon, B. D. (2024) · The American Journal of Human Genetics · doi:10.1016/j.ajhg.2024.07.011 · Cited by 26
Self-citation
Scientific figures interpreted by ChatGPT: strengths in plot recognition and limits in color perception
Wang, J., Ye, Q., Liu, L., Guo, N. L., & Hu, G. (2024) · npj Precision Oncology · doi:10.1038/s41698-024-00576-z · Cited by 30
FoodSky: A food-oriented large language model that can pass the chef and dietetic examinations
Zhou, P., Min, W., Fu, C., Jin, Y., Huang, M., Li, X., Mei, S., & Jiang, S. (2025) · Patterns · doi:10.1016/j.patter.2025.101234 · Cited by 32
An Overarching Framework for the Ethics of Artificial Intelligence in Pediatrics
Sisk, B. A., Antes, A. L., & DuBois, J. M. (2024) · Archives of Pediatrics and Adolescent Medicine · doi:10.1001/jamapediatrics.2023.5761 · Cited by 25
Evaluating the strengths and weaknesses of large language models in answering neurophysiology questions
Shojaee-Mend, H., Mohebbati, R., Amiri, M., & Atarodi, A. (2024) · Scientific Reports · doi:10.1038/s41598-024-60405-y · Cited by 24
Evaluation of ChatGPT's performance in providing treatment recommendations for pediatric diseases
Wei, Q., Wang, Y., Yao, Z., Cui, Y., Wei, B., Li, T., & Xu, X. (2023) · Pediatric Discovery · doi:10.1002/pdi3.42 · Cited by 23
SELL ME THIS ARTIFICIAL PEN: USING CHATGPT TO ENHANCE SALES ROLE PLAYS
Milovic, A., Gyomlai, M. D., Spaid, B. I., & Dingus, R. (2024) · Marketing Education Review · doi:10.1080/10528008.2024.2341278 · Cited by 24
Can artificial intelligence-strengthened ChatGPT or other large language models transform nucleic acid research?
Chatterjee, S., Bhattacharya, M., Lee, S., & Chakraborty, C. (2023) · Molecular Therapy — Nucleic Acids · doi:10.1016/j.omtn.2023.06.019 · Cited by 21
PhenoFlow: A Human-LLM Driven Visual Analytics System for Exploring Large and Complex Stroke Datasets
Kim, J., Lee, S., Jeon, H., Lee, K., Bae, H., Kim, B., & Seo, J. (2024) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2024.3456215 · Cited by 27
PrISM-Q&A: Step-Aware Voice Assistant on a Smartwatch Enabled by Multimodal Procedure Tracking and Large Language Models
Arakawa, R., Lehman, J. F., & Goel, M. (2024) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3699759 · Cited by 22
Comparison of the Performance of ChatGPT, Claude and Bard in Support of Myopia Prevention and Control
Wang, Y., Liang, L., Li, R., Wang, Y., & Hao, C. (2024) · Journal of Multidisciplinary Healthcare · doi:10.2147/jmdh.s473680 · Cited by 21
Applications and Future Prospects of Medical LLMs: A Survey Based on the M-KAT Conceptual Framework
Chang, Y., Yin, J., Li, J., Liu, C., Cao, L., & Lin, S. (2024) · Journal of Medical Systems · doi:10.1007/s10916-024-02132-5 · Cited by 20
Large and small-scale models’ fusion-driven proactive robotic manipulation control for human-robot collaborative assembly in industry 5.0
Ma, D., Zhang, C., Xu, Q., & Zhou, G. (2025) · Robotics and Computer-Integrated Manufacturing · doi:10.1016/j.rcim.2025.103078 · Cited by 21
Appraisal of ChatGPT’s aptitude for medical education: comparative analysis with third-year medical students in a pulmonology examination
Cherif, H., Moussa, C., Missaoui, A. M., Salouage, I., Mokaddem, S., & Dhahri, B. (2024) · JMIR Medical Education · doi:10.2196/52818 · Cited by 18
Identifying Generative Artificial Intelligence Chatbot Use on Multiple-Choice, General Chemistry Exams Using Rasch Analysis
Sorenson, B., & Hanson, K. (2024) · Journal of Chemical Education · doi:10.1021/acs.jchemed.4c00165 · Cited by 26
A Retrieval-augmented Generation application for Question-Answering in Nutrigenetics Domain
Benfenati, D., Filippis, G. M. D., Rinaldi, A. M., Russo, C., & Tommasino, C. (2024) · Procedia Computer Science · doi:10.1016/j.procs.2024.09.467 · Cited by 19
Perspectives of ChatGPT in Pharmacology Education, and Research in Health Care: A Narrative Review
Patel, C., Pandya, S. K., & Sojitra, B. (2023) · Journal of Pharmacology and Pharmacotherapeutics · doi:10.1177/0976500x231210427 · Cited by 17
Medical knowledge of ChatGPT in public health, infectious diseases, COVID-19 pandemic, and vaccines: multiple choice questions examination based performance
Meo, S. A., Alotaibi, M. S., Meo, M. Z. S., Meo, M. O. S., & Hamid, M. (2024) · Frontiers in Public Health · doi:10.3389/fpubh.2024.1360597 · Cited by 15
ChatGPT and Medicine: Together We Embrace the AI Renaissance
Hacking, S. M. (2024) · JMIR Bioinformatics and Biotechnology · doi:10.2196/52700 · Cited by 15
ChatGPT, a Friend or a Foe in Medical Education: A Review of Strengths, Challenges, and Opportunities
Zarei, M., Zarei, M., Hamzehzadeh, S., Oliyaei, S. S. B., & Hosseini, M. (2024) · Shiraz E-Medical Journal · doi:10.5812/semj-145840 · Cited by 21
Incorporating evidence into mental health Q&A: a novel method to use generative language models for validated clinical content extraction
Kharitonova, K., Pérez‐Fernández, D., Gutiérrez-Hernando, J., Gutiérrez-Fandiño, A., Callejas, Z., & Griol, D. (2024) · Behaviour and Information Technology · doi:10.1080/0144929x.2024.2321959 · Cited by 17
Unveiling the risks of ChatGPT in diagnostic surgical pathology
Guastafierro, V., Corbitt, D. N., Bressan, A., Fernandes, B., Mintemur, Ö., Magnoli, F., Ronchi, S., La Rosa, S., Uccella, S., & Renne, S. L. (2025) · Virchows Archiv · doi:10.1007/s00428-024-03918-1 · Cited by 14
Leveraging artificial intelligence and machine learning for unraveling pathogenesis and advancing precision medicine in autoimmune diseases
Cao, C., Zhao, W., Guo, J., Wang, Z., Lü, A., & Liang, C. C. (2025) · The Innovation Medicine · doi:10.59717/j.xinn-med.2025.100154 · Cited by 14
Evaluating ChatGPT’s competency in radiation oncology: A comprehensive assessment across clinical scenarios
Ramadan, S., Mutsaers, A., Chen, P.-H. C., Bauman, G. S., Velker, V. M., Ahmad, B., Arifin, A. J., Nguyen, T. K., Palma, D., & Goodman, C. D. (2024) · Radiotherapy and Oncology · doi:10.1016/j.radonc.2024.110645 · Cited by 13
Evaluating the Use of Generative Artificial Intelligence to Support Genetic Counseling for Rare Diseases
Jeon, S., Lee, S., Chung, H., Yun, J. Y., Park, E. A., So, M.-K., & Huh, J. (2025) · Diagnostics · doi:10.3390/diagnostics15060672 · Cited by 12
Will I soon be out of my job? Quality and guideline conformity of ChatGPT therapy suggestions to patient inquiries with gynecologic symptoms in a palliative setting
Braun, E.-M., Juhasz‐Böss, I., Solomayer, E., Truhn, D., Keller, C., Heinrich, V., & Braun, B. J. (2023) · Archives of Gynecology and Obstetrics · doi:10.1007/s00404-023-07272-6 · Cited by 13
Can ChatGPT understand genetics?
Emmert‐Streib, F. (2023) · European Journal of Human Genetics · doi:10.1038/s41431-023-01419-4 · Cited by 12
The impact of Chat Generative Pre-trained Transformer (ChatGPT) on oncology: application, expectations, and future prospects
Li, Y., Gao, W., Luan, Z., Zhou, Z., & Li, J. (2023) · Cureus · doi:10.7759/cureus.48670 · Cited by 11
The scientific knowledge of three large language models in cardiology: multiple-choice questions examination-based performance
Altamimi, I., Alhumimidi, A. S., Alshehri, S., AlRumayan, A. F., Al-khlaiwi, T., Meo, S. A., & Temsah, M.-H. A. (2024) · Annals of Medicine and Surgery · doi:10.1097/ms9.0000000000002120 · Cited by 11
Artificial intelligence and the impact on medical genetics
Solomon, B. D., & Chung, W. K. (2023) · American Journal of Medical Genetics Part C Seminars in Medical Genetics · doi:10.1002/ajmg.c.32060 · Cited by 11
Self-citation
Is artificial intelligence getting too much credit in medical genetics?
Alkuraya, I. F. (2023) · American Journal of Medical Genetics Part C Seminars in Medical Genetics · doi:10.1002/ajmg.c.32062 · Cited by 10
Perspectives on the Current and Future State of Artificial Intelligence in Medical Genetics
Solomon, B. D., Cheatham, M., de Guimarães, T. A. C., Duong, D., Haendel, M. A., Hsieh, T., Javanmardi, B., Johnson, B. A., Krawitz, P. M., Kruszka, P. S., et al. (2025) · American Journal of Medical Genetics Part A · doi:10.1002/ajmg.a.64118 · Cited by 9
Self-citation
HWREx: AI-enabled Hardware Weakness and Risk Exploration and Storytelling Framework with LLM-assisted Mitigation Suggestion
Ghimire, S., Lin, Y.-Z., Mamun, M., Chowdhury, M. A., Alemi, F., Cai, S., Guo, J., Zhu, M., Li, H., Latibari, B. S., et al. (2025) · ACM Transactions on Design Automation of Electronic Systems · doi:10.1145/3737459 · Cited by 15
Just how transformative will AI/ML be for immuno-oncology?
Bottomly, D., & McWeeney, S. K. (2024) · Journal for ImmunoTherapy of Cancer · doi:10.1136/jitc-2023-007841 · Cited by 9
A benchmark for large language models in bioinformaticsPreprint
Sarwal, V., Andreoletti, G., Munteanu, V., Suhodolschi, A., Ciorbă, D., Bostan, V., Dimian, M., Eskin, E., Wang, W. W., & Mangul, S. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.12.19.572483 · Cited by 12
Generative AI and the profession of genetic counseling
Meekins‐Doherty, L., Dive, L. L., McEwen, A., & Sexton, A. (2025) · Journal of Genetic Counseling · doi:10.1002/jgc4.2009 · Cited by 11
On the Responsible Use of Chatbots in Bioinformatics
Hu, G., Liu, L., & Xu, D. H. (2024) · Genomics Proteomics & Bioinformatics · doi:10.1093/gpbjnl/qzae002 · Cited by 8
Health Use Cases of AI Chatbots: Identification and Analysis of ChatGPT Prompts in Social Media Discourses
Karami, A., Qiao, Z., Zhang, X., Kharrazi, H. H. K., Bozorgi, P., & Bozorgi, A. (2024) · Big Data and Cognitive Computing · doi:10.3390/bdcc8100130 · Cited by 9
Importance of critical thinking to understand ChatGPT
Emmert‐Streib, F. (2023) · European Journal of Human Genetics · doi:10.1038/s41431-023-01443-4 · Cited by 7
Comparison of clinical geneticist and computer visual attention in assessing genetic conditions
Duong, D., Johny, A. R., Hanchard, S. E. L., Fortney, C., Flaharty, K. A., Hellmann, F., Hu, P., Javanmardi, B., Moosa, S., Patel, T., et al. (2024) · PLoS Genetics · doi:10.1371/journal.pgen.1011168 · Cited by 6
Self-citation
Applying IRT to Distinguish Between Human and Generative AI Responses to Multiple-Choice Assessments
Strugatski, A., & Alexandron, G. (2025) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/3706468.3706490 · Cited by 5
A Survey of Large-Scale Deep Learning Models in Medicine and Healthcare
Chen, Z., Liu, R., Huang, S., Guo, Y., & Ren, Y. (2025) · Computer Modeling in Engineering & Sciences · doi:10.32604/cmes.2025.067809 · Cited by 6
Evaluation of ChatGPT in providing appropriate fracture prevention recommendations and medical science question responses: A quantitative research
Meng, J., Zhang, Z., Tang, H., Xiao, Y., Liu, P., Gao, S., & He, M. (2024) · Medicine · doi:10.1097/md.0000000000037458 · Cited by 5
Is ChatGPT a Reliable Tool for Explaining Medical Terms?
Oztermeli, A. D. (2025) · Cureus · doi:10.7759/cureus.77258 · Cited by 5
SheepDoctor: A knowledge graph enhanced large language model for sheep disease diagnosis
Xiong, J., Zhou, Y., Tian, F., Ni, F., & Zhao, L. (2025) · Smart Agricultural Technology · doi:10.1016/j.atech.2025.101001 · Cited by 7
Does Using ChatGPT Result in Human Cognitive Augmentation?
Fulbright, R., & Morrison, M. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-61569-6_9 · Cited by 7
AI vs. Human Programmers: Complexity and Performance in Code Generation
Azeem, S., Naveed, M. S., Sajid, M., & Ali, I. (2025) · VAWKUM Transactions on Computer Sciences · doi:10.21015/vtcs.v13i1.2043 · Cited by 6

Authors: Duong, D., & Solomon, B. D.
Affiliations: National Institutes of Health (US)
Corresponding author: Solomon, Benjamin D.
ORCID: Solomon, Benjamin D.
Venue: European Journal of Human Genetics
Publisher: Springer Science and Business Media LLC
Published: 2023-05-29
Volume: 32
Issue: 4
Pages: 466-468
ISSN: 1018-4813 (ISSN-L)
1476-5438
DOI: 10.1038/s41431-023-01396-8
PMC: PMC10999420
OpenAlex: W4378603221
Cited by: 159
References: 12
Citations by year: 2026: 15 · 2025: 42 · 2024: 59 · 2023: 42
FWCI: 6.29
Type: journal article
Language: en
Open access: green
doi.org
Licence: www.springernature.com/gp/researchers/text-and-data-mining (tdm)
www.springernature.com/gp/researchers/text-and-data-mining (vor)
Locations: European Journal of Human Genetics (publishedVersion)
medRxiv (acceptedVersion)
PubMed Central (publishedVersion)
Europe PMC (PubMed Central) (publishedVersion)
PubMed (publishedVersion)
PubMed Central (submittedVersion)
Funders: U.S. Department of Health & Human Services | NIH | National Human Genome Research Institute (N/A)
Topics: Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Health Sciences › Medicine › Health Informatics › Biomedical Text Mining and Ontologies
Social Sciences › Education › Artificial Intelligence in Education
Keywords: large language models, ChatGPT, human genetics education, multiple-choice questions

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Analysis of large-language model versus human performance for genetics questions

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks