Analysis of large-language model versus human performance for genetics questions
Open accessPublished version of a preprint
Abstract
Large-language models like ChatGPT have recently received a great deal of attention. One area of interest pertains to how these models could be used in biomedical contexts, including related to human genetics. To assess one facet of this, we compared the performance of ChatGPT versus human respondents (13,642 human responses) in answering 85 multiple-choice questions about aspects of human genetics. Overall, ChatGPT did not perform significantly differently (p = 0.8327) than human respondents; ChatGPT was 68.2% accurate, compared to 66.6% accuracy for human respondents. Both ChatGPT and humans performed better on memorization-type questions versus critical thinking questions (p < 0.0001). When asked the same question multiple times, ChatGPT frequently provided different answers (16% of initial responses), including for both initially correct and incorrect answers, and gave plausible explanations for both correct and incorrect answers. ChatGPT's performance was impressive, but currently demonstrates significant shortcomings for clinical or other high-stakes use. Addressing these limitations will be important to guide adoption in real-life situations.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
A replication of this study might not yield the exact same results, as GPT-3.5 is inconsistent in answering 15% of medical questions.
“It should also be noted that a replication of this study might not yield the exact same results, as the literature suggests that GPT-3.5 is inconsistent in answering 15% of medical questions [59].”
From Comparison of the Performance of GPT-3.5 and GPT-4 With That of Medical Students on the Written German Medical Licensing Examination: Observational Study (2024) · cites this paper as [59]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Highly accurate protein structure prediction with AlphaFold
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03819-2 · Cited by 43,764 · see citing works
Large Language Models Encode Clinical Knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2212.13138 · Cited by 4,083 · see citing works
Large language models encode clinical knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2023) · Nature · doi:10.1038/s41586-023-06291-2 · Cited by 4,083 · see citing works
Predicting splicing from primary sequence with deep learning
Jaganathan, K., Kyriazopoulou Panagiotopoulou, S., McRae, J. F., Darbandi, S. F., Knowles, D., Li, Y. I., Kosmicki, J. A., Arbelaez, J., Cui, W., Schwartz, G. B., et al. (2019) · Cell · doi:10.1016/j.cell.2018.12.015 · Cited by 2,853 · see citing works
A universal SNP and small-indel variant caller using deep neural networks
Poplin, R., Chang, P.-C., Alexander, D., Schwartz, S., Colthurst, T., Ku, A., Newburger, D., Dijamco, J., Nguyen, N., Afshar, P. T., et al. (2018) · Nature Biotechnology · doi:10.1038/nbt.4235 · Cited by 1,840 · see citing works
A large language model for electronic health records
Yang, X., Chen, A., PourNejatian, N., Shin, H. C., Smith, K. E., Parisien, C., Compas, C., Martin, C., Costa, A. B., Flores, M. G., et al. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00742-2 · Cited by 847 · see citing works
AI for radiographic COVID-19 detection selects shortcuts over signal
DeGrave, A. J., Janizek, J. D., & Lee, S.-I. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00338-7 · Cited by 631 · see citing works
Artificial intelligence in clinical and genomic diagnostics
Dias, R., & Torkamani, A. (2019) · Genome Medicine · doi:10.1186/s13073-019-0689-8 · Cited by 474 · see citing works
The use of machine learning in rare diseases: a scoping review
Schaefer, J., Lehne, M., Schepers, J., Prasser, F., & Thun, S. (2020) · Orphanet Journal of Rare Diseases · doi:10.1186/s13023-020-01424-6 · Cited by 213 · see citing works
Shelmerdine, S. C., Martin, H. M., Shirodkar, K. K., Shamshuddin, S., & Weir-McCall, J. R. (2022) · BMJ · doi:10.1136/bmj-2022-072826 · Cited by 45 · see citing works
Rubinstein–Taybi syndrome in diverse populations
Tekendo‐Ngongang, C., Owosela, B., Fleischer, N., Addissie, Y. A., Malonga, B., Badoe, E. V., Gupta, N., Moresco, A., Huckstadt, V., Ashaat, E. A., et al. (2020) · American Journal of Medical Genetics Part A · doi:10.1002/ajmg.a.61888 · Cited by 32 · see citing works
Scoping review and classification of deep learning in medical genetics
Hanchard, S. E. L., Dwyer, M. C., Liu, S., Hu, P., Tekendo‐Ngongang, C., Waikel, R. L., Duong, D., & Solomon, B. D. (2022) · Genetics in Medicine · doi:10.1016/j.gim.2022.04.025 · Cited by 20 · see citing works
Self-citationSolomon, B. D. (2022) · Wiley eBooks · doi:10.1002/9781119847212 · Cited by 7 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Sallam, M. (2023) · Healthcare · doi:10.3390/healthcare11060887 · Cited by 3,126 · see citing works
A Survey on Evaluation of Large Language Models
Chang, Y., Wang, X., Wang, J., Wu, Y., Yang, L., Zhu, K., Chen, H., Yi, X., Wang, C., Wang, Y., et al. (2024) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3641289 · Cited by 2,842 · see citing works
“[19] / Duong and Solomon [36] / Hamidi and Roberts [58] / Holmes et al.”
BackgroundToward expert-level medical question answering with large language models
Singhal, K., Tu, T., Gottweis, J., Sayres, R., Wulczyn, E., Amin, M., Hou, L., Clark, K., Pfohl, S. R., Cole-Lewis, H., et al. (2025) · Nature Medicine · doi:10.1038/s41591-024-03423-7 · Cited by 1,734 · see citing works
A Survey on Large Language Models: Applications, Challenges, Limitations, and Practical UsagePreprint
Hadi, M. U., Al-Tashi, Q., Qureshi, R., Shah, A., Muneer, A., Irfan, M., Zafar, A., Shaikh, M. B., Akhtar, N., Hassan, S. Z., et al. (2023) · doi:10.36227/techrxiv.23589741.v1 · Cited by 429 · see citing works
ChatGPT in healthcare: A taxonomy and systematic review
Li, J., Dada, A., Puladi, B., Kleesiek, J., & Egger, J. (2024) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2024.108013 · Cited by 452 · see citing works
Assessing GPT-4 for cell type annotation in single-cell RNA-seq analysis
Hou, W., & Ji, Z. (2024) · Nature Methods · doi:10.1038/s41592-024-02235-4 · Cited by 277 · see citing works
Yu, P., Xu, H., Hu, X., & Deng, C. (2023) · Healthcare · doi:10.3390/healthcare11202776 · Cited by 267 · see citing works
Assessing Health Students' Attitudes and Usage of ChatGPT in Jordan: Validation Study
Sallam, M., Salim, N. A., Barakat, M., Al-Mahzoum, K., Al-Tammemi, A. B., Malaeb, D., Hallit, R., & Hallit, S. (2023) · JMIR Medical Education · doi:10.2196/48254 · Cited by 234 · see citing works
Generative AI: A systematic review using topic modelling techniques
Gupta, P., Ding, B., Guan, C., & Ding, D. (2024) · Data and Information Management · doi:10.1016/j.dim.2024.100066 · Cited by 262 · see citing works
Evaluation of ChatGPT-generated medical responses: A systematic review and meta-analysis
Wei, Q., Yao, Z., Cui, Y., Wei, B., Jin, Z., & Xu, X. (2024) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2024.104620 · Cited by 215 · see citing works
Educational Applications of the ChatGPT AI System: A Systematic Review Research
İPEK, Z. H., Gözüm, A. İ. C., Papadakis, S., & Kalogiannakis, M. (2023) · Educational Process International Journal · doi:10.22521/edupij.2023.123.2 · Cited by 209 · see citing works
Foundation and large language models: fundamentals, challenges, opportunities, and social impacts
Myers, D., Mohawesh, R., Chellaboina, V. I., Sathvik, A. L., Venkatesh, P., Ho, Y.-H., Henshaw, H., Al-Hawawreh, M. S., Berdik, D., & Jararweh, Y. (2023) · Cluster Computing · doi:10.1007/s10586-023-04203-7 · Cited by 197 · see citing works
Hadi, M. U., Al-Tashi, Q., Qureshi, R., Shah, A., Muneer, A., Irfan, M., Zafar, A., Shaikh, M. B., Akhtar, N., Hassan, S. Z., et al. (2023) · doi:10.36227/techrxiv.23589741.v4 · Cited by 173 · see citing works
He, K., Mao, R., Lin, Q., Ruan, Y., Lan, X., Feng, M., & Cambria, E. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.102963 · Cited by 147 · see citing works
MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoning
Tang, X., Zou, A., Zhang, Z., Li, Z., Zhao, Y., Zhang, X., Cohan, A., & Gerstein, M. (2024) · Findings of the Association for Computational Linguistics ACL 2024 · doi:10.18653/v1/2024.findings-acl.33 · Cited by 133 · see citing works
Jeyaraman, M., Ramasubramanian, S., Balaji, S., Jeyaraman, N., Nallakumarasamy, A., & Sharma, S. (2023) · World Journal of Methodology · doi:10.5662/wjm.v13.i4.170 · Cited by 125 · see citing works
Large language models: a survey of their development, capabilities, and applications
Yadagiri, A., & Pakray, P. (2024) · Knowledge and Information Systems · doi:10.1007/s10115-024-02310-4 · Cited by 135 · see citing works
Chen, Y., Hsiao, T., Lin, C., & Fann, Y. C. (2025) · Journal of Biomedical Science · doi:10.1186/s12929-024-01110-w · Cited by 137 · see citing works
Meyer, A., Riese, J., & Streichert, T. (2024) · JMIR Medical Education · doi:10.2196/50965 · Cited by 110 · see citing works
A Systematic Review of the Limitations and Associated Opportunities of ChatGPT
Cong‐Lem, N., Soyoof, A., & Tsering, D. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2344142 · Cited by 125 · see citing works
Tangadulrat, P., Sono, S., & Tangtrakulwanich, B. (2023) · JMIR Medical Education · doi:10.2196/50658 · Cited by 99 · see citing works
Chervenak, J., Lieman, H., Blanco-Breindel, M., & Jindal, S. (2023) · Fertility and Sterility · doi:10.1016/j.fertnstert.2023.05.151 · Cited by 87 · see citing works
Hadi, M. U., Al-Tashi, Q., Qureshi, R., Shah, A., Muneer, A., Irfan, M., Zafar, A., Shaikh, M. B., Akhtar, N., Wu, J., et al. (2023) · doi:10.36227/techrxiv.23589741 · Cited by 88 · see citing works
Fatima, A., Shafique, M. A., Alam, K., Ahmad, T. K. F., & Mustafa, M. S. (2024) · Medicine · doi:10.1097/md.0000000000039250 · Cited by 94 · see citing works
Evaluating large language models and agents in healthcare: key challenges in clinical applications
Chen, X., Xiang, J., Lu, S., Liu, Y., He, M., & Shi, D. (2025) · Intelligent Medicine · doi:10.1016/j.imed.2025.03.002 · Cited by 87 · see citing works
Meo, S. A., Al-Masri, A. A., Alotaibi, M., Meo, M. Z. S., & Meo, M. O. S. (2023) · Healthcare · doi:10.3390/healthcare11142046 · Cited by 74 · see citing works
ChatGPT performance in the medical specialty exam: An observational study
Oztermeli, A. D., & Öztermeli, A. (2023) · Medicine · doi:10.1097/md.0000000000034673 · Cited by 71 · see citing works
Hulmán, Á., Dollerup, O. L., Mortensen, J. F., Fenech, M. E., Norman, K., Støvring, H., & Hansen, T. K. (2023) · PLoS ONE · doi:10.1371/journal.pone.0290773 · Cited by 73 · see citing works
He, K., Mao, R., Lin, Q., Ruan, Y., Lan, X., Feng, M., & Cambria, E. (2024) · doi:10.2139/ssrn.4809363 · Cited by 69 · see citing works
Application of large language models in disease diagnosis and treatment
Yang, X., Li, T., Su, Q., Liu, Y., Kang, C., Lyu, Y., Zhao, L., Nie, Y., & Pan, Y. (2024) · Chinese Medical Journal · doi:10.1097/cm9.0000000000003456 · Cited by 68 · see citing works
A systematic review and meta-analysis on ChatGPT and its utilization in medical and dental research
Bagde, H. S., Dhopte, A., Alam, M. K., & Basri, R. (2023) · Heliyon · doi:10.1016/j.heliyon.2023.e23050 · Cited by 71 · see citing works
Sumbal, A., Sumbal, R., & Amir, A. (2024) · Journal of Medical Education and Curricular Development · doi:10.1177/23821205241238641 · Cited by 63 · see citing works
Large Language Models in Medical Diagnostics: Scoping Review With Bibliometric Analysis
Su, H., Sun, Y., Li, R., Zhang, A., Yang, Y., Xiao, F., Duan, Z., Chen, J., Hu, Q., Yang, T., et al. (2025) · Journal of Medical Internet Research · doi:10.2196/72062 · Cited by 66 · see citing works
Large Language Models in Biomedical and Health Informatics: A Review with Bibliometric Analysis
Yu, H., Fan, L., Li, L., Zhou, J., Ma, Z., Xian, L., Hua, W., He, S., Jin, M., Zhang, Y., et al. (2024) · Journal of Healthcare Informatics Research · doi:10.1007/s41666-024-00171-8 · Cited by 69 · see citing works
Kim, J., Wang, K., Weng, C., & Liu, C. (2024) · The American Journal of Human Genetics · doi:10.1016/j.ajhg.2024.08.010 · Cited by 53 · see citing works
Hadi, M. U., Al-Tashi, Q., Qureshi, R., Shah, A., Muneer, A., Irfan, M., Zafar, A., Shaikh, M. B., Akhtar, N., Hassan, S. Z., et al. (2023) · doi:10.36227/techrxiv.23589741.v3 · Cited by 62 · see citing works
The model student: GPT-4 performance on graduate biomedical science exams
Stribling, D., Xia, Y.-J., Amer, M., Graim, K. S., Mulligan, C. J., & Renne, R. (2024) · Scientific Reports · doi:10.1038/s41598-024-55568-7 · Cited by 47 · see citing works
Artificial intelligence in clinical genetics
Duong, D., & Solomon, B. D. (2025) · European Journal of Human Genetics · doi:10.1038/s41431-024-01782-w · Cited by 59 · see citing works
Self-citationDaher, M., Koa, J., Boufadel, P., Singh, J., Fares, M. Y., & Abboud, J. A. (2023) · JSES International · doi:10.1016/j.jseint.2023.07.018 · Cited by 48 · see citing works
Choi, W. (2023) · BMC Medical Education · doi:10.1186/s12909-023-04832-x · Cited by 48 · see citing works
Hadi, M. U., Al-Tashi, Q., Qureshi, R., Shah, A., Muneer, A., Irfan, M., Zafar, A., Shaikh, M. B., Akhtar, N., Hassan, S. Z., et al. (2023) · doi:10.36227/techrxiv.23589741.v2 · Cited by 47 · see citing works
Meo, S. A., Al-Khlaiwi, T., AbuKhalaf, A. A., Meo, A. S., & Klonoff, D. C. (2025) · Journal of Diabetes Science and Technology · doi:10.1177/19322968231203987 · Cited by 40 · see citing works
A comparative evaluation of ChatGPT 3.5 and ChatGPT 4 in responses to selected genetics questions
McGrath, S., Kozel, B. A., Graceffo, S., Sutherland, N., Danford, C. J., & Walton, N. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocae128 · Cited by 40 · see citing works
Deciphering Public Voices in the Digital Era
Fu, X., Sanchez, T. W., Li, C., & Junqueira, J. R. (2024) · Journal of the American Planning Association · doi:10.1080/01944363.2024.2309259 · Cited by 37 · see citing works
Diagnostic Accuracy of a Custom Large Language Model on Rare Pediatric Disease Case Reports
Young, C. C., Enichen, E., Rivera, C., Auger, C. A., Grant, N., Rao, A. S., & Succi, M. D. (2024) · American Journal of Medical Genetics Part A · doi:10.1002/ajmg.a.63878 · Cited by 37 · see citing works
Applying fine-tuned LLMs for reducing data needs in load profile analysis
Hu, Y., Kim, H., Ye, K., & Lü, N. (2024) · Applied Energy · doi:10.1016/j.apenergy.2024.124666 · Cited by 34 · see citing works
Evaluating search engines and large language models for answering health questions
Fernández-Pichel, M., Pichel, J. C., & Losada, D. E. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01546-w · Cited by 33 · see citing works
Survey of Multimodal Medical Question Answering
Demirhan, H., & Zadrozny, W. (2023) · BioMedInformatics · doi:10.3390/biomedinformatics4010004 · Cited by 32 · see citing works
Bioinformatics and biomedical informatics with ChatGPT: Year one review
Wang, J., Cheng, Z., Yao, Q., Liu, L., Xu, D. H., & Hu, G. (2024) · Quantitative Biology · doi:10.1002/qub2.67 · Cited by 34 · see citing works
Code Interpreter for Bioinformatics: Are We There Yet?
Wang, L., Ge, X., Liu, L., & Hu, G. (2023) · Annals of Biomedical Engineering · doi:10.1007/s10439-023-03324-9 · Cited by 31 · see citing works
Artificial Intelligence-Based ChatGPT Responses for Patient Questions on Optic Disc Drusen
Potapenko, I. O., Malmqvist, L., Subhi, Y., & Hamann, S. (2023) · Ophthalmology and Therapy · doi:10.1007/s40123-023-00800-2 · Cited by 29 · see citing works
Ergene, Ö., & ERGENE, B. Ç. (2024) · Education and Information Technologies · doi:10.1007/s10639-024-13121-z · Cited by 29 · see citing works
Generative artificial intelligence: implications for biomedical and health professions education
Hersh, W. (2025) · Annual Review of Biomedical Data Science · doi:10.1146/annurev-biodatasci-103123-094756 · Cited by 28 · see citing works
Flaharty, K. A., Hu, P., Hanchard, S. L., Ripper, M. E., Duong, D., Waikel, R. L., & Solomon, B. D. (2024) · The American Journal of Human Genetics · doi:10.1016/j.ajhg.2024.07.011 · Cited by 26 · see citing works
Self-citationWang, J., Ye, Q., Liu, L., Guo, N. L., & Hu, G. (2024) · npj Precision Oncology · doi:10.1038/s41698-024-00576-z · Cited by 25 · see citing works
FoodSky: A food-oriented large language model that can pass the chef and dietetic examinations
Zhou, P., Min, W., Fu, C., Jin, Y., Huang, M., Li, X., Mei, S., & Jiang, S. (2025) · Patterns · doi:10.1016/j.patter.2025.101234 · Cited by 26 · see citing works
An Overarching Framework for the Ethics of Artificial Intelligence in Pediatrics
Sisk, B. A., Antes, A. L., & DuBois, J. M. (2024) · Archives of Pediatrics and Adolescent Medicine · doi:10.1001/jamapediatrics.2023.5761 · Cited by 25 · see citing works
Shojaee-Mend, H., Mohebbati, R., Amiri, M., & Atarodi, A. (2024) · Scientific Reports · doi:10.1038/s41598-024-60405-y · Cited by 24 · see citing works
Evaluation of ChatGPT's performance in providing treatment recommendations for pediatric diseases
Wei, Q., Wang, Y., Yao, Z., Cui, Y., Wei, B., Li, T., & Xu, X. (2023) · Pediatric Discovery · doi:10.1002/pdi3.42 · Cited by 23 · see citing works
SELL ME THIS ARTIFICIAL PEN: USING CHATGPT TO ENHANCE SALES ROLE PLAYS
Milović, A., Gyomlai, M. D., Spaid, B. I., & Dingus, R. (2024) · Marketing Education Review · doi:10.1080/10528008.2024.2341278 · Cited by 23 · see citing works
Chatterjee, S., Bhattacharya, M., Lee, S., & Chakraborty, C. (2023) · Molecular Therapy — Nucleic Acids · doi:10.1016/j.omtn.2023.06.019 · Cited by 22 · see citing works
Kim, J., Lee, S., Jeon, H., Lee, K., Bae, H., Kim, B., & Seo, J. (2024) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2024.3456215 · Cited by 25 · see citing works
Arakawa, R., Lehman, J. F., & Goel, M. (2024) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3699759 · Cited by 22 · see citing works
Wang, Y., Liang, L., Li, R., Wang, Y., & Hao, C. (2024) · Journal of Multidisciplinary Healthcare · doi:10.2147/jmdh.s473680 · Cited by 20 · see citing works
Applications and Future Prospects of Medical LLMs: A Survey Based on the M-KAT Conceptual Framework
Chang, Y., Yin, J., Li, J., Liu, C., Cao, L., & Lin, S. (2024) · Journal of Medical Systems · doi:10.1007/s10916-024-02132-5 · Cited by 19 · see citing works
Ma, D., Zhang, C., Xu, Q., & Zhou, G. (2025) · Robotics and Computer-Integrated Manufacturing · doi:10.1016/j.rcim.2025.103078 · Cited by 21 · see citing works
Cherif, H., Moussa, C., Missaoui, A. M., Salouage, I., Mokaddem, S., & Dhahri, B. (2024) · JMIR Medical Education · doi:10.2196/52818 · Cited by 18 · see citing works
Sorenson, B., & Hanson, K. (2024) · Journal of Chemical Education · doi:10.1021/acs.jchemed.4c00165 · Cited by 17 · see citing works
A Retrieval-augmented Generation application for Question-Answering in Nutrigenetics Domain
Benfenati, D., Filippis, G. M. D., Rinaldi, A. M., Russo, C., & Tommasino, C. (2024) · Procedia Computer Science · doi:10.1016/j.procs.2024.09.467 · Cited by 20 · see citing works
Perspectives of ChatGPT in Pharmacology Education, and Research in Health Care: A Narrative Review
Patel, C., Pandya, S. K., & Sojitra, B. (2023) · Journal of Pharmacology and Pharmacotherapeutics · doi:10.1177/0976500x231210427 · Cited by 16 · see citing works
Meo, S. A., Alotaibi, M. S., Meo, M. Z. S., Meo, M. O. S., & Hamid, M. (2024) · Frontiers in Public Health · doi:10.3389/fpubh.2024.1360597 · Cited by 16 · see citing works
ChatGPT and Medicine: Together We Embrace the AI Renaissance
Hacking, S. M. (2024) · JMIR Bioinformatics and Biotechnology · doi:10.2196/52700 · Cited by 15 · see citing works
Zarei, M., Zarei, M., Hamzehzadeh, S., Oliyaei, S. S. B., & Hosseini, M. (2024) · Shiraz E-Medical Journal · doi:10.5812/semj-145840 · Cited by 17 · see citing works
Kharitonova, K., Pérez‐Fernández, D., Gutiérrez-Hernando, J., Gutiérrez-Fandiño, A., Callejas, Z., & Griol, D. (2024) · Behaviour and Information Technology · doi:10.1080/0144929x.2024.2321959 · Cited by 14 · see citing works
Unveiling the risks of ChatGPT in diagnostic surgical pathology
Guastafierro, V., Corbitt, D. N., Bressan, A., Fernandes, B., Mintemur, Ö., Magnoli, F., Ronchi, S., La Rosa, S., Uccella, S., & Renne, S. L. (2025) · Virchows Archiv · doi:10.1007/s00428-024-03918-1 · Cited by 14 · see citing works
Cao, C., Zhao, W., Guo, J., Wang, Z., Lü, A., & Liang, C. C. (2025) · The Innovation Medicine · doi:10.59717/j.xinn-med.2025.100154 · Cited by 14 · see citing works
Ramadan, S., Mutsaers, A., Chen, P.-H. C., Bauman, G. S., Velker, V. M., Ahmad, B., Arifin, A. J., Nguyen, T. K., Palma, D., & Goodman, C. D. (2024) · Radiotherapy and Oncology · doi:10.1016/j.radonc.2024.110645 · Cited by 13 · see citing works
Jeon, S., Lee, S., Chung, H., Yun, J. Y., Park, E. A., So, M.-K., & Huh, J. (2025) · Diagnostics · doi:10.3390/diagnostics15060672 · Cited by 12 · see citing works
Braun, E.-M., Juhasz‐Böss, I., Solomayer, E., Truhn, D., Keller, C., Heinrich, V., & Braun, B. J. (2023) · Archives of Gynecology and Obstetrics · doi:10.1007/s00404-023-07272-6 · Cited by 13 · see citing works
Can ChatGPT understand genetics?
Emmert‐Streib, F. (2023) · European Journal of Human Genetics · doi:10.1038/s41431-023-01419-4 · Cited by 12 · see citing works
Li, Y., Gao, W., Luan, Z., Zhou, Z., & Li, J. (2023) · Cureus · doi:10.7759/cureus.48670 · Cited by 11 · see citing works
Altamimi, I., Alhumimidi, A. S., Alshehri, S., AlRumayan, A. F., Al-khlaiwi, T., Meo, S. A., & Temsah, M.-H. A. (2024) · Annals of Medicine and Surgery · doi:10.1097/ms9.0000000000002120 · Cited by 11 · see citing works
Artificial intelligence and the impact on medical genetics
Solomon, B. D., & Chung, W. K. (2023) · American Journal of Medical Genetics Part C Seminars in Medical Genetics · doi:10.1002/ajmg.c.32060 · Cited by 10 · see citing works
Self-citationIs artificial intelligence getting too much credit in medical genetics?
Alkuraya, I. F. (2023) · American Journal of Medical Genetics Part C Seminars in Medical Genetics · doi:10.1002/ajmg.c.32062 · Cited by 10 · see citing works
Perspectives on the Current and Future State of Artificial Intelligence in Medical Genetics
Solomon, B. D., Cheatham, M., de Guimarães, T. A. C., Duong, D., Haendel, M. A., Hsieh, T., Javanmardi, B., Johnson, B. A., Krawitz, P. M., Kruszka, P. S., et al. (2025) · American Journal of Medical Genetics Part A · doi:10.1002/ajmg.a.64118 · Cited by 9 · see citing works
Self-citationGhimire, S., Lin, Y.-Z., Mamun, M., Chowdhury, M. A., Alemi, F., Cai, S., Guo, J., Zhu, M., Li, H., Latibari, B. S., et al. (2025) · ACM Transactions on Design Automation of Electronic Systems · doi:10.1145/3737459 · Cited by 9 · see citing works
Just how transformative will AI/ML be for immuno-oncology?
Bottomly, D., & McWeeney, S. K. (2024) · Journal for ImmunoTherapy of Cancer · doi:10.1136/jitc-2023-007841 · Cited by 9 · see citing works
A benchmark for large language models in bioinformaticsPreprint
Sarwal, V., Andreoletti, G., Munteanu, V., Suhodolschi, A., Ciorbă, D., Bostan, V., Dimian, M., Eskin, E., Wang, W., & Mangul, S. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.12.19.572483 · Cited by 9 · see citing works
Generative AI and the profession of genetic counseling
Meekins‐Doherty, L., Dive, L. L., McEwen, A., & Sexton, A. (2025) · Journal of Genetic Counseling · doi:10.1002/jgc4.2009 · Cited by 9 · see citing works
On the Responsible Use of Chatbots in Bioinformatics
Hu, G., Liu, L., & Xu, D. H. (2024) · Genomics Proteomics & Bioinformatics · doi:10.1093/gpbjnl/qzae002 · Cited by 8 · see citing works
Karami, A., Qiao, Z., Zhang, X., Kharrazi, H. H. K., Bozorgi, P., & Bozorgi, A. (2024) · Big Data and Cognitive Computing · doi:10.3390/bdcc8100130 · Cited by 8 · see citing works
Importance of critical thinking to understand ChatGPT
Emmert‐Streib, F. (2023) · European Journal of Human Genetics · doi:10.1038/s41431-023-01443-4 · Cited by 7 · see citing works
Comparison of clinical geneticist and computer visual attention in assessing genetic conditions
Duong, D., Johny, A. R., Hanchard, S. E. L., Fortney, C., Flaharty, K. A., Hellmann, F., Hu, P., Javanmardi, B., Moosa, S., Patel, T., et al. (2024) · PLoS Genetics · doi:10.1371/journal.pgen.1011168 · Cited by 6 · see citing works
Self-citationApplying IRT to Distinguish Between Human and Generative AI Responses to Multiple-Choice Assessments
Strugatski, A., & Alexandron, G. (2025) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/3706468.3706490 · Cited by 5 · see citing works
A Survey of Large-Scale Deep Learning Models in Medicine and Healthcare
Chen, Z., Liu, R., Huang, S., Guo, Y., & Ren, Y. (2025) · Computer Modeling in Engineering & Sciences · doi:10.32604/cmes.2025.067809 · Cited by 6 · see citing works
Meng, J., Zhang, Z., Tang, H., Xiao, Y., Liu, P., Gao, S., & He, M. (2024) · Medicine · doi:10.1097/md.0000000000037458 · Cited by 5 · see citing works
Is ChatGPT a Reliable Tool for Explaining Medical Terms?
Oztermeli, A. D. (2025) · Cureus · doi:10.7759/cureus.77258 · Cited by 5 · see citing works
SheepDoctor: A knowledge graph enhanced large language model for sheep disease diagnosis
Xiong, J., Zhou, Y., Tian, F., Ni, F., & Zhao, L. (2025) · Smart Agricultural Technology · doi:10.1016/j.atech.2025.101001 · Cited by 6 · see citing works
Does Using ChatGPT Result in Human Cognitive Augmentation?
Fulbright, R., & Morrison, M. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-61569-6_9 · Cited by 5 · see citing works
AI vs. Human Programmers: Complexity and Performance in Code Generation
Azeem, S., Naveed, M. S., Sajid, M., & Ali, I. (2025) · VAWKUM Transactions on Computer Sciences · doi:10.21015/vtcs.v13i1.2043 · Cited by 5 · see citing works
- Authors
- Duong, D., & Solomon, B. D.
- Affiliations
- National Institutes of Health (US)
- Corresponding author
- Solomon, Benjamin D.
- ORCID
- Solomon, Benjamin D.
- Venue
- European Journal of Human Genetics
- Published
- 2023-05-29
- Volume
- 32
- Issue
- 4
- Pages
- 466-468
- ISSN
- 1018-4813 (ISSN-L)
- 1476-5438
- PMC
- PMC10999420
- OpenAlex
- W4378603221
- Cited by
- 159
- References
- 12
- Citations by year
- 2026: 15 · 2025: 42 · 2024: 59 · 2023: 42 · 2019: 1
- FWCI
- 6.29
- Type
- journal article
- Language
- en
- Open access
- green
- doi.org
- Licence
- Locations
- European Journal of Human Genetics (publishedVersion)
- medRxiv (acceptedVersion)
- PubMed Central (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (submittedVersion)
- Funders
- U.S. Department of Health & Human Services | NIH | National Human Genome Research Institute (N/A)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Biomedical Text Mining and Ontologies
- Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
- Keywords
- large language models, ChatGPT, human genetics education, multiple-choice questions
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
