Assessing the Accuracy and Reliability of AI-Generated Medical Responses: An Evaluation of the Chat-GPT Model

Johnson, Douglas; Goodman, Rachel; Patrinely, J; Stone, Cosby; Zimmerman, Eli; Donald, Rebecca; Chang, Sam; Berkowitz, Sean; Finn, Avni; Jahangir, Eiman; Scoville, Elizabeth; Reese, Tyler; Friedman, Debra; Bastarache, Julie; Heijden, Yuri van der; Wright, Jordan; Carter, Nicholas; Alexander, Matthew; Choe, Jennifer; Chastain, Cody; Zic, John; Horst, Sara; Turker, Isik; Agarwal, Rajiv; Osmundson, Evan; Idrees, Kamran; Kiernan, Colleen; Padmanabhan, Chandrasekhar; Bailey, Christina; Schlegel, Cameron; Chambless, Lola; Gibson, Mike; Osterman, Travis; Wheless, Lee

doi:10.21203/rs.3.rs-2566942/v1

Assessing the Accuracy and Reliability of AI-Generated Medical Responses: An Evaluation of the Chat-GPT Model

Background: Natural language processing models such as ChatGPT can generate text-based content and are poised to become a major information source in medicine and beyond. The accuracy and completeness of ChatGPT for medical queries is not known. Methods: Thirty-three physicians across 17 specialt…