XAI for medicine by ChatGPT code interpreter

Abstract

In recent years, with the prevalence of Artificial Intelligence (AI), the interpretability of AI outputs has become a significant issue. Especially the interpretability of large language models (LLMs), including ChatGPT, has emerged as a major challenge. Consequently, there is a growing interest in the research of Explainable Artificial Intelligence (XAI), which seeks to elucidate the decision-making processes of AI in a manner that humans can comprehend. In the medical field, where trust and transparency are important, the use of AI becomes challenging when its decisions are unclear. Therefore, XAI techniques become critically important in the medical field. In this study, we propose the prompt named Code Base Prompt (CBP) to make the ChatGPT's decision-making process on medical texts explainable by using the Python code execution function of Chat GPT Code interpreter. In CBP, the medical decision-making algorithm is rewritten as Python code. Moreover, we propose an explainability evaluation system named Medical Algorithm Presentation Criteria (MAPC) for medical algorithm application tasks to medical text. MAPC is evaluated by five factors to align the human understanding process. To compare CBP with a Text Base Prompt (TBP), we conducted an experiment applying the heart failure classification algorithm to heart failure case report texts in three medical articles. With CBP, the results showed that the ChatGPT Code interpreter executed the Python code in all three cases and met all the five MAPC factors. In contrast, with TBP, no Python code execution was observed in any of the three cases, validating only one factor of MAPC. This study presents a new method for implementing XAI in the use of ChatGPT for medical tasks.