Towards Medical Complex Reasoning with LLMs through Medical Verifiable Problems
Open accessAbstract
The breakthrough of OpenAI o1 highlights the potential of enhancing reasoning to improve LLM.Yet, most research in reasoning has focused on mathematical tasks, leaving domains like medicine underexplored.The medical domain, though distinct from mathematics, also demands robust reasoning to provide reliable answers, given the high standards of healthcare.However, verifying medical reasoning is challenging, unlike those in mathematics.To address this, we propose Medical Verifiable Problems with a medical verifier to check the correctness of model outputs.This verifiable nature enables advancements in medical reasoning through a two-stage approach: (1) using the verifier to guide the search for a complex reasoning trajectory for fine-tuning LLMs, (2) applying reinforcement learning (RL) with verifier-based rewards to enhance complex reasoning further.Finally, we introduce HuatuoGPT-o1, a medical LLM capable of complex reasoning, which outperforms general and medical-specific baselines using only 40K verifiable problems.Experiments show complex reasoning improves medical problem-solving and benefits more from RL.We hope our approach inspires advancements in reasoning across medical and other specialized domains.Code, datasets, and models are publicly available at https://github.com/FreedomIntelligence/Huatuo GPT-o1.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Reasoning traces used to train HuatuoGPT-o1 were generated by GPT-4o, while those used for m1 were synthesized by DeepSeek-R1.
“While the reasoning traces used to train HuatuoGPT-o1 were generated by GPT-4o [6,29], those used for m1 were synthesized by DeepSeek-R1 [9,34].”
From Model and Task-Aware Test-Time Scaling Strategies for Large Language and Vision-Language Models in Medicine: Evaluation Study (2026) · cites this paper as [29]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 27
- Sina Alemohammad, Josue Casco-Rodriguez, Lorenzo Luzi, Ahmed Imtiaz Humayun, Hossein Babaei, Daniel LeJeune, Ali Siahkoohi, and Richard G. Bara- niuk. 2024. Self-consuming generative models go MAD. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Aus- tria, May 7-11, 2024. OpenReview.net.
- Daixuan Cheng, Shaohan Huang, and Furu Wei. 2023. Adapting large language models via reading compre- hension. In The Twelfth International Conference on Learning Representations.
- Zi-Yi Dou, Cheng-Fu Yang, Xueqing Wu, Kai-Wei Chang, and Nanyun Peng. 2024. Re-rest: Reflection- reinforced self-training for language agents. In Pro- ceedings of the 2024 Conference on Empirical Meth- ods in Natural Language Processing, EMNLP 2024, Miami, FL, USA, November 12-16, 2024, pages 15394-15411. Association for Computational Lin- guistics.
- Alexander Havrilla, Sharath Chandra Raparthy, Christo- foros Nalmpantis, Jane Dwivedi-Yu, Maksym Zhu- ravinskyi, Eric Hambro, and Roberta Raileanu. 2024. Glore: When, where, and how to improve LLM rea- soning via global and local refinements. In Forty- first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024. Open- Review.net.
- Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xiny- ing Song, and Denny Zhou. 2024. Large language models cannot self-correct reasoning yet. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net.
- Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. 2021. What disease does this patient have? a large-scale open domain question answering dataset from medical exams. Ap- plied Sciences, 11(14):6421.
- Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harri- son Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2024. Let's verify step by step. In The Twelfth In- ternational Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. Open- Review.net.
- Junling Liu, Peilin Zhou, Yining Hua, Dading Chong, Zhongyu Tian, Andrew Liu, Helin Wang, Chenyu You, Zhenhua Guo, Lei Zhu, et al. 2024. Bench- marking large language models on cmexam-a com- prehensive chinese medical exam dataset. Advances in Neural Information Processing Systems, 36.
- Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdan- bakhsh, and Peter Clark. 2023. Self-refine: Itera- tive refinement with self-feedback. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Sys- tems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 -16, 2023.
- Ankit Pal, Logesh Kumar Umapathi, and Malaikan- nan Sankarasubbu. 2022a. Medmcqa: A large-scale multi-subject multi-choice dataset for medical do- main question answering. In Conference on health, inference, and learning, pages 248-260.
- PMLR.
- Ankit Pal, Logesh Kumar Umapathi, and Malaikan- nan Sankarasubbu. 2022b. Medmcqa: A large-scale multi-subject multi-choice dataset for medical do- main question answering. In Conference on Health, Inference, and Learning, pages 248-260. PMLR.
- Vimla L Patel, José F Arocha, and Jiajie Zhang. 2005. Thinking and reasoning in medicine. The Cambridge handbook of thinking and reasoning, 14:727-750.
- Debjit Paul, Mete Ismayilzada, Maxime Peyrard, Beat- riz Borges, Antoine Bosselut, Robert West, and Boi Faltings. 2024. REFINER: reasoning feedback on intermediate representations. In Proceedings of the 18th Conference of the European Chapter of the As- sociation for Computational Linguistics, EACL 2024 -Volume 1: Long Papers, St. Julian's, Malta, March 17-22, 2024, pages 1100-1126. Association for Com- putational Linguistics.
- Martin Riedmiller, Roland Hafner, Thomas Lampe, Michael Neunert, Jonas Degrave, Tom Wiele, Vlad Mnih, Nicolas Heess, and Jost Tobias Springenberg. 2018. Learning by playing solving sparse reward tasks from scratch. In International conference on machine learning, pages 4344-4353. PMLR.
- David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, et al. 2017. Mastering the game of go without human knowledge. nature, 550(7676):354-359.
- Karan Singhal, Shekoofeh Azizi, Tao Tu, S Sara Mah- davi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, et al. 2023a. Large language models encode clinical knowledge. Nature, 620(7972):172-180.
- Mohamad-Hani Temsah, Amr Jamal, Khalid Alhasan, Abdulkarim A Temsah, and Khalid H Malki. 2024. Openai o1-preview vs. chatgpt in healthcare: A new frontier in medical ai reasoning. Cureus, 16(10):e70640.
- Alexander Trott, Stephan Zheng, Caiming Xiong, and Richard Socher. 2019. Keeping your distance: Solv- ing sparse reward tasks using self-balancing shaped rewards. Advances in Neural Information Processing Systems, 32.
- Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, Sharan Narang, Aakanksha Chowd- hery, and Denny Zhou. 2023d. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net.
- Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. In Ad- vances in Neural Information Processing Systems 35: Annual Conference on Neural Information Process- ing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 -December 9, 2022.
- Sean Welleck, Ximing Lu, Peter West, Faeze Brah- man, Tianxiao Shen, Daniel Khashabi, and Yejin Choi. 2023. Generating sequences by learning to self-correct. In The Eleventh International Confer- ence on Learning Representations, ICLR 2023, Ki- gali, Rwanda, May 1-5, 2023. OpenReview.net.
- Chaoyi Wu, Weixiong Lin, Xiaoman Zhang, Ya Zhang, Weidi Xie, and Yanfeng Wang. 2024. Pmc-llama: toward building open-source language models for medicine. Journal of the American Medical Infor- matics Association, page ocae045.
- Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah D. Goodman. 2022. Star: Bootstrapping reasoning with reasoning. In Advances in Neural Information Pro- cessing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 -December 9, 2022.
- Aohan Zeng, Xiao Liu, Zhengxiao Du, Zihan Wang, Hanyu Lai, Ming Ding, Zhuoyi Yang, Yifan Xu, Wendi Zheng, Xiao Xia, Weng Lam Tam, Zixuan Ma, Yufei Xue, Jidong Zhai, Wenguang Chen, Zhiyuan Liu, Peng Zhang, Yuxiao Dong, and Jie Tang. 2023. GLM-130b: An open bilingual pre-trained model. In The Eleventh International Conference on Learning Representations (ICLR).
- Kai Zhang, Jun Yu, Eashan Adhikarla, Rong Zhou, Zhiling Yan, Yixin Liu, Zhengliang Liu, Lifang He, Brian Davison, Xiang Li, et al. 2023. Biomedgpt: a unified and generalist biomedical generative pre- trained transformer for vision, language, and multi- modal tasks. arXiv e-prints, pages arXiv-2305.
- Aojun Zhou, Ke Wang, Zimu Lu, Weikang Shi, Sichun Luo, Zipeng Qin, Shaoqing Lu, Anya Jia, Linqi Song, Mingjie Zhan, and Hongsheng Li. 2024. Solving challenging math word problems using GPT-4 code interpreter with code-based self-verification. In The Twelfth International Conference on Learning Rep- resentations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net.
Cited by
Sorted by citations
Automating expert-level medical reasoning evaluation of large language models
Zhou, S., Xie, W., Li, J., Zhan, Z., Song, M., Yang, H., Espinoza, C., Welton, L., Mai, X., Jin, Y., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-02208-7 · Cited by 30 · see citing works
Abdallah, M., Nakken, S., Georges, M., Bierkens, M., Galvis, J., Groppi, A., Karkar, S., Meiqari, L., Rujano, M. A., Canham, S., et al. (2026) · Nature Communications · doi:10.1038/s41467-026-70509-w · Cited by 14 · see citing works
BRIDGE: benchmarking large language models for understanding real-world clinical practice texts
Wu, J., Gu, B., Zhou, R., Xie, K., Snyder, D., Jiang, Y., Carducci, V., Wyss, R., Desai, R. J., Alsentzer, E., et al. (2026) · Nature Biomedical Engineering · doi:10.1038/s41551-026-01719-2 · Cited by 14 · see citing works
Self-citationZhu, Y., Gao, J., Wang, Z., Liao, W., Zheng, X., Liang, L., Bernabéu, M. O., Wang, Y., Yu, L., Pan, C., et al. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02539-z · Cited by 12 · see citing works
Byun, H., Lee, D., Jung, M., & Jang, B. (2026) · Journal of Medical Systems · doi:10.1007/s10916-026-02440-y · Cited by 3 · see citing works
MediQAl: A French Medical Question Answering Dataset for Knowledge and Reasoning Evaluation
Bazoge, A. (2026) · Scientific Data · doi:10.1038/s41597-026-06680-y · Cited by 2 · see citing works
Oh, G., Kim, S., Park, S., & Kim, B.-H. (2026) · Journal of Medical Internet Research · doi:10.2196/90693 · Cited by 1 · see citing works
“While the reasoning traces used to train HuatuoGPT-o1 were generated by GPT-4o [6,29], those used for m1 were synthesized by DeepSeek-R1 [9,34].”
InfluentialA Context Adaptive Instruction Tuning Framework for Diverse Clinical NLP Tasks
Mukalazi, S. H., Ouyang, J., & Katende, J. (2025) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm66473.2025.11356497 · Cited by 1 · see citing works
Learning to Reason for Multi-Step Retrieval of Personal Context in Personalized Question Answering
Amirizaniani, M., Salemi, A., & Zamani, H. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3809944 · Cited by 1 · see citing works
The economics of accuracy for medical reasoning with large language models
Bhattacharyya, K., & Kamabattula, S. (2026) · PLOS Digital Health · doi:10.1371/journal.pdig.0001182
MeDRNet: A Knowledge-Augmented Multi-model Framework for Robust Medical Language Understanding
Long, X., Zhao, Y., & Liu, J. (2026) · Lecture notes in computer science · doi:10.1007/978-981-95-7081-2_22
Robust Medical Text Correction via Domain Adaptation and Term-Aware Safety Control
Yin, B., & Huang, S. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3417-2_22
Towards Automatic Incremental Learning: A Self-adaptive Framework for Continual Instruction Tuning
Lin, P., Zhang, F., Niu, K., Lam, S.-K., & Fu, H. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3426-4_4
Zhu, Y.-H., Wang, Z.-X., Qi, Y.-F., Gu, L., Sui, D.-H., Hu, H., Zhang, X.-C., He, Z., Wang, Y.-S., He, J., et al. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-03097-0 · Cited by 1 · see citing works
MEMRL: Aligning Medical Reasoning through Maximum Entropy Implicit Process Supervision
Zhu, X., & He, S. (2026) · IEEE International Conference on Electronics, Energy Systems and Power Engineering (EESPE) · doi:10.1109/eespe68405.2026.11649286
- Authors
- Chen, J., Cai, Z., Ji, K., Wang, X., Liu, W., Wang, R., & Wang, B.
- Affiliations
- Chinese University of Hong Kong, Shenzhen (CN)
- Shenzhen Research Institute of Big Data (CN)
- ORCID
- Ji, Ke
- Venue
- Findings of the Association for Computational Linguistics: ACL 2025
- Published
- 2025
- Pages
- 14552-14573
- OpenAlex
- W4412888062
- Cited by
- 14
- References
- 0
- Citations by year
- 2026: 12 · 2025: 2
- FWCI
- 14.64
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Findings of the Association for Computational Linguistics: ACL (Findings of ACL) (publishedVersion, cc-by)
- Funders
- National Natural Science Foundation of China (12326608)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- medical reasoning, large language models, reinforcement learning, LLM fine-tuning, OpenAI o1
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
