CoChat
Sign Up
PreprintCited by 1Artificial Intelligence in Healthcare and EducationArtificial Intelligence in HealthcareMachine Learning in Healthcare

Orchestrator Multi-Agent Clinical Decision Support System for Secondary Headache Diagnosis in Primary Care

Open access
Wu, Xizhi, Garduno-Rapp, Nelly Estefanie, Rousseau, Justin F, Thakkallapally, Mounika, Zhang, Hang, Ji, Yuelyu, Visweswaran, Shyam, Peng, Yifan +1 more·2025·arXiv
DOI 10.48550/arxiv.2512.04207arXiv 2512.04207PMC PMC13419605
Open this paper in CoChat — free account

Abstract

Unlike most primary headaches, secondary headaches need specialized care and can have devastating consequences if not treated promptly. Clinical guidelines highlight several 'red flag' features, such as thunderclap onset, meningismus, papilledema, focal neurologic deficits, signs of temporal arteritis, systemic illness, and the 'worst headache of their life' presentation. Despite these guidelines, determining which patients require urgent evaluation remains challenging in primary care settings. Clinicians often work with limited time, incomplete information, and diverse symptom presentations, which can lead to under-recognition and inappropriate care. We present a large language model (LLM)-based multi-agent clinical decision support system built on an orchestrator-specialist architecture, designed to perform explicit and interpretable secondary headache diagnosis from free-text clinical vignettes. The multi-agent system decomposes diagnosis into seven domain-specialized agents, each producing a structured and evidence-grounded rationale, while a central orchestrator performs task decomposition and coordinates agent routing. We evaluated the multi-agent system using 90 expert-validated secondary headache cases and compared its performance with a single-LLM baseline across two prompting strategies: question-based prompting (QPrompt) and clinical practice guideline-based prompting (GPrompt). We tested five open-source LLMs (Qwen-30B, GPT-OSS-20B, Qwen-14B, Qwen-8B, and Llama-3.1-8B), and found that the orchestrated multi-agent system with GPrompt consistently achieved the highest F1 scores, with larger gains in smaller models. These findings demonstrate that structured multi-agent reasoning improves accuracy beyond prompt engineering alone and offers a transparent, clinically aligned approach for explainable decision support in secondary headache diagnosis.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2512.04207

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Multi-agent systems distribute clinical reasoning across specialized agents such as history-takers, evidence retrievers, diagnosticians, and decision-makers, with inter-agent communication channels serving as inherently traceable explanation mechanisms.

    “These multi-agent systems [66,77,88,102,118] distribute clinical reasoning across specialized agents such as history-takers, evidence retrievers, diagnosticians, and decision-makers, with inter-agent communication channels serving as inherently traceable explanation mechanisms.”

    From Explainability of decoder-only clinical large language models: A scoping review (2026) · cites this paper as [88]

    Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

    Experimental — accuracy not yet measured

  • Others expose intermediate reasoning steps such as agentic traces or structured decoding chains that allow clinicians to audit the logic.

    “Others expose intermediate reasoning steps such as agentic traces, structured decoding chains etc that allow clinicians to audit the logic , as in agentic systems [66,77,88,102,118] and structureddecoding approaches [65,101].”

    From Explainability of decoder-only clinical large language models: A scoping review (2026) · cites this paper as [88]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 29

  1. [1]Do TP, Remmers A, Schytz HW, et al Red and orange flags for secondary headaches in clinical practice: SNNOOP10 list. Neurology. 2019; 92: 134- 144. PMC6340385 30587518
  2. [2]Hernandez J, Molina E, Rodriguez A, et al Headache disorders: differentiating primary and secondary etiologies. J Integr Neurosci. 2024; 23: 43. 38419454
  3. [3]Zhao WX, Zhou K, Li J, et al A survey of large language models. Frontiers of Computer Science. 2026 Dec;20(12):2012627.
  4. [4]Guo T, Chen X, Wang Y, Chang R, Pei S, Chawla NV, Wiest O, Zhang X. Large language model based multi-agents: A survey of progress and challenges. arXiv preprint arXiv:2402.01680. 2024 Jan 21. · arXiv
  5. [5]Hong S, Zhuge M, Chen J et al MetaGPT: meta programming for a multi-agent collaborative framework. In: International Conference on Learning Representations, 2023.
  6. [6]Juliana CC-CV. Diagnosing secondary headaches. Pract. Neurol. 2020; 20: 31- 40.
  7. [7]Dorri A, Kanhere S, Jurdak R. Multi-agent systems: a survey. IEEE Access. 2018; 6: 28573- 28593.
  8. [8]Tran KT, Dao D, Nguyen MD, Pham QV, O'Sullivan B, Nguyen HD. Multi-agent collaboration mechanisms: A survey of llms. arXiv preprint arXiv:2501.06322. 2025 Jan 10. · arXiv
  9. [9]Dang Y, Qian C, Luo X, Fan J, Xie Z, Shi R, Chen W, Yang C, Che X, Tian Y, Xiong X. Multi-agent collaboration via evolving orchestration. Advances in neural information processing systems. 2026 Apr 23;38: 165025– 59.
  10. [10]Yang F, Meng T, Torben-Nielsen B, et al A machine learning approach to support triaging of primary versus secondary headache patients using complete blood count. PLoS One. 2023; 18: e0282237. PMC9987784 36877693
  11. [11]Acosta JN, Dorr F, Goicochea MT, Fernández Slezak D, Farez MF. Acute headache diagnosis in the emergency department: accuracy and safety of an artificial intelligence system (P5.10-002). Neurology. 2019; 92: P5.10-002.
  12. [12]Garduno Rapp NE, Thakkallapally M, Rousseau J. Capturing clinical red flags for secondary headaches: a guideline-based ontology approach [Poster session]. 2025 Texas Regional CTSCA Consortium Conference, June 12, 2025. https://hdl.handle.net/2152.5/10736
  13. [15]Bittel B, Husmann K. A case report of thunderclap headache with sub-arachnoid hemorrhage and negative angiography: a review of Call-Fleming syndrome and the use of transcranial dopplers in predicting morbidity. J Vasc Interv Neurol. 2011; 4: 5- 8. PMC3317281 22518264
  14. [16]Boddhula SK, Boddhula S, Gunasekaran K, Bischof E. An unusual cause of thunderclap headache after eating the hottest pepper in the world—“The Carolina Reaper”. BMJ Case Rep. 2018; 2018: bcr2017224085. PMC5893965 29632122
  15. [17]Lwin ZT, Yong Z. Case report of a rare condition of thunderclap headache. G Med Sci. 2020; 1: 001- 004.
  16. [18]Byrum EP, McGregor JM, Christoforidis GA. Thunderclap headache without subarachnoid hemorrhage associated with regrowth of previously coil-occluded aneurysms. AJNR Am J Neuroradiol. 2009; 30: 1059- 1061. PMC7051640 19039039
  17. [20]Tang S, May JL, Lee JH. The case of a 19-year-old woman with headache, papilledema, and diplopia. Ann Clin Transl Neurol. 2021; 8: 2038- 2039. PMC8528459 34533291
  18. [21]Tagoe NN, Beyuo VM, Amissah-Arthur KN. Case series of six patients diagnosed and managed for idiopathic intracranial hypertension at a tertiary institution eye centre. Ghana Med J. 2019; 53: 79- 87. PMC6527823 31138948
  19. [22]Toljan K, Aboseif A, Bireley JD, Moss B. Headache and papilledema: a case of neurosarcoidosis with cerebrovascular involvement. Neuroimmunol Rep. 2023; 3: 100165.
  20. [23]Laldinpuii J, Sanchetee P, Borah AL, Ghose M, Borah NC. Giant cell arteritis (temporal arteritis): a report of four cases from north east India. Ann Indian Acad Neurol. 11: 185- 189. PMC2771971 19893667
  21. [24]Devi S, Dash A, Purkait S, Sahoo B. Giant cell arteritis masquerading as migraine: a case report. Cureus. 2023; 15: e44107. PMC10518159 37750130
  22. [25]Silverman A, Dugue R, George PM. Clinical problem solving: a 38-year-old woman with systemic lupus erythematosus presenting with headache, nausea, and vomiting. Neurohospitalist. 2023; 13: 394- 398. PMC10494810 37701245
  23. [26]Zhao Z, Jin Q, Chen F, Peng T, Yu S. A large-scale dataset of patient summaries for retrieval-based clinical decision support systems. Sci Data. 2023; 10: 909. PMC10728216 38110415
  24. [27]Olesen J. International Classification of Headache Disorders. Lancet Neurol. 2018; 17: 396- 397. 29550365
  25. [28]Shorten C, Pierse C, Smith TB, Cardenas E, Sharma A, Trengrove J, van Luijt B. Structuredrag: Json response formatting with large language models. arXiv preprint arXiv:2408.11061. 2024 Aug 7. · arXiv
  26. [29]Zhou J, Chen J, Lu Q, Zhao D, Zhu L. Shielda: Structured handling of exceptions in llm-driven agentic workflows. arXiv preprint arXiv:2508.07935. 2025 Aug 11. · arXiv
  27. [33]Yang A, Li A, Yang B, Zhang B, Hui B, Zheng B, Yu B, Gao C, Huang C, Lv C, Zheng C. Qwen3 technical report. arXiv preprint arXiv:2505.09388. 2025 May 14. · arXiv
  28. [34]Grattafiori A, Dubey A, Jauhri A, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783. 2024 Jul 31. · arXiv
  29. [35]Agarwal S, Ahmad L, Ai J, Altman S, Applebaum A, Arbus E, Arora RK, Bai Y, Baker B, Bao H, Barak B. gpt-oss-120b & gpt-oss-20b model card. arXiv preprint arXiv:2508.10925. 2025 Aug 8. · arXiv

Cited by

Sorted by citations

Authors
Wu, X., Garduno-Rapp, N. E., Rousseau, J. F., Thakkallapally, M., Zhang, H., Ji, Y., Visweswaran, S., Peng, Y., & Wang, Y.
Affiliations
  • University of Pittsburgh (US)
  • The University of Texas Southwestern Medical Center (US)
  • Weill Cornell Medicine (US)
Corresponding author
Wang, Yanshan
ORCID
Venue
arXiv
Publisher
arXiv
Published
2025
Volume
33
Issue
10
Pages
1910-1920
ISSN
1067-5027 (ISSN-L)
DOI
10.48550/arxiv.2512.04207
arXiv
2512.04207
PMC
PMC13419605
OpenAlex
W4417086514
Cited by
1
References
0
Citations by year
2026: 1
FWCI
1.97
Type
journal article
Language
en
Open access
Licence
Locations
  • Journal of the American Medical Informatics Association (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • PubMed Central (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • National Institutes of Health (UL1TR001857, U24TR004111, 1r01lm014588-01a1)
  • U.S. National Library of Medicine (R01LM014306)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
Keywords
secondary headache, clinical decision support, multi-agent LLM systems, large language models, primary care, explainable AI, prompt engineering, interpretability
MeSH terms
Large Language Models, Headache, Humans, Primary Health Care, Practice Guidelines as Topic, Decision Support Systems, Clinical

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review