Cited by 36Multi-Agent Systems and NegotiationNatural Language Processing TechniquesSpeech and dialogue systems

MasRouter: Learning to Route LLMs for Multi-Agent Systems

Open access
Yue, Yanwei, Zhang, Guibin, Liu, Boyang, Wan, Guancheng, Wang, Kun, Cheng, Dawei, Qi, Yiyan·2025·Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

Abstract

Yanwei Yue, Guibin Zhang, Boyang Liu, Guancheng Wan, Kun Wang, Dawei Cheng, Yiyan Qi. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.18653/v1/2025.acl-long.757

Cites

0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

The publisher hasn’t deposited this paper’s reference list.

Research works printed but not linked: 57

  1. Maryam Akhavan Aghdam, Hongpeng Jin, and Yanzhao Wu. 2024. Da-moe: Towards dynamic expert al- location for mixture-of-experts models. Preprint, arXiv:2409.06669. · arXiv
  2. Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, and Charles Sutton. 2021. Program synthesis with large language models. Preprint, arXiv:2108.07732. · arXiv
  3. Simone Barandoni, Filippo Chiarello, Lorenzo Cascone, Emiliano Marrale, and Salvatore Puccio. 2024. Au- tomating customer needs analysis: A comparative study of large language models in the travel industry. Preprint, arXiv:2404.17975. · arXiv
  4. Chi-Min Chan, Weize Chen, Yusheng Su, Jianxuan Yu, Wei Xue, Shanghang Zhang, Jie Fu, and Zhiyuan Liu. 2023. ChatEval: Towards Better LLM-based Evalua- tors through Multi-Agent Debate. arXiv e-prints.
  5. Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, Wei Ye, Yue Zhang, Yi Chang, Philip S. Yu, Qiang Yang, and Xing Xie. 2023. A survey on evaluation of large language mod- els. Preprint, arXiv:2307.03109. · arXiv
  6. Lingjiao Chen, Matei Zaharia, and James Zou. 2023a. Frugalgpt: How to use large language models while reducing cost and improving performance. Preprint, arXiv:2305.05176. · arXiv
  7. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understand- ing. Preprint, arXiv:1810.04805. · arXiv
  8. Dujian Ding, Ankur Mallick, Chi Wang, Robert Sim, Subhabrata Mukherjee, Victor Ruhle, Laks V. S. Lak- shmanan, and Ahmed Hassan Awadallah. 2024. Hy- brid llm: Cost-efficient and quality-aware query rout- ing. Preprint, arXiv:2404.14618. · arXiv
  9. Yilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum, and Igor Mordatch. 2023. Improving factuality and reasoning in language models through multiagent debate. CoRR, abs/2305.14325. · arXiv
  10. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783. · arXiv
  11. Tao Feng, Yanzhen Shen, and Jiaxuan You. 2024. Graphrouter: A graph-based router for llm selections. Preprint, arXiv:2410.03834. · arXiv
  12. Yao Fu, Hao Peng, Ashish Sabharwal, Peter Clark, and Tushar Khot. 2022. Complexity-based prompting for multi-step reasoning. In The Eleventh International Conference on Learning Representations.
  13. Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Y. Wu, Y. K. Li, Fuli Luo, Yingfei Xiong, and Wen- feng Liang. 2024. Deepseek-coder: When the large language model meets programming -the rise of code intelligence. Preprint, arXiv:2401.14196. · arXiv
  14. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Stein- hardt. 2021a. Measuring massive multitask language understanding. Proceedings of the International Conference on Learning Representations (ICLR).
  15. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021b. Measuring mathematical problem solving with the math dataset. NeurIPS.
  16. Qitian Jason Hu, Jacob Bieker, Xiuyu Li, Nan Jiang, Benjamin Keigwin, Gaurav Ranganath, Kurt Keutzer, and Shriyash Kaustubh Upadhyay. 2024a. Router- bench: A benchmark for multi-llm routing system. Preprint, arXiv:2403.12031. · arXiv
  17. Shengran Hu, Cong Lu, and Jeff Clune. 2024b. Auto- mated Design of Agentic Systems. arXiv preprint. ArXiv:2408.08435. · arXiv
  18. Yoichi Ishibashi and Yoshimasa Nishimura. 2024. Self- organized agents: A llm multi-agent framework to- ward ultra large-scale code generation and optimiza- tion. arXiv preprint arXiv:2404.02183. · arXiv
  19. Can Jin, Hongwu Peng, Qixin Zhang, Yujin Tang, Dimitris N Metaxas, and Tong Che. 2025. Two heads are better than one: Test-time scaling of multi-agent collaborative reasoning. arXiv preprint arXiv:2504.09772. · arXiv
  20. Pierre Lepagnol, Thomas Gerald, Sahar Ghannay, Christophe Servan, and Sophie Rosset. 2024. Small language models are good too: An empirical study of zero-shot classification. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 14923-14936, Torino, Italia. ELRA and ICCL.
  21. Zijun Liu, Yanzhe Zhang, Peng Li, Yang Liu, and Diyi Yang. 2023. Dynamic llm-agent network: An llm- agent collaboration framework with agent team opti- mization. CoRR, abs/2310.02170. · arXiv
  22. Shervin Minaee, Tomas Mikolov, Narjes Nikzad, Meysam Chenaghlu, Richard Socher, Xavier Am- atriain, and Jianfeng Gao. 2024. Large language models: A survey. Preprint, arXiv:2402.06196. · arXiv
  23. Alireza Mohammadshahi, Arshad Rafiq Shaikh, and Majid Yazdani. 2024. Routoo: Learning to route to large language models effectively. Preprint, arXiv:2401.13979. · arXiv
  24. Isaac Ong, Amjad Almahairi, Vincent Wu, Wei-Lin Chi- ang, Tianhao Wu, Joseph E. Gonzalez, M Waleed Kadous, and Ion Stoica. 2024. Routellm: Learn- ing to route llms with preference data. Preprint, arXiv:2406.18665. · arXiv
  25. Chen Qian, Zihao Xie, Yifei Wang, Wei Liu, Yu- fan Dang, Zhuoyun Du, Weize Chen, Cheng Yang, Zhiyuan Liu, and Maosong Sun. 2024. Scaling large-language-model-based multi-agent collabora- tion. arXiv preprint arXiv:2406.07155. · arXiv
  26. Frederike Ramin, Christoph Matthies, and Ralf Teusner. 2020. More than code: Contributions in scrum software engineering teams. In Proceedings of the IEEE/ACM 42nd International Conference on Software Engineering Workshops, ICSE '20, page 137-140. ACM. · DOI
  27. N Reimers. 2019. Sentence-bert: Sentence embed- dings using siamese bert-networks. arXiv preprint arXiv:1908.10084. · arXiv
  28. Yu Shang, Yu Li, Keyu Zhao, Likai Ma, Jiahe Liu, Fengli Xu, and Yong Li. 2024. Agentsquare: Au- tomatic llm agent search in modular design space. Preprint, arXiv:2410.06153. · arXiv
  29. Weizhou Shen, Chenliang Li, Hongzhan Chen, Ming Yan, Xiaojun Quan, Hehong Chen, Ji Zhang, and Fei Huang. 2024. Small llms are weak tool learners: A multi-llm agent. arXiv preprint arXiv:2401.07324. · arXiv
  30. KV Aditya Srivatsa, Kaushal Kumar Maurya, and Eka- terina Kochmar. 2024. Harnessing the power of multiple minds: Lessons learned from llm routing. Preprint, arXiv:2405.00467. · arXiv
  31. Melanie Swan, Takashi Kido, Eric Roland, and Renato P. dos Santos. 2023. Math agents: Computational in- frastructure, mathematical embedding, and genomics. Preprint, arXiv:2307.02502. · arXiv
  32. Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. 2024. Gemini 1.5: Unlocking multimodal un- derstanding across millions of tokens of context. Preprint, arXiv:2403.05530. · arXiv
  33. Jen tse Huang, Eric John Li, Man Ho Lam, Tian Liang, Wenxuan Wang, Youliang Yuan, Wenxiang Jiao, Xing Wang, Zhaopeng Tu, and Michael R. Lyu. 2025a. How far are we on the decision-making of llms? evaluating llms' gaming ability in multi-agent environments. Preprint, arXiv:2403.11807. · arXiv
  34. Jen tse Huang, Jiaxu Zhou, Tailin Jin, Xuhui Zhou, Zixi Chen, Wenxuan Wang, Youliang Yuan, Michael R. Lyu, and Maarten Sap. 2025b. On the resilience of llm-based multi-agent collaboration with faulty agents. Preprint, arXiv:2408.00989. · arXiv
  35. Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Man- dlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. 2023. Voyager: An Open- Ended Embodied Agent with Large Language Mod- els. arXiv e-prints, arXiv:2305.16291. · arXiv
  36. Qian Wang, Zhenheng Tang, ZICHEN JIANG, Nuo Chen, Tianyu Wang, and Bingsheng He. Agenttaxo: Dissecting and benchmarking token distribution of llm multi-agent systems. In ICLR 2025 Workshop on Foundation Models in the Wild.
  37. Qian Wang, Tianyu Wang, Qinbin Li, Jingsheng Liang, and Bingsheng He. 2024a. Megaagent: A practical framework for autonomous cooperation in large-scale llm agent systems. arXiv preprint arXiv:2408.09955. · arXiv
  38. Qian Wang, Jiaying Wu, Zhenheng Tang, Bingqiao Luo, Nuo Chen, Wei Chen, and Bingsheng He. 2025a. What limits llm-based human simulation: Llms or our design? arXiv preprint arXiv:2501.08579. · arXiv
  39. Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, and Ming Zhou. 2020. Minilm: Deep self- attention distillation for task-agnostic compression of pre-trained transformers. Advances in Neural Information Processing Systems, 33:5776-5788.
  40. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023a. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations.
  41. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, Sharan Narang, Aakanksha Chowd- hery, and Denny Zhou. 2023b. Self-consistency im- proves chain of thought reasoning in language mod- els. In ICLR. OpenReview.net.
  42. Yutong Wang, Jiali Zeng, Xuebo Liu, Fandong Meng, Jie Zhou, and Min Zhang. 2024b. TasTe: Teach- ing large language models to translate through self- reflection. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6144- 6158, Bangkok, Thailand. Association for Computa- tional Linguistics. · DOI
  43. Zhenhailong Wang, Shaoguang Mao, Wenshan Wu, Tao Ge, Furu Wei, and Heng Ji. 2024c. Unleashing the emergent cognitive synergy in large language mod- els: A task-solving agent through multi-persona self- collaboration. In NAACL. Association for Computa- tional Linguistics.
  44. Zhexuan Wang, Yutong Wang, Xuebo Liu, Liang Ding, Miao Zhang, Jie Liu, and Min Zhang. 2025b. Agentdropout: Dynamic agent elimination for token- efficient and high-performance llm-based multi-agent collaboration. Preprint, arXiv:2503.18891. · arXiv
  45. Yibo Yan, Shen Wang, Jiahao Huo, Philip S Yu, Xum- ing Hu, and Qingsong Wen. 2025. Mathagent: Leveraging a mixture-of-math-agent framework for real-world multimodal mathematical error detection. arXiv preprint arXiv:2503.18132. · arXiv
  46. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. 2023. React: Synergizing reasoning and acting in language models. In The Eleventh International Conference on Learning Representations.
  47. Zhangyue Yin, Qiushi Sun, Cheng Chang, Qipeng Guo, Junqi Dai, Xuanjing Huang, and Xipeng Qiu. 2023. Exchange-of-thought: Enhancing large lan- guage model capabilities through cross-model com- munication. Preprint, arXiv:2312.01823. · arXiv
  48. Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James T. Kwok, Zhen- guo Li, Adrian Weller, and Weiyang Liu. 2024. Metamath: Bootstrap your own mathematical questions for large language models. Preprint, arXiv:2309.12284. · arXiv
  49. Guibin Zhang, Kaijie Chen, Guancheng Wan, Heng Chang, Hong Cheng, Kun Wang, Shuyue Hu, and Lei Bai. 2025. Evoflow: Evolving diverse agentic work- flows on the fly. arXiv preprint arXiv:2502.07373. · arXiv
  50. Guibin Zhang, Yanwei Yue, Zhixun Li, Sukwon Yun, Guancheng Wan, Kun Wang, Dawei Cheng, Jef- frey Xu Yu, and Tianlong Chen. 2024a. Cut the crap: An economical communication pipeline for llm-based multi-agent systems. Preprint, arXiv:2410.02506. · arXiv
  51. Guibin Zhang, Yanwei Yue, Zhixun Li, Sukwon Yun, Guancheng Wan, Kun Wang, Dawei Cheng, Jef- frey Xu Yu, and Tianlong Chen. 2024b. Cut the crap: An economical communication pipeline for llm-based multi-agent systems. arXiv preprint arXiv:2410.02506. · arXiv
  52. Jiayi Zhang, Jinyu Xiang, Zhaoyang Yu, Fengwei Teng, Xionghui Chen, Jiaqi Chen, Mingchen Zhuge, Xin Cheng, Sirui Hong, Jinlin Wang, Bingnan Zheng, Bang Liu, Yuyu Luo, and Chenglin Wu. 2024c. AFlow: Automating Agentic Workflow Generation. arXiv preprint. ArXiv:2410.10762. · arXiv
  53. Jintian Zhang, Xin Xu, and Shumin Deng. 2023. Ex- ploring collaboration mechanisms for llm agents: A social psychology view. arXiv preprint arXiv:2310.02124. · arXiv
  54. Qinlin Zhao, Jindong Wang, Yixuan Zhang, Yiqiao Jin, Kaijie Zhu, Hao Chen, and Xing Xie. 2023. Com- peteai: Understanding the competition behaviors in large language model-based agents. arXiv preprint arXiv:2310.17512. · arXiv
  55. Xuhui Zhou, Zhe Su, Sophie Feng, Jiaxu Zhou, Jen tse Huang, Hsien-Te Kao, Spencer Lynch, Svitlana Volkova, Tongshuang Sherry Wu, Anita Woolley, Hao Zhu, and Maarten Sap. 2025. Sotopia-s4: a user- friendly system for flexible, customizable, and large- scale social simulation. Preprint, arXiv:2504.16122. · arXiv
  56. Mingchen Zhuge, Wenyi Wang, Louis Kirsch, Francesco Faccio, Dmitrii Khizbullin, and Jürgen Schmidhuber. 2024. Gptswarm: Language agents as optimizable graphs. In Forty-first International Conference on Machine Learning.
  57. Christian Zingg, Alexander von Gernler, Carsten Arzig, Frank Schweitzer, and Christoph Gote. 2023. De- tecting and optimising team interactions in software development. Preprint, arXiv:2302.14609. · arXiv

Cited by

Sorted by citations

Authors
Yue, Y., Zhang, G., Liu, B., Wan, G., Wang, K., Cheng, D., & Qi, Y.
Affiliations
Tongji University (CN)
Venue
Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Published
2025
Pages
15549-15572
ISSN
0736-587X (ISSN-L)
OpenAlex
W4412945204
Cited by
14
References
0
Citations by year
2026: 12 · 2025: 2
FWCI
13.35
Type
conference paper
Language
en
Open access
Licence
cc-by
Locations
Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Multi-Agent Systems and Negotiation
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
Keywords
large language models, multi-agent systems, routing, agent routing

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review