CoChat
Sign Up
PreprintCited by 94Artificial Intelligence in LawLegal Education and Practice InnovationsArtificial Intelligence Applications

Chatlaw: A Multi-Agent Legal Assistant based on a Role-Aligned Mixture-of-Experts Architecture

Open access
Cui, Jiaxi, Ning, Munan, Li, Zongjian, Chen, Bohua, Yan, Yang, Li, Hao, Ling, Bin, Tian, Yonghong +1 more·2023·arXiv
DOI 10.48550/arxiv.2306.16092arXiv 2306.16092

Preprint — published as 10.1016/j.fmre.2026.03.026

Open this paper in CoChat — free account

Abstract

Artificial Intelligence (AI) holds great potential in legal services, yet Large Language Models (LLMs) face two major challenges: limited knowledge of the Chinese legal system and vulnerability to hallucinations. To address these issues, we present Chatlaw, a multi-agent legal assistant. Chatlaw's framework is designed to emulate the Standard Operating Procedures (SOP) of real law firms, where different roles (e.g., assistant, researcher, senior lawyer) collaborate on a case. To computationally mirror this collaborative structure, we developed a novel Role-Aligned Mixture-of-Experts (RA-MoE) architecture. In this system, the internal "experts" are specifically trained to align with the distinct tasks of each agent role (e.g., inquiry, analysis, drafting). These specialized agents (Legal Assistant, Researcher, etc.) then form the collaborative framework. When they interact with users, retrieve legal knowledge, analyze case details, or generate reliable consultations, the RA-MoE architecture intelligently routes their computations to the corresponding dedicated expert, ensuring each step is handled by the most qualified parameters. In evaluations, Chatlaw surpasses general-purpose AI models, including GPT-4, achieving a 7.73% improvement in accuracy on the LawBench benchmark and an 11-point higher score on the Unified Qualification Exam for Legal Professionals. Real-case studies and expert assessments further confirm its robustness. Chatlaw enhances the accessibility and reliability of legal services, advancing the provision of legal support to the public.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2306.16092

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • LLM-based agents can serve as aids in legal decision-making processes, facilitating more informed judgements.

    “Jurisprudence: LLM-based agents can serve as aids in legal decision-making processes, facilitating more informed judgements [112,113].”

    From A survey on large language model based autonomous agents (2024) · cites this paper as [112]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 23 · Cited by 94

Research works printed but not linked: 51

  1. [1][1] Aminabadi, R.Y., Rajbhandari, S., et al., 2022. Deepspeed-inference: Enabling efficient inference of transformer models at unprecedented scale. arXiv preprint arXiv:2207.03669 . - · arXiv
  2. [3][3] Auerbach, J.S., 1977. Unequal justice: Lawyers and social change in modern America. Oxford University Press, New York. -
  3. [4][4] Bai, J., Bai, S., Chu, Y., Cui, Z., Dang, K., Deng, X., Fan, Y., Ge, W., Han, Y., Huang, F., et al., 2023. Qwen technical report. arXiv preprint arXiv:2309.16609 . - · arXiv
  4. [6][6] Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al., 2020. Language models are few-shot learners, in: Advances in neural information processing systems, pp. 1877-1901. -
  5. [8][8] Chalkidis, I., Jana, A., Hartung, D., Bommarito, M., Androutsopoulos, I., Katz, D.M., Aletras, N., 2021. Lexglue: A benchmark dataset for legal language understanding in english. arXiv preprint arXiv:2110.00976 . - · arXiv
  6. [7][7] Cai, Z., Cao, M., Chen, H., Chen, K., Chen, K., Chen, X., Chen, X., Chen, Z., Chen, Z., Chu, P., et al., 2024. Internlm2 technical report. arXiv preprint arXiv:2403.17297 . - · arXiv
  7. [11][11] Dao, T., 2023. Flashattention-2: Faster attention with better parallelism and work partitioning. arXiv preprint arXiv:2307.08691 . - · arXiv
  8. [12][12] Dao, T., Fu, D.Y., Ermon, S., et al., 2022. Flashattention: Fast and memory-efficient exact attention with io-awareness. Advances in Neural Information Processing Systems . -
  9. [14][14] Dettmers, T., Pagnoni, A., Holtzman, A., Zettlemoyer, L., 2023. Qlora: Efficient finetuning of quantized llms. arXiv preprint arXiv:2305.14314 . - · arXiv
  10. [13][13] Deng, W., Pei, J., Kong, K., Chen, Z., Wei, F., Li, Y., Ren, Z., Chen, Z., Ren, P., 2023. Syllogistic reasoning for legal judgment analysis, in: Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 13997-14009. -
  11. [15][15] Dhuliawala, S., Komeili, M., Xu, J., Raileanu, R., Li, X., Celikyilmaz, A., Weston, J., 2023. Chain-of-verification reduces hallucination in large language models. arXiv preprint arXiv:2309.11495 . - · arXiv
  12. [16][16] Du, Z., Qian, Y., Liu, X., Ding, M., Qiu, J., Yang, Z., Tang, J., 2021. Glm: General language model pretraining with autoregressive blank infilling. arXiv preprint arXiv:2103.10360 . - · arXiv
  13. [20][20] Fei, Z., Shen, X., Zhu, D., Zhou, F., Han, Z., Huang, A., Zhang, S., Chen, K., Yin, Z., Shen, Z., et al., 2024. Lawbench: Benchmarking legal knowledge of large language models, in: Proceedings of the 2024 conference on empirical methods in natural language processing, pp. 7933-7962. -
  14. [19][19] Fedus, W., Zoph, B., Shazeer, N., 2022. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. Journal of Machine Learning Research 23, 1-39. -
  15. [21][21] Fei, Z., Shen, X., Zhu, D., Zhou, F., Han, Z., Zhang, S., Chen, K., Shen, Z., Ge, J., 2023. Lawbench: Benchmarking legal knowledge of large language models. arXiv preprint arXiv:2309.16289 . - · arXiv
  16. [23][23] Frantar, E., Ashkboos, S., Hoefler, T., Alistarh, D., 2023. Gptq: Accurate post-training quantization for generative pre-trained transformers, in: ICLR. -
  17. [24][24] Gan, B., Zhao, Y., Zhang, T., Huang, J., Li, Y., Teo, S.X., Zhang, C., Shi, W., 2025. Master: A multi-agent system with llm specialized mcts. arXiv preprint arXiv:2501.14304 . - · arXiv
  18. [25][25] Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Wang, H., Wang, H., 2023. Retrieval-augmented generation for large language models: A survey. arXiv preprint arXiv:2312.10997 2. - · arXiv
  19. [27][27] Heinz, J.P., 2005. Urban lawyers: The new social structure of the bar. University of Chicago Press. -
  20. [29][29] Hua, Y., Cao, H., Tao, Z., Li, B., Wu, Z., Liu, C., Xu, L., 2025. Input domain aware moe: Decoupling routing decisions from task optimization in mixture of experts. arXiv preprint arXiv:2510.16448 . - · arXiv
  21. [28][28] Hong, S., Zhuge, M., Chen, J., Zheng, X., Cheng, Y., Wang, J., Zhang, C., Wang, Z., Yau, S.K.S., Lin, Z., et al., 2023. Metagpt: Meta programming for a multi-agent collaborative framework, in: The Twelfth International Conference on Learning Representations. -
  22. [30][30] Huang, Q., Tao, M., An, Z., Zhang, C., Jiang, C., Chen, Z., Wu, Z., Feng, Y., 2023a. Lawyer llama technical report. ArXiv abs/2305.15062. - · arXiv
  23. [32][32] Ji, Z., Lee, N., Frieske, R., et al., 2023. Survey of hallucination in natural language generation. ACM Computing Surveys . -
  24. [31][31] Huang, Q., Tao, M., An, Z., Zhang, C., Jiang, C., Chen, Z., Wu, Z., Feng, Y., 2023b. Lawyer llama technical report. arXiv preprint arXiv:2305.15062 . - · arXiv
  25. [33][33] Jiang, A.Q., Sablayrolles, A., Roux, A., Mensch, A., Savary, B., Bamford, C., Chaplot, D.S., Casas, D.d.l., Hanna, E.B., Bressand, F., et al., 2024. Mixtral of experts. arXiv preprint arXiv:2401.04088 . - · arXiv
  26. [36][36] Krajewski, J., Ludziejewski, J., Adamczewski, K., Pióro, M., Krutul, M., Antoniak, S., Ciebiera, K., Król, K., Odrzygóźdź, T., Sankowski, P., et al., 2024. Scaling laws for fine-grained mixture of experts. arXiv preprint arXiv:2402.07871 . - · arXiv
  27. [39][39] Kwon, W., Li, Z., Zhuang, S., et al., 2023. Efficient memory management for large language model serving with pagedattention, in: Proceedings of the 29th Symposium on Operating Systems Principles. -
  28. [40][40] Lepikhin, D., Lee, H., Xu, Y., Chen, D., Firat, O., Huang, Y., Krikun, M., Shazeer, N., Chen, Z., 2020a. Gshard: Scaling giant models with conditional computation and automatic sharding. arXiv preprint arXiv:2006.16668 . - · arXiv
  29. [42][42] Li, D., Ma, Y., Wang, N., Ye, Z., Cheng, Z., Tang, Y., Zhang, Y., Duan, L., Zuo, J., Yang, C., et al., 2024. Mixlora: Enhancing large language models fine-tuning with lora-based mixture of experts. arXiv preprint arXiv:2404.15159 . - · arXiv
  30. [41][41] Lepikhin, D., Loh, H.J., Fedus, W., Chen, Y., et al., 2020b. Gshard: Scaling giant models with conditional computation and automatic sharding, in: ICLR. -
  31. [43][43] Lin, S., Hilton, J., Evans, O., 2021. Truthfulqa: Measuring how models mimic human falsehoods. arXiv preprint arXiv:2109.07958 . - · arXiv
  32. [45][45] Liu, A., Feng, B., Xue, B., Wang, B., Wu, B., Lu, C., Zhao, C., Deng, C., Zhang, C., Ruan, C., et al., 2024. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437 . - · arXiv
  33. [44][44] Lin, S., Hilton, J., Evans, O., 2022. Teaching models to express their uncertainty in words. arXiv preprint arXiv:2205.14334 . - · arXiv
  34. [46][46] Lv, L., Lin, Z., Li, H., Liu, Y., Cui, J., Chen, C.Y.C., Yuan, L., Tian, Y., 2024. Prollama: A protein large language model for multi-task protein language processing. arXiv preprint arXiv:2402.16445 . - · arXiv
  35. [48][48] McCoy, R.T., Yao, S., Friedman, D., Hardy, M., Griffiths, T.L., 2023. Embers of autoregression: Understanding large language models through the problem they are trained to solve. arXiv preprint arXiv:2309.13638 . - · arXiv
  36. [47][47] Maynez, J., Narayan, S., Bohnet, B., McDonald, R., 2020. On faithfulness and factuality in abstractive summarization. arXiv preprint arXiv:2005.00661 . - · arXiv
  37. [49][49] Mik, E., 2024. Caveat lector: Large language models in legal practice. arXiv preprint arXiv:2403.09163 . - · arXiv
  38. [50][50] Nguyen, H.T., 2023. A brief report on lawgpt 1.0: A virtual legal assistant based on gpt-3. arXiv preprint arXiv:2302.05729 . - · arXiv
  39. [53][53] OpenAI, 2023. Gpt-4 technical report. arXiv:2303.08774 . - · arXiv
  40. [55][55] Pei, C., Wang, Z., Liu, F., Li, Z., Liu, Y., He, X., Kang, R., Zhang, T., Chen, J., Li, J., et al., 2025. Flow-of-action: Sop enhanced llmbased multi-agent system for root cause analysis, in: Companion Proceedings of the ACM on Web Conference 2025, pp. 422-431. -
  41. [58][58] Shazeer, N., Mirhoseini, A., Maziarz, K., Davis, A., Le, Q., Hinton, G., Dean, J., 2017. Outrageously large neural networks: The sparselygated mixture-of-experts layer. arXiv preprint arXiv:1701.06538 . - · arXiv
  42. [61][61] Sun, X., Chen, Y., Huang, Y., Xie, R., Zhu, J., Zhang, K., Li, S., Yang, Z., Han, J., Shu, X., et al., 2024. Hunyuan-large: An open-source moe model with 52 billion activated parameters by tencent. arXiv preprint arXiv:2411.02265 . - · arXiv
  43. [67][67] Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al., 2023a. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 . - · arXiv
  44. [68][68] Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al., 2023b. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 . - · arXiv
  45. [70][70] Wu,Q., Bansal, G., Zhang, J., Wu, Y., Li, B., Zhu, E., Jiang, L., Zhang, X., Zhang, S., Liu, J., et al., 2024. Autogen: Enabling next-gen llm applications via multi-agent conversations, in: First Conference on Language Modeling. -
  46. [73][73] Yang, A., Xiao, B., Wang, B., Zhang, B., Bian, C., Yin, C., Lv, C., Pan, D., Wang, D., Yan, D., et al., 2023. Baichuan 2: Open large-scale language models. arXiv preprint arXiv:2309.10305 . - · arXiv
  47. [72][72] Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Gao, C., Huang, C., Lv, C., et al., 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388 . - · arXiv
  48. [74][74] Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K.R., Cao, Y., 2022. React: Synergizing reasoning and acting in language models, in: The eleventh international conference on learning representations. -
  49. [76][76] Zhou, Y., Lei, T., Liu, H., Du, N., et al., 2022. Mixture-of-experts with expert choice routing. Advances in Neural Information Processing Systems . -
  50. [75][75] Zeng, A., Liu, X., Du, Z., Wang, Z., Lai, H., Ding, M., Yang, Z., Xu, Y., Zheng, W., Xia, X., et al., 2022. Glm-130b: An open bilingual pre-trained model. arXiv preprint arXiv:2210.02414 . - · arXiv
  51. [77][77] Zhou, Z., Shi, J.X., Song, P.X., Yang, X.W., Jin, Y.X., Guo, L.Z., Li, Y.F., 2024. Lawgpt: A chinese legal knowledge-enhanced large language model. arXiv preprint arXiv:2406.04614 . Table 6 Inference Performance and Throughput of Chatlaw-MoE under Various Configurations · arXiv

Authors
Cui, J., Ning, M., Li, Z., Chen, B., Yan, Y., Li, H., Ling, B., Tian, Y., & Yuan, L.
Affiliations
Peking University (CN)
Corresponding author
Yuan, Li
ORCID
Venue
arXiv
Publisher
arXiv
Published
2023
ISSN
2096-9457 (ISSN-L)
DOI
10.48550/arxiv.2306.16092
arXiv
2306.16092
OpenAlex
W4382618722
Cited by
94
References
23
Citations by year
2026: 9 · 2025: 27 · 2024: 50 · 2023: 8
FWCI
177.69
Type
journal article
Language
en
Open access
Licence
Locations
  • Fundamental Research (publishedVersion, cc-by-nc-nd)
  • arXiv (Cornell University) (submittedVersion, cc-by-nc-nd)
  • arXiv (Cornell University)
Funders
  • National Natural Science Foundation of China (2023ZT10\u202F\u00D7\u202F075, 62425101, 62332002)
  • Shenzhen Science and Technology Innovation Program (KQTD20240729102051063)
Topics
  • Social Sciences › Political Science and International Relations › Artificial Intelligence in Law
  • Social Sciences › Law › Legal Education and Practice Innovations
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
Keywords
legal assistants, multi-agent systems, mixture of experts, role alignment, Chinese legal system, hallucination mitigation, large language models, legal advice

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review