CoChat
Sign Up
PreprintCited by 1Topic ModelingNatural Language Processing TechniquesReinforcement Learning in Robotics

Meta-Thinking in LLMs via Multi-Agent Reinforcement Learning: A Survey

Open access
Bilal, Ahsan, Mohsin, Muhammad Ahmed, Umer, Muhammad, Bangash, Muhammad Awais Khan, Jamshed, Muhammad Ali·2025·arXiv
DOI 10.48550/arxiv.2504.14520arXiv 2504.14520
Open this paper in CoChat — free account

Abstract

This survey explores the development of meta-thinking capabilities in Large Language Models (LLMs) from a Multi-Agent Reinforcement Learning (MARL) perspective. Meta-thinking self-reflection, assessment, and control of thinking processes is an important next step in enhancing LLM reliability, flexibility, and performance, particularly for complex or high-stakes tasks. The survey begins by analyzing current LLM limitations, such as hallucinations and the lack of internal self-assessment mechanisms. It then talks about newer methods, including RL from human feedback (RLHF), self-distillation, and chain-of-thought prompting, and each of their limitations. The crux of the survey is to talk about how multi-agent architectures, namely supervisor-agent hierarchies, agent debates, and theory of mind frameworks, can emulate human-like introspective behavior and enhance LLM robustness. By exploring reward mechanisms, self-play, and continuous learning methods in MARL, this survey gives a comprehensive roadmap to building introspective, adaptive, and trustworthy LLMs. Evaluation metrics, datasets, and future research avenues, including neuroscience-inspired architectures and hybrid symbolic reasoning, are also discussed.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2504.14520

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 121

  1. [1][1] M. Karwowski, J. Dul, J. Gralewski, E. Jauk, D. M. Jankowska, A. Gajda, M. H. Chruszczewski, and M. Benedek, 'Is creativity without intelligence possible? a necessary condition analysis,' Intelligence , vol. 57, pp. 105-117, 2016. -
  2. [2][2] T. Wu, J. Lan, W. Yuan, J. Jiao, J. Weston, and S. Sukhbaatar, 'Thinking llms: General instruction following with thought generation,' arXiv preprint arXiv:2410.10630 , 2024. - · arXiv
  3. [3][3] A. Bilal, D. Ebert, and B. Lin, 'Llms for explainable ai: A comprehensive survey,' arXiv preprint arXiv:2504.00125 , 2025. - · arXiv
  4. [4][4] J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V. Le, D. Zhou, et al. , 'Chain-of-thought prompting elicits reasoning in large language models,' Advances in neural information processing systems , vol. 35, pp. 24 824-24 837, 2022. -
  5. [5][5] Z. Sprague, F. Yin, J. D. Rodriguez, D. Jiang, M. Wadhwa, P. Singhal, X. Zhao, X. Ye, K. Mahowald, and G. Durrett, 'To cot or not to cot? chain-of-thought helps mainly on math and symbolic reasoning,' arXiv preprint arXiv:2409.12183 , 2024. - · arXiv
  6. [6][6] M. A. Mohsin, A. Bilal, S. Bhattacharya, and J. M. Cioffi, 'Retrieval augmented generation with multi-modal llm framework for wireless environments,' arXiv preprint arXiv:2503.07670 , 2025. - · arXiv
  7. [7][7] L. Huang, W. Yu, W. Ma, W. Zhong, Z. Feng, H. Wang, Q. Chen, W. Peng, X. Feng, B. Qin, et al. , 'A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions,' ACM Transactions on Information Systems , vol. 43, no. 2, pp. 1-55, 2025. -
  8. [8][8] S. K. Gallagher, J. Ratchford, T. Brooks, B. P. Brown, E. Heim, W. R. Nichols, S. Mcmillan, S. Rallapalli, C. J. Smith, N. VanHoudnos, et al. , 'Assessing llms for high stakes applications,' in Proceedings of the 46th International Conference on Software Engineering: Software Engineering in Practice , 2024, pp. 103-105. -
  9. [9][9] Y. Zhang, Y. Li, L. Cui, D. Cai, L. Liu, T. Fu, X. Huang, E. Zhao, Y. Zhang, Y. Chen, et al. , 'Siren's song in the ai ocean: A survey on hallucination in large language models,' arXiv preprint arXiv:2309.01219 , vol. 2, no. 5, 2023. - · arXiv
  10. [10][10] Z. Yin, 'A review of methods for alleviating hallucination issues in large language models,' Applied and Computational Engineering , vol. 76, pp. 258-266, 2024. -
  11. [11][11] P. Singhal, T. Goyal, J. Xu, and G. Durrett, 'A long way to go: Investigating length correlations in rlhf,' arXiv preprint arXiv:2310.03716 , 2023. - · arXiv
  12. [12][12] V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, et al. , 'Human-level control through deep reinforcement learning,' nature , vol. 518, no. 7540, pp. 529-533, 2015. -
  13. [13][13] T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra, 'Continuous control with deep reinforcement learning,' arXiv preprint arXiv:1509.02971 , 2015. - · arXiv
  14. [14][14] D. Guo, D. Yang, H. Zhang, J. Song, R. Zhang, R. Xu, Q. Zhu, S. Ma, P. Wang, X. Bi, et al. , 'Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning,' arXiv preprint arXiv:2501.12948 , 2025. - · arXiv
  15. [15][15] J. Zhang, B. Li, X. Cao, M. Zhang, and W. Zhao, 'Mixcl: Mixed contrastive learning for relation extraction,' in Pacific-Asia Conference on Knowledge Discovery and Data Mining . Springer, 2024, pp. 92103. -
  16. [16][16] N. Dziri, A. Madotto, O. Za¨ ıane, and A. J. Bose, 'Neural path hunter: Reducing hallucination in dialogue systems via path grounding,' arXiv preprint arXiv:2104.08455 , 2021. - · arXiv
  17. [17][17] X. Guan, Y. Liu, H. Lin, Y. Lu, B. He, X. Han, and L. Sun, 'Mitigating large language model hallucinations via autonomous knowledge graphbased retrofitting,' in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 38, no. 16, 2024, pp. 18 126-18 134. -
  18. [18][18] N. Varshney, W. Yao, H. Zhang, J. Chen, and D. Yu, 'A stitch in time saves nine: Detecting and mitigating hallucinations of llms by validating low-confidence generation,' arXiv preprint arXiv:2307.03987 , 2023. - · arXiv
  19. [19][19] L. Luo, Y.-F. Li, G. Haffari, and S. Pan, 'Reasoning on graphs: Faithful and interpretable large language model reasoning,' arXiv preprint arXiv:2310.01061 , 2023. - · arXiv
  20. [20][20] S. Casper, X. Davies, C. Shi, T. K. Gilbert, J. Scheurer, J. Rando, R. Freedman, T. Korbak, D. Lindner, P. Freire, et al. , 'Open problems and fundamental limitations of reinforcement learning from human feedback,' arXiv preprint arXiv:2307.15217 , 2023. - · arXiv
  21. [21][21] P. Manakul, A. Liusie, and M. J. Gales, 'Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models,' arXiv preprint arXiv:2303.08896 , 2023. - · arXiv
  22. [22][22] S. Dhuliawala, M. Komeili, J. Xu, R. Raileanu, X. Li, A. Celikyilmaz, and J. Weston, 'Chain-of-verification reduces hallucination in large language models,' arXiv preprint arXiv:2309.11495 , 2023. - · arXiv
  23. [23][23] X. Zhang, B. Peng, Y. Tian, J. Zhou, L. Jin, L. Song, H. Mi, and H. Meng, 'Self-alignment for factuality: Mitigating hallucinations in llms via self-evaluation,' arXiv preprint arXiv:2402.09267 , 2024. - · arXiv
  24. [24][24] N. M¨ undler, J. He, S. Jenko, and M. Vechev, 'Self-contradictory hallucinations of large language models: Evaluation, detection and mitigation,' arXiv preprint arXiv:2305.15852 , 2023. - · arXiv
  25. [25][25] S. Choi, T. Fang, Z. Wang, and Y. Song, 'Kcts: knowledge-constrained tree search decoding with token-level hallucination detection,' arXiv preprint arXiv:2310.09044 , 2023. - · arXiv
  26. [26][26] K. Li, O. Patel, F. Vi´ egas, H. Pfister, and M. Wattenberg, 'Inferencetime intervention: Eliciting truthful answers from a language model,' Advances in Neural Information Processing Systems , vol. 36, pp. 41 451-41 530, 2023. -
  27. [27][27] Y. Chuang, Y. Xie, H. Luo, Y. Kim, J. Glass, and P. D. He, 'Decoding by contrasting layers improves factuality in large language models. arxiv 2023,' arXiv preprint arXiv:2309.03883 . - · arXiv
  28. [28][28] A. Madaan, N. Tandon, P. Gupta, S. Hallinan, L. Gao, S. Wiegreffe, U. Alon, N. Dziri, S. Prabhumoye, Y. Yang, et al. , 'Self-refine: Iterative refinement with self-feedback,' Advances in Neural Information Processing Systems , vol. 36, pp. 46 534-46 594, 2023. -
  29. [29][29] K. Valmeekam, M. Marquez, and S. Kambhampati, 'Can large language models really improve by self-critiquing their own plans?' arXiv preprint arXiv:2310.08118 , 2023. - · arXiv
  30. [30][30] M. Griot, C. Hemptinne, J. Vanderdonckt, and D. Yuksel, 'Large language models lack essential metacognition for reliable medical reasoning,' Nature communications , vol. 16, no. 1, p. 642, 2025. -
  31. [31][31] J. Huang, X. Chen, S. Mishra, H. S. Zheng, A. W. Yu, X. Song, and D. Zhou, 'Large language models cannot self-correct reasoning yet,' arXiv preprint arXiv:2310.01798 , 2023. - · arXiv
  32. [32][32] C. Sun, S. Huang, and D. Pompili, 'Llm-based multi-agent reinforcement learning: Current and future directions,' arXiv preprint arXiv:2405.11106 , 2024. - · arXiv
  33. [33][33] Z. Liu, Y. Zhang, P. Li, Y. Liu, and D. Yang, 'Dynamic llm-agent network: An llm-agent collaboration framework with agent team optimization,' arXiv preprint arXiv:2310.02170 , 2023. - · arXiv
  34. [35][35] S. Hong, X. Zheng, J. Chen, Y. Cheng, J. Wang, C. Zhang, Z. Wang, S. K. S. Yau, Z. Lin, L. Zhou, et al. , 'Metagpt: Meta programming for multi-agent collaborative framework,' arXiv preprint arXiv:2308.00352 , vol. 3, no. 4, p. 6, 2023. - · arXiv
  35. [36][36] H. Zhang, W. Du, J. Shan, Q. Zhou, Y. Du, J. B. Tenenbaum, T. Shu, and C. Gan, 'Building cooperative embodied agents modularly with large language models,' arXiv preprint arXiv:2307.02485 , 2023. - · arXiv
  36. [37][37] S. S. Kannan, V. L. Venkatesh, and B.-C. Min, 'Smart-llm: Smart multi-agent robot task planning using large language models,' in 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2024, pp. 12 140-12 147. -
  37. [38][38] Z. Mandi, S. Jain, and S. Song, 'Roco: Dialectic multi-robot collaboration with large language models,' in 2024 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2024, pp. 286-299. -
  38. [39][39] B. Yu, H. Kasaei, and M. Cao, 'Co-navgpt: Multi-robot cooperative visual semantic navigation using large language models,' arXiv preprint arXiv:2310.07937 , 2023. - · arXiv
  39. [40][40] X. Guo, K. Huang, J. Liu, W. Fan, N. V´ elez, Q. Wu, H. Wang, T. L. Griffiths, and M. Wang, 'Embodied llm agents learn to cooperate in organized teams,' arXiv preprint arXiv:2403.12482 , 2024. - · arXiv
  40. [41][41] H. Chen, W. Ji, L. Xu, and S. Zhao, 'Multi-agent consensus seeking via large language models,' arXiv preprint arXiv:2310.20151 , 2023. - · arXiv
  41. [42][42] J. Chua, C. Wang, and L. Yao, 'Learning natural language constraints for safe reinforcement learning of language agents,' arXiv preprint arXiv:2504.03185 , 2025. - · arXiv
  42. [43][43] Z. Wan, Y. Li, Y. Song, H. Wang, L. Yang, M. Schmidt, J. Wang, W. Zhang, S. Hu, and Y. Wen, 'Rema: Learning to meta-think for llms with multi-agent reinforcement learning,' arXiv preprint arXiv:2503.09501 , 2025. - · arXiv
  43. [44][44] H. Li, Y. Q. Chong, S. Stepputtis, J. Campbell, D. Hughes, M. Lewis, and K. Sycara, 'Theory of mind for multi-agent collaboration via large language models,' arXiv preprint arXiv:2310.10701 , 2023. - · arXiv
  44. [45][45] L. Yang, Z. Yu, T. Zhang, S. Cao, M. Xu, W. Zhang, J. E. Gonzalez, and B. Cui, 'Buffer of thoughts: Thought-augmented reasoning with large language models,' Advances in Neural Information Processing Systems , vol. 37, pp. 113 519-113 544, 2024. -
  45. [46][46] H.-y. Lee, S.-W. Li, and N. T. Vu, 'Meta learning for natural language processing: A survey,' arXiv preprint arXiv:2205.01500 , 2022. - · arXiv
  46. [47][47] T. Wu, L. Luo, Y.-F. Li, S. Pan, T.-T. Vu, and G. Haffari, 'Continual learning for large language models: A survey,' arXiv preprint arXiv:2402.01364 , 2024. - · arXiv
  47. [48][48] H. Shi, Z. Xu, H. Wang, W. Qin, W. Wang, Y. Wang, Z. Wang, S. Ebrahimi, and H. Wang, 'Continual learning of large language models: A comprehensive survey,' arXiv preprint arXiv:2404.16789 , 2024. - · arXiv
  48. [49][49] M. Jovanovic and P. Voss, 'Towards incremental learning in large language models: A critical review,' arXiv preprint arXiv:2404.18311 , 2024. - · arXiv
  49. [50][50] H. Zheng, L. Shen, A. Tang, Y. Luo, H. Hu, B. Du, and D. Tao, 'Learn from model beyond fine-tuning: A survey,' arXiv preprint arXiv:2310.08184 , 2023. - · arXiv
  50. [51][51] P. Gao, A. Xie, S. Mao, W. Wu, Y. Xia, H. Mi, and F. Wei, 'Meta reasoning for large language models,' arXiv preprint arXiv:2406.11698 , 2024. - · arXiv
  51. [52][52] A. Plaat, A. Wong, S. Verberne, J. Broekens, N. van Stein, and T. Back, 'Reasoning with large language models, a survey,' arXiv preprint arXiv:2407.11511 , 2024. - · arXiv
  52. [53][53] E. Sayed, M. Maher, O. Sedeek, A. Eldamaty, A. Kamel, and R. El Shawi, 'Gizaml: A collaborative meta-learning based framework using llm for automated time-series forecasting.' in EDBT , 2024, pp. 830-833. -
  53. [54][54] J. Zheng, S. Qiu, C. Shi, and Q. Ma, 'Towards lifelong learning of large language models: A survey,' ACM Computing Surveys , vol. 57, no. 8, pp. 1-35, 2025. -
  54. [55][55] D. Bandyopadhyay, S. Bhattacharjee, and A. Ekbal, 'Thinking machines: A survey of llm based reasoning strategies,' arXiv preprint arXiv:2503.10814 , 2025. - · arXiv
  55. [56][56] Z. Zeng, Y. Liu, Y. Wan, J. Li, P. Chen, J. Dai, Y. Yao, R. Xu, Z. Qi, W. Zhao, et al. , 'Mr-ben: A meta-reasoning benchmark for evaluating system-2 thinking in llms,' arXiv preprint arXiv:2406.13975 , 2024. - · arXiv
  56. [57][57] N. Patel, M. Kulkarni, M. Parmar, A. Budhiraja, M. Nakamura, N. Varshney, and C. Baral, 'Multi-logieval: Towards evaluating multistep logical reasoning ability of large language models,' arXiv preprint arXiv:2406.17169 , 2024. - · arXiv
  57. [58][58] N. Ferguson, L. Guillou, A. Bundy, and K. Nuamah, 'Evaluating the meta-and object-level reasoning of large language models for question answering,' arXiv preprint arXiv:2502.10338 , 2025. - · arXiv
  58. [59][59] S. Sonkar, N. Liu, M. Le, and R. Baraniuk, 'Malalgoqa: Pedagogical evaluation of counterfactual reasoning in large language models and implications for ai in education,' in Findings of the Association for Computational Linguistics: EMNLP 2024 , 2024, pp. 15 554-15 567. -
  59. [60][60] R. Hada, V. Gumma, M. Ahmed, K. Bali, and S. Sitaram, 'Metal: Towards multilingual meta-evaluation,' arXiv preprint arXiv:2404.01667 , 2024. - · arXiv
  60. [61][61] S. Min, M. Lewis, L. Zettlemoyer, and H. Hajishirzi, 'Metaicl: Learning to learn in context,' arXiv preprint arXiv:2110.15943 , 2021. - · arXiv
  61. [62][62] Y. Esteitieh, S. Mandal, and G. Laliotis, 'Towards metacognitive clinical reasoning: Benchmarking md-pie against state-of-the-art llms in medical decision-making,' medRxiv , pp. 2025-01, 2025. -
  62. [63][63] Z. Yin, Q. Sun, Q. Guo, J. Wu, X. Qiu, and X. Huang, 'Do large language models know what they don't know?' arXiv preprint arXiv:2305.18153 , 2023. - · arXiv
  63. [64][64] H. Li, H. Nourkhiz Mahjoub, B. Chalaki, V. Tadiparthi, K. Lee, E. Moradi Pari, C. Lewis, and K. Sycara, 'Language grounded multiagent reinforcement learning with human-interpretable communication,' Advances in Neural Information Processing Systems , vol. 37, pp. 87 908-87 933, 2024. -
  64. [65][65] L. Cross, V. Xiang, A. Bhatia, D. L. Yamins, and N. Haber, 'Hypothetical minds: Scaffolding theory of mind for multi-agent tasks with large language models,' arXiv preprint arXiv:2407.07086 , 2024. - · arXiv
  65. [66][66] G. Hinton, O. Vinyals, and J. Dean, 'Distilling the knowledge in a neural network,' arXiv preprint arXiv:1503.02531 , 2015. - · arXiv
  66. [67][67] J. Gou, B. Yu, S. J. Maybank, and D. Tao, 'Knowledge distillation: A survey,' International Journal of Computer Vision , vol. 129, no. 6, pp. 1789-1819, 2021. -
  67. [68][68] D. Liu, A. Nassereldine, Z. Yang, C. Xu, Y. Hu, J. Li, U. Kumar, C. Lee, R. Qin, Y. Shi, et al. , 'Large language models have intrinsic self-correction ability,' arXiv preprint arXiv:2406.15673 , 2024. - · arXiv
  68. [69][69] H. Chen, J. He, K. Narasimhan, and D. Chen, 'Can rationalization improve robustness?' arXiv preprint arXiv:2204.11790 , 2022. - · arXiv
  69. [70][70] J. Andreas, 'Measuring compositionality in representation learning,' arXiv preprint arXiv:1902.07181 , 2019. - · arXiv
  70. [71][71] J. Z. Leibo, V. Zambaldi, M. Lanctot, J. Marecki, and T. Graepel, 'Multi-agent reinforcement learning in sequential social dilemmas,' arXiv preprint arXiv:1702.03037 , 2017. - · arXiv
  71. [72][72] R. Shu, N. Das, M. Yuan, M. Sunkara, and Y. Zhang, 'Towards effective genai multi-agent collaboration: Design and evaluation for enterprise applications,' arXiv preprint arXiv:2412.05449 , 2024. - · arXiv
  72. [73][73] G. Irving, P. Christiano, and D. Amodei, 'Ai safety via debate,' arXiv preprint arXiv:1805.00899 , 2018. - · arXiv
  73. [74][74] D. Ziegler, S. Nix, L. Chan, T. Bauman, P. Schmidt-Nielsen, T. Lin, A. Scherlis, N. Nabeshima, B. Weinstein-Raun, D. de Haas, et al. , 'Adversarial training for high-stakes reliability,' Advances in neural information processing systems , vol. 35, pp. 9274-9286, 2022. -
  74. [75][75] H. Dai, Y. Li, Z. Liu, L. Zhao, Z. Wu, S. Song, Y. Shen, D. Zhu, X. Li, S. Li, et al. , 'Ad-autogpt: an autonomous gpt for alzheimer's disease infodemiology,' arXiv preprint arXiv:2306.10095 , 2023. - · arXiv
  75. [76][76] P. F. Christiano, J. Leike, T. Brown, M. Martic, S. Legg, and D. Amodei, 'Deep reinforcement learning from human preferences,' Advances in neural information processing systems , vol. 30, 2017. -
  76. [77][77] L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, et al. , 'Training language models to follow instructions with human feedback,' Advances in neural information processing systems , vol. 35, pp. 27 730-27 744, 2022. -
  77. [78][78] A. Kumar, V. Zhuang, R. Agarwal, Y. Su, J. D. Co-Reyes, A. Singh, K. Baumli, S. Iqbal, C. Bishop, R. Roelofs, et al. , 'Training language models to self-correct via reinforcement learning,' arXiv preprint arXiv:2409.12917 , 2024. - · arXiv
  78. [79][79] J. Beck, R. Vuorio, E. Z. Liu, Z. Xiong, L. Zintgraf, C. Finn, and S. Whiteson, 'A survey of meta-reinforcement learning,' arXiv preprint arXiv:2301.08028 , 2023. - · arXiv
  79. [80][80] Y. J. Ma, W. Liang, G. Wang, D.-A. Huang, O. Bastani, D. Jayaraman, Y. Zhu, L. Fan, and A. Anandkumar, 'Eureka: Humanlevel reward design via coding large language models,' arXiv preprint arXiv:2310.12931 , 2023. - · arXiv
  80. [81][81] T. Wu, W. Yuan, O. Golovneva, J. Xu, Y. Tian, J. Jiao, J. Weston, and S. Sukhbaatar, 'Meta-rewarding language models: Self-improving alignment with llm-as-a-meta-judge,' arXiv preprint arXiv:2407.19594 , 2024. - · arXiv
  81. [82][82] N. Stiennon, L. Ouyang, J. Wu, D. Ziegler, R. Lowe, C. Voss, A. Radford, D. Amodei, and P. F. Christiano, 'Learning to summarize with human feedback,' Advances in neural information processing systems , vol. 33, pp. 3008-3021, 2020. -
  82. [83][83] L. Li, Z. Chen, G. Chen, Y. Zhang, Y. Su, E. Xing, and K. Zhang, 'Confidence matters: Revisiting intrinsic self-correction capabilities of large language models,' arXiv preprint arXiv:2402.12563 , 2024. - · arXiv
  83. [84][84] H. Sun, Y. Chai, S. Wang, Y. Sun, H. Wu, and H. Wang, 'Curiositydriven reinforcement learning from human feedback,' arXiv preprint arXiv:2501.11463 , 2025. - · arXiv
  84. [85][85] Q. Zheng, M. Henaff, A. Zhang, A. Grover, and B. Amos, 'Online intrinsic rewards for decision making agents from large language model feedback,' arXiv preprint arXiv:2410.23022 , 2024. - · arXiv
  85. [86][86] Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y. Li, Y. Wu, et al. , 'Deepseekmath: Pushing the limits of mathematical reasoning in open language models,' arXiv preprint arXiv:2402.03300 , 2024. - · arXiv
  86. [87][87] O. Vinyals, I. Babuschkin, W. M. Czarnecki, M. Mathieu, A. Dudzik, J. Chung, D. H. Choi, R. Powell, T. Ewalds, P. Georgiev, et al. , 'Grandmaster level in starcraft ii using multi-agent reinforcement learning,' nature , vol. 575, no. 7782, pp. 350-354, 2019. -
  87. [88][88] M. F. A. R. D. T. (FAIR)†, A. Bakhtin, N. Brown, E. Dinan, G. Farina, C. Flaherty, D. Fried, A. Goff, J. Gray, H. Hu, et al. , 'Human-level play in the game of diplomacy by combining language models with strategic reasoning,' Science , vol. 378, no. 6624, pp. 1067-1074, 2022. -
  88. [89][89] Z. Wu, R. Peng, S. Zheng, Q. Liu, X. Han, B. I. Kwon, M. Onizuka, S. Tang, and C. Xiao, 'Shall we team up: Exploring spontaneous cooperation of competing llm agents,' arXiv preprint arXiv:2402.12327 , 2024. - · arXiv
  89. [90][90] P. Cheng, T. Hu, H. Xu, Z. Zhang, Y. Dai, L. Han, X. Li, et al. , 'Selfplaying adversarial language game enhances llm reasoning,' Advances in Neural Information Processing Systems , vol. 37, pp. 126 515126 543, 2024. -
  90. [91][91] M. Nakamura, S. Mashetty, M. Parmar, N. Varshney, and C. Baral, 'Logicattack: Adversarial attacks for evaluating logical consistency of natural language inference,' in Findings of the Association for Computational Linguistics: EMNLP 2023 , 2023, pp. 13 322-13 334. -
  91. [92][92] C. Finn, P. Abbeel, and S. Levine, 'Model-agnostic meta-learning for fast adaptation of deep networks,' in International conference on machine learning . PMLR, 2017, pp. 1126-1135. -
  92. [93][93] W. Yin, 'Meta-learning for few-shot natural language processing: A survey,' arXiv preprint arXiv:2007.09604 , 2020. - · arXiv
  93. [94][94] G. Li, P. Wang, J. Liu, Y. Guo, K. Ji, Z. Shang, and Z. Xu, 'Meta incontext learning makes large language models better zero and few-shot relation extractors,' arXiv preprint arXiv:2404.17807 , 2024. - · arXiv
  94. [95][95] H. Raj, V. Gupta, D. Rosati, and S. Majumdar, 'Improving consistency in large language models through chain of guidance,' arXiv preprint arXiv:2502.15924 , 2025. - · arXiv
  95. [96][96] K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, et al. , 'Training verifiers to solve math word problems,' arXiv preprint arXiv:2110.14168 , 2021. - · arXiv
  96. [97][97] X. Wen, B. Li, T. Huang, and M. Chen, 'Red teaming language models for contradictory dialogues,' arXiv e-prints , pp. arXiv-2405, 2024. -
  97. [98][98] A. Jaech, A. Kalai, A. Lerer, A. Richardson, A. El-Kishky, A. Low, A. Helyar, A. Madry, A. Beutel, A. Carney, et al. , 'Openai o1 system card,' arXiv preprint arXiv:2412.16720 , 2024. - · arXiv
  98. [99][99] M. Geva, D. Khashabi, E. Segal, T. Khot, D. Roth, and J. Berant, 'Did aristotle use a laptop? a question answering benchmark with implicit reasoning strategies,' Transactions of the Association for Computational Linguistics , vol. 9, pp. 346-361, 2021. -
  99. [100][100] Y. Zhang, X. Yang, S. Feng, D. Wang, Y. Zhang, and K. Song, 'Can llms beat humans in debating? a dynamic multi-agent framework for competitive debate,' arXiv preprint arXiv:2408.04472 , 2024. - · arXiv
  100. [101][101] J. Z. Leibo, E. A. Due˜ nez-Guzman, A. Vezhnevets, J. P. Agapiou, P. Sunehag, R. Koster, J. Matyas, C. Beattie, I. Mordatch, and T. Graepel, 'Scalable evaluation of multi-agent reinforcement learning with melting pot,' in International conference on machine learning . PMLR, 2021, pp. 6187-6199. -
  101. [102][102] A. Srivastava, A. Rastogi, A. Rao, A. A. M. Shoeb, A. Abid, A. Fisch, A. R. Brown, A. Santoro, A. Gupta, A. Garriga-Alonso, et al. , 'Beyond the imitation game: Quantifying and extrapolating the capabilities of language models,' arXiv preprint arXiv:2206.04615 , 2022. - · arXiv
  102. [103][103] D. Zhang, Z. Hu, S. Zhoubian, Z. Du, K. Yang, Z. Wang, Y. Yue, Y. Dong, and J. Tang, 'Sciinstruct: A self-reflective instruction annotated dataset for training scientific language models,' Advances in Neural Information Processing Systems , vol. 37, pp. 1443-1473, 2024. -
  103. [104][104] R. Xu, X. Qi, Z. Qi, W. Xu, and Z. Guo, 'Debateqa: Evaluating question answering on debatable knowledge,' arXiv preprint arXiv:2408.01419 , 2024. - · arXiv
  104. [105][105] W. Du and S. Ding, 'A survey on multi-agent deep reinforcement learning: from the perspective of challenges and applications,' Artificial Intelligence Review , vol. 54, no. 5, pp. 3215-3238, 2021. -
  105. [106][106] G. Papoudakis, F. Christianos, A. Rahman, and S. V. Albrecht, 'Dealing with non-stationarity in multi-agent deep reinforcement learning,' arXiv preprint arXiv:1906.04737 , 2019. - · arXiv
  106. [107][107] M. Mohebbi Moghaddam, B. Boroomand, M. Jalali, A. Zareian, A. Daeijavad, M. H. Manshaei, and M. Krunz, 'Games of gans: Gametheoretical models for generative adversarial networks,' Artificial Intelligence Review , vol. 56, no. 9, pp. 9771-9807, 2023. -
  107. [108][108] C. Denison, M. MacDiarmid, F. Barez, D. Duvenaud, S. Kravec, S. Marks, N. Schiefer, R. Soklaski, A. Tamkin, J. Kaplan, et al. , 'Sycophancy to subterfuge: Investigating reward-tampering in large language models,' arXiv preprint arXiv:2406.10162 , 2024. - · arXiv
  108. [109][109] Y. Yang, J. Luo, Y. Wen, O. Slumbers, D. Graves, H. B. Ammar, J. Wang, and M. E. Taylor, 'Diverse auto-curriculum is critical for successful real-world multiagent learning systems,' arXiv preprint arXiv:2102.07659 , 2021. - · arXiv
  109. [110][110] H. F. Ahmad, 'Multi-agent systems: overview of a new paradigm for distributed systems,' in 7th IEEE International Symposium on High Assurance Systems Engineering, 2002. Proceedings. IEEE, 2002, pp. 101-107. -
  110. [111][111] M. Umer, M. A. Mohsin, A. Ul-Hasan, and F. Shafait, 'Pyramidtabnet: Transformer-based table recognition in image-based documents,' in Document Analysis and Recognition - ICDAR 2023 , G. A. Fink, R. Jain, K. Kise, and R. Zanibbi, Eds. Cham: Springer Nature Switzerland, 2023, pp. 420-437. -
  111. [112][112] A. Dorri, S. S. Kanhere, and R. Jurdak, 'Multi-agent systems: A survey,' Ieee Access , vol. 6, pp. 28 573-28 593, 2018. -
  112. [113][113] N. Krothapalli and A. V. Deshmukh, 'Distributed task allocation in multi-agent systems,' in Proceedings of the Institute of Industrial Engineers Annual Conference . Citeseer, 2002. -
  113. [114][114] L. Weidinger, J. Uesato, M. Rauh, C. Griffin, P.-S. Huang, J. Mellor, A. Glaese, M. Cheng, B. Balle, A. Kasirzadeh, et al. , 'Taxonomy of risks posed by language models,' in Proceedings of the 2022 ACM conference on fairness, accountability, and transparency , 2022, pp. 214-229. -
  114. [115][115] M. Mitchell, S. Wu, A. Zaldivar, P. Barnes, L. Vasserman, B. Hutchinson, E. Spitzer, I. D. Raji, and T. Gebru, 'Model cards for model reporting,' in Proceedings of the conference on fairness, accountability, and transparency , 2019, pp. 220-229. -
  115. [116][116] D. Hassabis, D. Kumaran, C. Summerfield, and M. Botvinick, 'Neuroscience-inspired artificial intelligence,' Neuron , vol. 95, no. 2, pp. 245-258, 2017. -
  116. [117][117] Y. Bengio, 'The consciousness prior,' arXiv preprint arXiv:1709.08568 , 2017. - · arXiv
  117. [118][118] K. Acharya, W. Raza, C. Dourado, A. Velasquez, and H. H. Song, 'Neurosymbolic reinforcement learning and planning: A survey,' IEEE Transactions on Artificial Intelligence , vol. 5, no. 5, pp. 1939-1953, 2023. -
  118. [119][119] B. P. Bhuyan, A. Ramdane-Cherif, R. Tomar, and T. Singh, 'Neurosymbolic artificial intelligence: a survey,' Neural Computing and Applications , vol. 36, no. 21, pp. 12 809-12 844, 2024. -
  119. [120][120] F. Yang, D. Lyu, B. Liu, and S. Gustafson, 'Peorl: Integrating symbolic planning and hierarchical reinforcement learning for robust decisionmaking,' arXiv preprint arXiv:1804.07779 , 2018. - · arXiv
  120. [121][121] S. I. Ross, F. Martinez, S. Houde, M. Muller, and J. D. Weisz, 'The programmer's assistant: Conversational interaction with a large language model for software development,' in Proceedings of the 28th International Conference on Intelligent User Interfaces , 2023, pp. 491514. -
  121. [122][122] A. Bilal, B. Lin, and M. Zaeifi, 'Onrl-rag: Real-time personalized mental health dialogue system,' arXiv preprint arXiv:2504.02894 , 2025. · arXiv

Cited by

Sorted by citations

Authors
Bilal, A., Mohsin, M. A., Umer, M., Bangash, M. A. K., & Jamshed, M. A.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2025
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2504.14520
arXiv
2504.14520
OpenAlex
W6967210593
Cited by
0
References
0
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
Keywords
large language models, multi-agent reinforcement learning, self-reflection, hallucination mitigation, self-assessment, self-distillation, chain-of-thought prompting, theory of mind, LLM robustness, reward processing, self-play, continuous learning, evaluation metrics

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review