CoChat
Sign Up
PreprintCited by 2AI-based Problem Solving and PlanningReinforcement Learning in RoboticsPsychiatry, Mental Health, Neuroscience

Pangu-Agent: A Fine-Tunable Generalist Agent with Structured Reasoning

Open access
Christianos, Filippos, Papoudakis, Georgios, Zimmer, Matthieu, Coste, Thomas, Wu, Zhihao, Chen, Jingxuan, Khandelwal, Khyati, Doran, James +8 more·2023·arXiv
DOI 10.48550/arxiv.2312.14878arXiv 2312.14878
Open this paper in CoChat — free accountOpen PDF

Abstract

A key method for creating Artificial Intelligence (AI) agents is Reinforcement Learning (RL). However, constructing a standalone RL policy that maps perception to action directly encounters severe problems, chief among them being its lack of generality across multiple tasks and the need for a large amount of training data. The leading cause is that it cannot effectively integrate prior information into the perception-action cycle when devising the policy. Large language models (LLMs) emerged as a fundamental way to incorporate cross-domain knowledge into AI agents but lack crucial learning and adaptation toward specific decision problems. This paper presents a general framework model for integrating and learning structured reasoning into AI agents' policies. Our methodology is motivated by the modularity found in the human brain. The framework utilises the construction of intrinsic and extrinsic functions to add previous understandings of reasoning structures. It also provides the adaptive ability to learn models inside every module or function, consistent with the modular structure of cognitive processes. We describe the framework in-depth and compare it with other AI pipelines and existing frameworks. The paper explores practical applications, covering experiments that show the effectiveness of our method. Our results indicate that AI agents perform and adapt far better when organised reasoning and prior knowledge are embedded. This opens the door to more resilient and general AI agent systems.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2312.14878

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 72

  1. J. S. Albus, "Outline for a theory of intelligence," IEEE transactions on systems, man, and cybernetics, vol. 21, no. 3, pp. 473-509, 1991.
  2. D. Silver, T. Hubert, J. Schrittwieser, I. Antonoglou, M. Lai, A. Guez, M. Lanctot, L. Sifre, D. Kumaran, T. Graepel et al., "A general reinforcement learning algorithm that masters chess, shogi, and go through self-play," Science, vol. 362, no. 6419, pp. 1140-1144, 2018.
  3. D. J. Mankowitz, A. Michi, A. Zhernov, M. Gelmi, M. Selvi, C. Paduraru, E. Leurent, S. Iqbal, J.-B. Lespiau, A. Ahern et al., "Faster sorting algorithms discovered using deep reinforcement learning," Nature, vol. 618, no. 7964, pp. 257-263, 2023.
  4. A. Fawzi, M. Balog, A. Huang, T. Hubert, B. Romera-Paredes, M. Barekatain, A. Novikov, F. J. R Ruiz, J. Schrittwieser, G. Swirszcz et al., "Discovering faster matrix multiplication algorithms with reinforcement learning," Nature, vol. 610, no. 7930, pp. 47-53, 2022.
  5. E. Kaufmann, L. Bauersfeld, A. Loquercio, M. Müller, V. Koltun, and D. Scaramuzza, "Champion-level drone racing using deep reinforcement learning," Nature, vol. 620, no. 7976, pp. 982-987, 2023.
  6. J. Degrave, F. Felici, J. Buchli, M. Neunert, B. Tracey, F. Carpanese, T. Ewalds, R. Hafner, A. Abdolmaleki, D. de Las Casas et al., "Magnetic control of tokamak plasmas through deep reinforcement learning," Nature, vol. 602, no. 7897, pp. 414-419, 2022.
  7. K. Rakelly, A. Zhou, D. Quillen, C. Finn, and S. Levine, "Efficient off-policy meta-reinforcement learning via probabilistic context variables," in International Conference on Machine Learning, 2019. [Online]. Available: https://api.semanticscholar.org/CorpusID:84187276
  8. A. Gupta, R. Mendonca, Y. Liu, P. Abbeel, and S. Levine, "Meta-reinforcement learning of structured exploration strategies," in Neural Information Processing Systems, 2018. [Online]. Available: https://api.semanticscholar.org/CorpusID:3418899
  9. J. X. Wang, Z. Kurth-Nelson, D. Kumaran, D. Tirumala, H. Soyer, J. Z. Leibo, D. Hassabis, and M. M. Botvinick, "Prefrontal cortex as a meta-reinforcement learning system," Nature Neuroscience, vol. 21, pp. 860 -868, 2018. [Online]. Available: https://api.semanticscholar.org/CorpusID:44137923
  10. I. Clavera, A. Nagabandi, S. Liu, R. S. Fearing, P. Abbeel, S. Levine, and C. Finn, "Learning to adapt in dynamic, real-world environments through meta-reinforcement learning," in International Conference on Learning Representations, 2019. [Online]. Available: https://openreview.net/forum?id=HyztsoC5Y7
  11. A. G. Barto, "Intrinsic motivation and reinforcement learning," in Intrinsically Motivated Learning in Natural and Artificial Systems, 2013. [Online]. Available: https://api.semanticscholar.org/CorpusID: 2326055
  12. H. Tang, R. Houthooft, D. Foote, A. Stooke, O. Xi Chen, Y. Duan, J. Schulman, F. DeTurck, and P. Abbeel, "#exploration: A study of count-based exploration for deep reinforcement learning," in Advances in Neural Information Processing Systems, I.
  13. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, Eds., vol. 30. Curran Associates, Inc., 2017. [Online]. Available: https: //proceedings.neurips.cc/paper_files/paper/2017/file/3a20f62a0af1aa152670bab3c602feed-Paper.pdf
  14. Y. Burda, H. Edwards, A. Storkey, and O. Klimov, "Exploration by random network distillation," in 7th International Conference on Learning Representations (ICLR 2019), May 2019, pp. 1-17, seventh International Conference on Learning Representations, ICLR 2019 ; Conference date: 06-05-2019 Through 09-05-2019. [Online]. Available: https://iclr.cc/
  15. M. Jaderberg, V. Mnih, W. M. Czarnecki, T. Schaul, J. Z. Leibo, D. Silver, and K. Kavukcuoglu, "Reinforcement learning with unsupervised auxiliary tasks," in International Conference on Learning Representations, 2017. [Online]. Available: https://openreview.net/forum?id=SJ6yPD5xg
  16. E. Shelhamer, P. Mahmoudieh, M. Argus, and T. Darrell, "Loss is its own reward: Self- supervision for reinforcement learning," ArXiv, vol. abs/1612.07307, 2016. [Online]. Available: https://api.semanticscholar.org/CorpusID:16561904 · arXiv
  17. S. Li, R. Wang, M. Tang, and C. Zhang, "Hierarchical reinforcement learning with advantage-based auxiliary rewards," Advances in Neural Information Processing Systems, vol. 32, 2019.
  18. A. Ng, D. Harada, and S. J. Russell, "Policy invariance under reward transformations: Theory and application to reward shaping," in International Conference on Machine Learning, 1999. [Online]. Available: https://api.semanticscholar.org/CorpusID:5730166
  19. S. Devlin and D. Kudenko, "Dynamic potential-based reward shaping," in Adaptive Agents and Multi-Agent Systems, 2012. [Online]. Available: https://api.semanticscholar.org/CorpusID:17251664
  20. P. Goyal, S. Niekum, and R. J. Mooney, "Using natural language for reward shaping in reinforcement learning," in International Joint Conference on Artificial Intelligence, 2019. [Online]. Available: https://api.semanticscholar.org/CorpusID:70350059
  21. E. Brochu, V. M. Cora, and N. de Freitas, "A tutorial on bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning," ArXiv, vol. abs/1012.2599, 2010. [Online]. Available: https://api.semanticscholar.org/CorpusID:1640103 · arXiv
  22. P. Poupart, N. Vlassis, J. Hoey, and K. Regan, "An analytic solution to discrete bayesian reinforcement learning," in Proceedings of the 23rd international conference on Machine learning, 2006, pp. 697-704.
  23. M. Ghavamzadeh, S. Mannor, J. Pineau, A. Tamar et al., "Bayesian reinforcement learning: A survey," Foundations and Trends® in Machine Learning, vol. 8, no. 5-6, pp. 359-483, 2015.
  24. T. D. Kulkarni, K. Narasimhan, A. Saeedi, and J. Tenenbaum, "Hierarchical deep reinforcement learning: Integrating temporal abstraction and intrinsic motivation," in Advances in Neural Information Processing Systems, D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett, Eds., vol. 29. Curran Associates, Inc., 2016. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2016/file/ f442d33fa06832082290ad8544a8da27-Paper.pdf
  25. A. G. Barto and S. Mahadevan, "Recent advances in hierarchical reinforcement learning," Discrete event dynamic systems, vol. 13, no. 1-2, pp. 41-77, 2003.
  26. O. Nachum, S. S. Gu, H. Lee, and S. Levine, "Data-efficient hierarchical reinforcement learning," Advances in neural information processing systems, vol. 31, 2018.
  27. G. Andersen, P. Vrancx, and H. Bou-Ammar, "Learning high-level representations from demonstrations," CoRR, vol. abs/1802.06604, 2018. [Online]. Available: http://arxiv.org/abs/1802.06604 · arXiv
  28. Q. Wu, G. Bansal, J. Zhang, Y. Wu, B. Li, E. Zhu, L. Jiang, X. Zhang, S. Zhang, J. Liu, A. H. Awadallah, R. W. White, D. Burger, and C. Wang, "AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation," arXiv e-prints, p. arXiv:2308.08155, Aug. 2023. · arXiv
  29. T. Schick, J. Dwivedi-Yu, R. Dessì, R. Raileanu, M. Lomeli, L. Zettlemoyer, N. Cancedda, and T. Scialom, "Toolformer: Language models can teach themselves to use tools," Advances in Neural Information Processing Systems, vol. 36, 2023.
  30. Y. Shen, K. Song, X. Tan, D. Li, W. Lu, and Y. Zhuang, "Hugginggpt: Solving ai tasks with chatgpt and its friends in huggingface," Advances in Neural Information Processing Systems, vol. 36, 2023.
  31. G. Li, H. A. A. K. Hammoud, H. Itani, D. Khizbullin, and B. Ghanem, "CAMEL: Communicative agents for "mind" exploration of large language model society," in Thirty-seventh Conference on Neural Information Processing Systems, 2023. [Online]. Available: https://openreview.net/forum?id=3IyL2XWDkG
  32. X. Wang, J. Wei, D. Schuurmans, Q. Le, E. Chi, S. Narang, A. Chowdhery, and D. Zhou, "Self-consistency improves chain of thought reasoning in language models," arXiv preprint arXiv:2203.11171, 2022. · arXiv
  33. S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao, "React: Synergizing reasoning and acting in language models," arXiv preprint arXiv:2210.03629, 2022. · arXiv
  34. D. Zhou, N. Schärli, L. Hou, J. Wei, N. Scales, X. Wang, D. Schuurmans, C. Cui, O. Bousquet, Q. V. Le, and E. H. Chi, "Least-to-most prompting enables complex reasoning in large language models," in The Eleventh International Conference on Learning Representations, 2023. [Online]. Available: https://openreview.net/forum?id=WZH7099tgfM
  35. J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V. Le, D. Zhou et al., "Chain-of-thought prompting elicits reasoning in large language models," Advances in Neural Information Processing Systems, vol. 35, pp. 24 824-24 837, 2022.
  36. M. Shridhar, X. Yuan, M. Côté, Y. Bisk, A. Trischler, and M. J. Hausknecht, "Alfworld: Aligning text and embodied environments for interactive learning," CoRR, vol. abs/2010.03768, 2020. [Online]. Available: https://arxiv.org/abs/2010.03768 · arXiv
  37. C.-C. Lin, A. Jaech, X. Li, M. R. Gormley, and J. Eisner, "Limitations of autoregressive models and their alternatives," arXiv preprint arXiv:2010.11939, 2020. · arXiv
  38. C. Wei, Y. Chen, and T. Ma, "Statistically meaningful approximation: a case study on approximating turing machines with transformers," Advances in Neural Information Processing Systems, vol. 35, pp. 12 071-12 083, 2022.
  39. S. Yao, D. Yu, J. Zhao, I. Shafran, T. L. Griffiths, Y. Cao, and K. Narasimhan, "Tree of Thoughts: Deliberate Problem Solving with Large Language Models," arXiv e-prints, p. arXiv:2305.10601, May 2023. · arXiv
  40. S. Hao, Y. Gu, H. Ma, J. J. Hong, Z. Wang, D. Z. Wang, and Z. Hu, "Reasoning with language model is planning with world model," arXiv preprint arXiv:2305.14992, 2023. · arXiv
  41. X. Feng, Z. Wan, M. Wen, Y. Wen, W. Zhang, and J. Wang, "Alphazero-like tree-search can guide large language model decoding and training," in NeurIPS 2023 Foundation Models for Decision Making Workshop, 2023. [Online]. Available: https://openreview.net/forum?id=PJfc4x2jXY
  42. Z. Yang, P. Qi, S. Zhang, Y. Bengio, W. Cohen, R. Salakhutdinov, and C. D. Manning, "HotpotQA: A dataset for diverse, explainable multi-hop question answering," in Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Brussels, Belgium: Association for Computational Linguistics, Oct.-Nov. 2018, pp. 2369-2380. [Online]. Available: https://aclanthology.org/D18-1259
  43. S. Yao, H. Chen, J. Yang, and K. Narasimhan, "Webshop: Towards scalable real-world web interaction with grounded language agents," in ArXiv, preprint.
  44. E. A. Hansen, D. S. Bernstein, and S. Zilberstein, "Dynamic programming for partially observable stochastic games," in AAAI, vol. 4, 2004, pp. 709-715.
  45. E. J. Hu, yelong shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, "LoRA: Low-rank adaptation of large language models," in International Conference on Learning Representations, 2022. [Online]. Available: https://openreview.net/forum?id=nZeVKeeFYf9
  46. J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, "Proximal policy optimization algorithms," CoRR, vol. abs/1707.06347, 2017. [Online]. Available: http://arxiv.org/abs/1707.06347 · arXiv
  47. W. Kwon, Z. Li, S. Zhuang, Y. Sheng, L. Zheng, C. H. Yu, J. Gonzalez, H. Zhang, and I. Stoica, "Efficient memory management for large language model serving with pagedattention," in Proceedings of the 29th Symposium on Operating Systems Principles, 2023, pp. 611-626.
  48. H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale, D. Bikel, L. Blecher, C. Canton-Ferrer, M. Chen, G. Cucurull, D. Esiobu, J. Fernandes, J. Fu, W. Fu, B. Fuller, C. Gao, V. Goswami, N. Goyal, A. Hartshorn, S. Hosseini, R. Hou, H. Inan, M. Kardas, V. Kerkez, M. Khabsa, I. Kloumann, A. Korenev, P. S. Koura, M. Lachaux, T. Lavril, J. Lee, D. Liskovich, Y. Lu, Y. Mao, X. Martinet, T. Mihaylov, P. Mishra, I. Molybog, Y. Nie, A. Poulton, J. Reizenstein, R. Rungta, K. Saladi, A. Schelten, R. Silva, E. M. Smith, R. Subramanian, X. E. Tan, B. Tang, R. Taylor, A. Williams, J. X. Kuan, P. Xu, Z. Yan, I. Zarov, Y. Zhang, A. Fan, M. Kambadur, S. Narang, A. Rodriguez, R. Stojnic, S. Edunov, and T. Scialom, "Llama 2: Open foundation and fine-tuned chat models," CoRR, vol. abs/2307.09288, 2023. [Online]. Available: https://doi.org/10.48550/arXiv.2307.09288 · DOI · Paper page
  49. G. Wang, S. Cheng, X. Zhan, X. Li, S. Song, and Y. Liu, "Openchat: Advancing open-source language models with mixed-quality data," CoRR, vol. abs/2309.11235, 2023. [Online]. Available: https://doi.org/10.48550/arXiv.2309.11235 · DOI
  50. A. Q. Jiang, A. Sablayrolles, A. Mensch, C. Bamford, D. S. Chaplot, D. de Las Casas, F. Bressand, G. Lengyel, G. Lample, L. Saulnier, L. R. Lavaud, M. Lachaux, P. Stock, T. L. Scao, T. Lavril, T. Wang, T. Lacroix, and W. E. Sayed, "Mistral 7b," CoRR, vol. abs/2310.06825, 2023. [Online]. Available: https://doi.org/10.48550/arXiv.2310.06825 · DOI · Paper page
  51. T. Kojima, S. S. Gu, M. Reid, Y. Matsuo, and Y. Iwasawa, "Large language models are zero-shot reasoners," in Advances in Neural Information Processing Systems, S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, Eds., vol. 35. Curran Associates, Inc., 2022, pp. 22 199-22 213. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2022/file/ 8bb0d291acd4acf06ef112099c16f326-Paper-Conference.pdf
  52. N. Shinn, B. Labash, and A. Gopinath, "Reflexion: an autonomous agent with dynamic memory and self-reflection," arXiv preprint arXiv:2303.11366, 2023. · arXiv
  53. Q. Dong, L. Li, D. Dai, C. Zheng, Z. Wu, B. Chang, X. Sun, J. Xu, and Z. Sui, "A survey for in-context learning," arXiv preprint arXiv:2301.00234, 2022. · arXiv
  54. T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al., "Language models are few-shot learners," Advances in neural information processing systems, vol. 33, pp. 1877-1901, 2020.
  55. M. Shridhar, J. Thomason, D. Gordon, Y. Bisk, W. Han, R. Mottaghi, L. Zettlemoyer, and D. Fox, "Alfred: A benchmark for interpreting grounded instructions for everyday tasks," in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Los Alamitos, CA, USA: IEEE Computer Society, jun 2020, pp. 10 737-10 746. [Online]. Available: https://doi.ieeecomputersociety.org/10.1109/CVPR42600.2020.01075 · DOI · Paper page
  56. M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. de Oliveira Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, A. Ray, R. Puri, G. Krueger, M. Petrov, H. Khlaaf, G. Sastry, P. Mishkin, B. Chan, S. Gray, N. Ryder, M. Pavlov, A. Power, L. Kaiser, M. Bavarian, C. Winter, P. Tillet, F. P. Such, D. Cummings, M. Plappert, F. Chantzis, E. Barnes, A. Herbert-Voss, W. H. Guss, A. Nichol, A. Paino, N. Tezak, J. Tang, I. Babuschkin, S. Balaji, S. Jain, W. Saunders, C. Hesse, A. N. Carr, J. Leike, J. Achiam, V. Misra, E. Morikawa, A. Radford, M. Knight, M. Brundage, M. Murati, K. Mayer, P. Welinder, B. McGrew, D. Amodei, S. McCandlish, I. Sutskever, and W. Zaremba, "Evaluating large language models trained on code," 2021.
  57. T. Carta, C. Romac, T. Wolf, S. Lamprier, O. Sigaud, and P.-Y. Oudeyer, "Grounding Large Language Models in Interactive Environments with Online Reinforcement Learning," vol. 202, no. 3676-3713.
  58. PMLR, 2023. [Online]. Available: https://hal.science/hal-03970122
  59. P. Brookins and J. M. DeBacker, "Playing games with gpt: What can we learn about a large language model from canonical strategic games?" Available at SSRN 4493398, 2023.
  60. S. V. Albrecht, F. Christianos, and L. Schäfer, Multi-Agent Reinforcement Learning: Foundations and Modern Approaches. MIT Press, 2024. [Online]. Available: https://www.marl-book.com
  61. E. Akata, L. Schulz, J. Coda-Forno, S. J. Oh, M. Bethge, and E. Schulz, "Playing repeated games with large language models," CoRR, vol. abs/2305.16867, 2023. [Online]. Available: https://doi.org/10.48550/arXiv.2305.16867 · DOI · Paper page
  62. J. Huang, X. Chen, S. Mishra, H. S. Zheng, A. W. Yu, X. Song, and D. Zhou, "Large language models cannot self-correct reasoning yet," arXiv preprint arXiv:2310.01798, 2023. · arXiv
  63. K. Stechly, M. Marquez, and S. Kambhampati, "Gpt-4 doesn't know it's wrong: An analysis of iterative prompting for reasoning problems," arXiv preprint arXiv:2310.12397, 2023. · arXiv
  64. J. Long, "Large Language Model Guided Tree-of-Thought," arXiv e-prints, p. arXiv:2305.08291, May 2023. · arXiv
  65. Y. Yao, Z. Li, and H. Zhao, "Beyond Chain-of-Thought, Effective Graph-of-Thought Reasoning in Large Language Models," arXiv e-prints, p. arXiv:2305.16582, May 2023. · arXiv
  66. M. Besta, N. Blach, A. Kubicek, R. Gerstenberger, L. Gianinazzi, J. Gajda, T. Lehmann, M. Podstawski, H. Niewiadomski, P. Nyczyk, and T. Hoefler, "Graph of Thoughts: Solving Elaborate Problems with Large Language Models," arXiv e-prints, p. arXiv:2308.09687, Aug. 2023. · arXiv
  67. A. Madaan, N. Tandon, P. Gupta, S. Hallinan, L. Gao, S. Wiegreffe, U. Alon, N. Dziri, S. Prabhumoye, Y. Yang et al., "Self-refine: Iterative refinement with self-feedback," arXiv preprint arXiv:2303.17651, 2023. · arXiv
  68. B. Xu, Z. Peng, B. Lei, S. Mukherjee, Y. Liu, and D. Xu, "Rewoo: Decoupling reasoning from observations for efficient augmented language models," arXiv preprint arXiv:2305.18323, 2023. · arXiv
  69. H. Liu, C. Sferrazza, and P. Abbeel, "Languages are rewards: Hindsight finetuning using human feedback," arXiv preprint arXiv:2302.02676, 2023. · arXiv
  70. T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Fun- towicz et al., "Huggingface's transformers: State-of-the-art natural language processing," arXiv preprint arXiv:1910.03771, 2019. · arXiv
  71. Z. Liu, W. Yao, J. Zhang, L. Xue, S. Heinecke, R. Murthy, Y. Feng, Z. Chen, J. C. Niebles, D. Arpit et al., "Bolaa: Benchmarking and orchestrating llm-augmented autonomous agents," arXiv preprint arXiv:2308.05960, 2023. · arXiv
  72. G. Chen, S. Dong, Y. Shu, G. Zhang, S. Jaward, K. Börje, J. Fu, and Y. Shi, "Autoagents: The automatic agents generation framework," arXiv preprint, 2023.

Cited by

Sorted by citations

Authors
Christianos, F., Papoudakis, G., Zimmer, M., Coste, T., Wu, Z., Chen, J., Khandelwal, K., Doran, J., Feng, X., Liu, J., Xiong, Z., Luo, Y., Hao, J., Shao, K., Bou-Ammar, H., & Wang, J.
Affiliations
  • Huawei Technologies (China) (CN)
  • University College London (GB)
  • University of Oxford (GB)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2023
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2312.14878
arXiv
2312.14878
OpenAlex
W4390215454
Cited by
2
References
0
Citations by year
2025: 2
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
  • Physical Sciences › Computer Science › Artificial Intelligence › Psychiatry, Mental Health, Neuroscience
Keywords
AI agents, structured reasoning, large language models, modular architecture, prior knowledge incorporation, perception-action cycle, fine-tuning, cognitive modularity, reinforcement learning policy

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review