Emergence of grounded compositional language in multi-agent populations
Open accessAbstract
By capturing statistical patterns in large corpora, machine learning has enabled significant advances in natural language processing, including in machine translation, question answering, and sentiment analysis. However, for agents to intelligently interact with humans, simply capturing the statistical patterns is insufficient. In this paper we investigate if, and how, grounded compositional language can emerge as a means to achieve goals in multi-agent populations. Towards this end, we propose a multi-agent learning environment and learning methods that bring about emergence of a basic compositional language. This language is represented as streams of abstract discrete symbols uttered by agents over time, but nonetheless has a coherent structure that possesses a defined vocabulary and syntax. We also observe emergence of non-verbal communication such as pointing and guiding when language communication is unavailable.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Studies in machine-learning emergent communication that focused on other reinforcement learning settings have not made direct contact with general, cognitively-motivated principles believed to underlie human language, such as IB.
“While some studies in the machine-learning emergent communication literature have focused on other reinforcement learning settings (Foerster et al., 2016;Mordatch & Abbeel, 2018, e.g.,), they have not made direct contact with general, cognitively-motivate principles that are believed to underlie human language, such as IB.”
From Towards Human-Like Emergent Communication via Utility, Informativeness, and Complexity (2025) · cites this paper as Mordatch & Abbeel, 2018
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Grice, H. P. (1975) · Speech Acts · doi:10.1163/9789004368811_003 · Cited by 23,221 · see citing works
Austin, J. L. (1975) · doi:10.1093/acprof:oso/9780198245537.001.0001 · Cited by 21,496 · see citing works
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · see citing works
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213 · see citing works
Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank
Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A., & Potts, C. (2013) · Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1170 · Cited by 6,878 · see citing works
Categorical Reparameterization with Gumbel-Softmax
Jang, E., Gu, S., & Poole, B. (2016) · arXiv · doi:10.48550/arxiv.1611.01144 · Cited by 3,616 · see citing works
Markov Games as a Framework for Multi-Agent Reinforcement Learning
Littman, M. L. (1994) · International Conference on Machine Learning · doi:10.1016/b978-1-55860-335-6.50027-1 · Cited by 3,406 · see citing works
Building machines that learn and think like people
Lake, B. M., Ullman, T. D., Tenenbaum, J. B., & Gershman, S. J. (2017) · Behavioral and Brain Sciences · doi:10.1017/s0140525x16001837 · Cited by 2,550 · see citing works
The Concrete Distribution: A Continuous Relaxation of Discrete Random VariablesPreprint
Maddison, C. J., Mnih, A., & Teh, Y. W. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.00712 · Cited by 1,771 · see citing works
Learning Multiagent Communication with BackpropagationPreprint
Sukhbaatar, S., Szlam, A., & Fergus, R. (2016) · Neural Information Processing Systems · doi:10.48550/arxiv.1605.07736 · Cited by 1,479 · see citing works
Learning to Communicate with Deep Multi-Agent Reinforcement Learning
Foerster, J. N., Assael, Y. M., de Freitas, N., & Whiteson, S. (2016) · arXiv · doi:10.48550/arxiv.1605.06676 · Cited by 1,217 · see citing works
Predicting pragmatic reasoning in language games
Frank, M. C., & Goodman, N. D. (2012) · Science · doi:10.1126/science.1218633 · Cited by 868 · see citing works
Kirby, S. (2001) · IEEE Transactions on Evolutionary Computation · doi:10.1109/4235.918430 · Cited by 544 · see citing works
The evolution of syntactic communication
Nowak, M. A., Plotkin, J. B., & Jansen, V. A. A. (2000) · Nature · doi:10.1038/35006635 · Cited by 359 · see citing works
Iterated learning and the evolution of language
Kirby, S., Griffiths, T., & Smith, K. (2014) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2014.07.014 · Cited by 313 · see citing works
A Self-Organizing Spatial Vocabulary
Steels, L. (1995) · Artificial Life · doi:10.1162/artl.1995.2.319 · Cited by 310 · see citing works
Multi-Agent Cooperation and the Emergence of (Natural) Language
Lazaridou, A., Peysakhovich, A., & Baroni, M. (2016) · arXiv · doi:10.48550/arxiv.1612.07182 · Cited by 231 · see citing works
The Predictron: End-To-End Learning and PlanningPreprint
Silver, D., van Hasselt, H. P., Hessel, M., Schaul, T., Guez, A., Harley, T., Dulac-Arnold, G., Reichert, D., Rabinowitz, N. C., Barreto, A. D. M. S., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.08810 · Cited by 187 · see citing works
Learning-Based Single-Document Summarization with Compression and Anaphoricity Constraints
Durrett, G., Berg-Kirkpatrick, T., & Klein, D. (2016) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p16-1188 · Cited by 150 · see citing works
A Game-Theoretic Approach to Generating Spatial Descriptions
Golland, D., Liang, P., & Klein, D. (2010) · Cited by 132 · see citing works
Learning Language Games through Interaction
Wang, S. I., Liang, P., & Manning, C. D. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1224 · Cited by 132 · see citing works
Reasoning about Pragmatics with Neural Listeners and Speakers
Andreas, J., & Klein, D. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1125 · Cited by 115 · see citing works
A procedural model of language understanding
Winograd, T. (1986) · Morgan Kaufmann Publishers Inc. eBooks · Cited by 104 · see citing works
Agent-Based Models of Strategies for the Emergence and Evolution of Grammatical Agreement
Beuls, K., & Steels, L. (2013) · PLoS ONE · doi:10.1371/journal.pone.0058960 · Cited by 101 · see citing works
Andreas, J., Dragan, A., & Klein, D. (2017) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p17-1022 · Cited by 61 · see citing works
Learning to Act by Predicting the FuturePreprint
Dosovitskiy, A., & Koltun, V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01779 · Cited by 60 · see citing works
The Pragmatics of Spatial Language.
Ullman, T. D., Xu, Y., & Goodman, N. D. (2016) · eScholarship (California Digital Library) · Cited by 40 · see citing works
Kirby, S. (1999) · Lecture notes in computer science · doi:10.1007/3-540-48304-7_91 · Cited by 38 · see citing works
Towards End-to-End Reinforcement Learning of Dialogue Agents for Information AccessPreprint
Dhingra, B., Li, L. C., Li, X., Gao, J., Chen, Y.-N., Ahmed, F., & Deng, L. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1609.00777 · Cited by 31 · see citing works
What triggers the emergence of grammar
Aoiz, Y. G., Marcos, J., Lahoz, L. J. A., Aladrén, E. F., Usón, J., de Andrés, M. D., & Ferreira, V. (2013) · Annual Meeting of the Association for Computational Linguistics · doi:10.13039/501100000782 · Cited by 28 · see citing works
A Paradigm for Situated and Goal-Driven Language LearningPreprint
Gauthier, J., & Mordatch, I. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.03585 · Cited by 24 · see citing works
Self-citationTowards Multi-Agent Communication-Based Language LearningPreprint
Lazaridou, A., Pham, N. T., & Baroni, M. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.07133 · Cited by 19 · see citing works
Bratman, J., Shvartsman, M., Lewis, R. L., & Singh, S. P. (2010) · Cited by 13 · see citing works
Learning to Reason Pragmatically with Cognitive Limitations
Vogel, A. P., Gómez-Emilsson, A., Frank, M. C., Jurafsky, D., & Potts, C. (2014) · eScholarship (California Digital Library) · Cited by 12 · see citing works
Flagged first, then by citations
5 of the 39 references couldn’t be matched to a known paper.
Research works printed but not linked: 2
- Ng, A. 2011. Sparse autoencoder. CS294A Lecture notes 72(2011):1-19.
- Teh, Y. W. 2011. Dirichlet process. In Encyclopedia of machine learning . Springer. 280-287.
Cited by
Sorted by citations
Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments
Lowe, R., Wu, Y., Tamar, A., Harb, J., Abbeel, P., & Mordatch, I. (2017) · arXiv · doi:10.48550/arxiv.1706.02275 · Cited by 3,377 · see citing works
Self-citationCounterfactual Multi-Agent Policy Gradients
Foerster, J., Farquhar, G., Afouras, T., Nardelli, N., & Whiteson, S. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11794 · Cited by 1,873 · see citing works
Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms
Zhang, K., Yang, Z., & Başar, T. (2021) · Studies in systems, decision and control · doi:10.1007/978-3-030-60990-0_12 · Cited by 1,403 · see citing works
Multi-agent deep reinforcement learning: a survey
Gronauer, S., & Diepold, K. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-021-09996-w · Cited by 957 · see citing works
A survey and critique of multiagent deep reinforcement learning
Hernández-Leal, P., Kartal, B., & Taylor, M. E. (2019) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-019-09421-1 · Cited by 747 · see citing works
Human-level performance in 3D multiplayer games with population-based reinforcement learning
Jaderberg, M., Czarnecki, W. M., Dunning, I., Marris, L., Lever, G., Castañeda, A. G., Beattie, C., Rabinowitz, N. C., Morcos, A. S., Ruderman, A., et al. (2019) · Science · doi:10.1126/science.aau6249 · Cited by 662 · see citing works
A review of cooperative multi-agent deep reinforcement learning
Oroojlooy, A., & Hajinezhad, D. (2022) · Applied Intelligence · doi:10.1007/s10489-022-04105-y · Cited by 607 · see citing works
Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learning
Das, A., Kottur, S., Moura, J. M. F., Lee, S., & Batra, D. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.321 · Cited by 362 · see citing works
Deal or No Deal? End-to-End Learning of Negotiation Dialogues
Lewis, M., Yarats, D., Dauphin, Y., Parikh, D., & Batra, D. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1259 · Cited by 347 · see citing works
Deep Communicating Agents for Abstractive Summarization
Çelikyılmaz, A., Bosselut, A., He, X., & Choi, Y. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-1150 · Cited by 327 · see citing works
Robust Multi-Agent Reinforcement Learning via Minimax Deep Deterministic Policy Gradient
Li, S., Wu, Y., Cui, X., Dong, H., Fang, F., & Russell, S. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33014213 · Cited by 304 · see citing works
Peng, P., Wen, Y., Yang, Y., Yuan, Q., Tang, Z., Long, H., & Wang, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.10069 · Cited by 257 · see citing works
MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learning
Li, Q., Peng, Z., Feng, L., Zhang, Q., Xue, Z., & Zhou, B. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3190471 · Cited by 246 · see citing works
TarMAC: Targeted Multi-Agent CommunicationPreprint
Das, A., Gervet, T., Romoff, J., Batra, D., Parikh, D., Rabbat, M., & Pineau, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.11187 · Cited by 231 · see citing works
Zhang, R., Zong, Q., Zhang, X., Dou, L., & Tian, B. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3146976 · Cited by 218 · see citing works
Hu, D., Ye, Z., Gao, Y., Ye, Z., Peng, Y., & Yu, N. (2022) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2022.3185975 · Cited by 216 · see citing works
Learning with Opponent-Learning Awareness
Foerster, J., Chen, R. Y., Al-Shedivat, M., Whiteson, S., Abbeel, P., & Mordatch, I. (2018) · International Joint Conference on Autonomous Agents and Multiagent Systems · doi:10.65109/hgwa8807 · Cited by 215 · see citing works
Self-citationMessage-Aware Graph Attention Networks for Large-Scale Multi-Robot Path Planning
Li, Q., Lin, W., Liu, Z., & Prorok, A. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3077863 · Cited by 200 · see citing works
A Deep Reinforcement Learning ChatbotPreprint
Serban, I. V., Sankar, C., Germain, M., Zhang, S., Lin, Z., Subramanian, S., Kim, T., Pieper, M., Chandar, S., Ke, N. R., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.02349 · Cited by 195 · see citing works
Emergence of Language with Multi-agent Games: Learning to Communicate with Sequences of Symbols
Havrylov, S., & Titov, I. (2017) · arXiv · doi:10.48550/arxiv.1705.11192 · Cited by 195 · see citing works
Transparency: Motivations and Challenges
Weller, A. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-28954-6_2 · Cited by 198 · see citing works
A survey on multi-agent reinforcement learning and its application
Ning, Z., & Xie, L.-H. (2024) · Journal of Automation and Intelligence · doi:10.1016/j.jai.2024.02.003 · Cited by 257 · see citing works
Survey on reinforcement learning for language processing
Uc-Cetina, V., Navarro-Guerrero, N., Martin-Gonzalez, A., Weber, C., & Wermter, S. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-022-10205-5 · Cited by 177 · see citing works
PettingZoo: Gym for Multi-Agent Reinforcement LearningPreprint
Terry, J. K., Black, B., Grammel, N., Jayakumar, M., Hari, A., Sullivan, R., Santos, L., Perez, R., Horsch, C., Dieffendahl, C., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.14471 · Cited by 193 · see citing works
Real-Time Bidding with Multi-Agent Reinforcement Learning in Display Advertising
Jin, J., Song, C., Li, H., Gai, K., Wang, J., & Zhang, W. (2018) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3269206.3272021 · Cited by 169 · see citing works
Natural language does not emerge ‘naturally’ in multi-agent dialog
Kottur, S., Moura, J., Lee, S., & Batra, D. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1321 · Cited by 164 · see citing works
A survey of multi-agent deep reinforcement learning with communication
Zhu, C.-X., Dastani, M., & Wang, S.-H. (2022) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-023-09633-6 · Cited by 364 · see citing works
Dark, Beyond Deep: A Paradigm Shift to Cognitive AI with Humanlike Common Sense
Zhu, Y., Gao, T., Fan, L., Huang, S., Edmonds, M., Liu, H., Gao, F., Zhang, C., Qi, S., Wu, Y. N., et al. (2020) · Engineering · doi:10.1016/j.eng.2020.01.011 · Cited by 135 · see citing works
Modeling Others using Oneself in Multi-Agent Reinforcement LearningPreprint
Răileanu, R., Denton, E., Szlam, A. D., & Fergus, R. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.09640 · Cited by 126 · see citing works
Learning to Delay in Ride-Sourcing Systems: A Multi-Agent Deep Reinforcement Learning Framework
Ke, J., Xiao, F., Yang, H., & Ye, J. (2020) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2020.3006084 · Cited by 156 · see citing works
Learning when to Communicate at Scale in Multiagent Cooperative and Competitive TasksPreprint
Singh, A., Jain, T., & Sukhbaatar, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.09755 · Cited by 124 · see citing works
If software is narrative: Joseph Weizenbaum, artificial intelligence and the biographies of ELIZA
Natale, S. (2019) · New Media & Society · doi:10.1177/1461444818804980 · Cited by 126 · see citing works
Zhao, T., Xie, K., & Eskénazi, M. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1123 · Cited by 114 · see citing works
Context-Aware Captions from Context-Agnostic Supervision
Vedantam, R., Samy Bengio, A., Murphy, K. P., Parikh, D., & Chechik, G. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.120 · Cited by 113 · see citing works
Cooperative Trajectory Design of Multiple UAV Base Stations With Heterogeneous Graph Neural Networks
Zhang, X., Zhao, H., Wei, J., Yan, C., Xiong, J., & Liu, X. (2022) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2022.3204794 · Cited by 110 · see citing works
Ray RLLib: A Composable and Scalable Reinforcement Learning LibraryPreprint
Liang, E., Liaw, R., Nishihara, R., Moritz, P., Fox, R., Gonzalez, J. E., Goldberg, K. Y., & Stoica, I. G. (2017) · arXiv (Cornell University) · Cited by 111 · see citing works
Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Survey
Majid, A. Y., Majid, A. Y., Saaybi, S., van Rietbergen, T., François-Lavet, V., Prasad, R. R. V., & Verhoeven, C. J. M. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3264540 · Cited by 108 · see citing works
A Review of the Gumbel-max Trick and its Extensions for Discrete Stochasticity in Machine Learning
Huijben, I. A. M., Kool, W., Paulus, M. B., & van Sloun, R. J. G. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3157042 · Cited by 85 · see citing works
Compositionality and Generalization In Emergent Languages
Chaabouni, R., Kharitonov, E., Bouchacourt, D., Emmanuel Dupoux, A., & Baroni, M. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.407 · Cited by 78 · see citing works
MO-MIX: Multi-Objective Multi-Agent Cooperative Decision-Making With Deep Reinforcement Learning
Hu, T., Luo, B., Yang, C., & Huang, T. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3283537 · Cited by 78 · see citing works
Contrasting Centralized and Decentralized Critics in Multi-Agent Reinforcement Learning
Lyu, X., Xiao, Y., Daley, B., & Amato, C. (2021) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/jcle4281 · Cited by 74 · see citing works
Emergent Communication in a Multi-Modal, Multi-Step Referential GamePreprint
Evtimova, K., Drozdov, A., Kiela, D., & Cho, K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.10369 · Cited by 72 · see citing works
Multiagent Trust Region Policy Optimization
Li, H., & He, H. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3265358 · Cited by 69 · see citing works
Kouzeghar, M., Song, Y., Meghjani, M., & Bouffanais, R. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10160919 · Cited by 74 · see citing works
Emergence of Linguistic Communication from Referential Games with Symbolic and Pixel Input
Lazaridou, A., Hermann, K. M., Tuyls, K., & Clark, S. (2018) · arXiv · doi:10.48550/arxiv.1804.03984 · Cited by 86 · see citing works
R-MADDPG for Partially Observable Environments and Limited CommunicationPreprint
Wang, R., Everett, M., & How, J. P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.06684 · Cited by 66 · see citing works
Wachi, A. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/832 · Cited by 64 · see citing works
Two Body Problem: Collaborative Visual Task Completion
Jain, U., Weihs, L., Kolve, E., Rastegari, M., Lazebnik, S., Farhadi, A., Schwing, A. G., & Kembhavi, A. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00685 · Cited by 62 · see citing works
From partners to populations: A hierarchical Bayesian account of coordination and convention.
Hawkins, R. D., Franke, M., Frank, M. C., Goldberg, A. E., Smith, K., Griffiths, T. L., & Goodman, N. D. (2022) · Psychological Review · doi:10.1037/rev0000348 · Cited by 63 · see citing works
Umbrello, S., & Yampolskiy, R. V. (2021) · International Journal of Social Robotics · doi:10.1007/s12369-021-00790-w · Cited by 62 · see citing works
Deep Multi Agent Reinforcement Learning for Autonomous Driving
Bhalla, S., Subramanian, S. G., & Crowley, M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-47358-7_7 · Cited by 59 · see citing works
Game of Drones: Intelligent Online Decision Making of Multi-UAV Confrontation
Liu, D., Zong, Q., Zhang, X., Zhang, R., Dou, L., & Tian, B. (2024) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2024.3360282 · Cited by 59 · see citing works
UAV Swarm Confrontation Using Hierarchical Multiagent Reinforcement Learning
Wang, B., Li, S., Xian-zhong, G., & Xie, T. (2021) · International Journal of Aerospace Engineering · doi:10.1155/2021/3360116 · Cited by 57 · see citing works
Message-Dropout: An Efficient Training Method for Multi-Agent Deep Reinforcement Learning
Kim, W., Cho, M.-S., & Sung, Y. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33016079 · Cited by 56 · see citing works
Measuring Compositionality in Representation LearningPreprint
Andreas, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.07181 · Cited by 64 · see citing works
Learning structured communication for multi-agent reinforcement learning
Sheng, J., Wang, X., Jin, B., Yan, J., Li, W., Chang, T., Wang, Y.-X., & Hongyuan Zha, B. (2022) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-022-09580-8 · Cited by 52 · see citing works
Multi-Agent Reinforcement Learning for Problems with Combined Individual and Team Reward
Sheikh, H. U., & Bölöni, L. L. (2020) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9206879 · Cited by 53 · see citing works
PIC: Permutation Invariant Critic for Multi-Agent Deep Reinforcement LearningPreprint
Liu, I.-J., Yeh, R. A., & Schwing, A. G. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.00025 · Cited by 52 · see citing works
LLM-Based Multi-Agent Decision-Making: Challenges and Future Directions
Sun, C., Huang, S., & Pompili, D. (2025) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2025.3562371 · Cited by 54 · see citing works
Target-Driven Structured Transformer Planner for Vision-Language Navigation
Zhao, Y., Chen, J., Gao, C., Wang, W., Yang, L., Ren, H., Xia, H., & Liu, S. (2022) · Proceedings of the 30th ACM International Conference on Multimedia · doi:10.1145/3503161.3548281 · Cited by 54 · see citing works
On the Pitfalls of Measuring Emergent Communication
Lowe, R., Foerster, J. N., Boureau, Y.-L., Pineau, J., & Dauphin, Y. (2019) · Adaptive Agents and Multi-Agent Systems · doi:10.65109/knvj7743 · Cited by 170 · see citing works
A Cordial Sync: Going Beyond Marginal Policies for Multi-agent Embodied Tasks
Jain, U., Weihs, L., Kolve, E., Farhadi, A., Lazebnik, S., Kembhavi, A., & Schwing, A. G. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58558-7_28 · Cited by 47 · see citing works
Wang, S., Lin, C., Han, G., Zhu, S., Li, Z., Wang, Z., & Ma, Y. (2024) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2024.3521889 · Cited by 46 · see citing works
Xie, J., Zhou, R., Liu, Y., Luo, J. T., Xie, S., Peng, Y., & Pu, H. (2021) · Applied Sciences · doi:10.3390/app11020546 · Cited by 43 · see citing works
Neural MMO: A Massively Multiagent Game Environment for Training and Evaluating Intelligent AgentsPreprint
Suárez, J., Du, Y., Isola, P., & Mordatch, I. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.00784 · Cited by 43 · see citing works
Self-citationReinforced Structured State-Evolution for Vision-Language Navigation
Chen, J., Gao, C., Meng, E., Zhang, Q., & Liu, S. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01501 · Cited by 45 · see citing works
Multiagent Adversarial Collaborative Learning via Mean-Field Theory
Luo, G.-Y., Zhang, H., He, H.-B., Li, J.-L., & Wang, F.-Y. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2020.3025491 · Cited by 34 · see citing works
Wang, Y., & Zhao, Y. (2025) · Ocean Engineering · doi:10.1016/j.oceaneng.2024.120244 · Cited by 40 · see citing works
Task Assignment of UAV Swarms Based on Deep Reinforcement Learning
Liu, B., Wang, S., Li, Q., Zhao, X., Pan, Y., & Wang, C. (2023) · Drones · doi:10.3390/drones7050297 · Cited by 39 · see citing works
Multi-Agent Reinforcement Learning for Dynamic Ocean Monitoring by a Swarm of Buoys
Kouzehgar, M., Meghjani, M., & Bouffanais, R. (2020) · Global Oceans 2020: Singapore – U.S. Gulf Coast · doi:10.1109/ieeeconf38699.2020.9389128 · Cited by 38 · see citing works
A Survey of Opponent Modeling in Adversarial Domains
Nashed, S. B., & Shlomo Zilberstein, B. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.12889 · Cited by 38 · see citing works
Games for Artificial Intelligence Research: A Review and Perspectives
Hu, C., Zhao, Y., Wang, Z., Du, H., & Liu, J. (2024) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2024.3410935 · Cited by 51 · see citing works
Mao, H., Gong, Z., Yan, N., & Xiao, Z. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.03235 · Cited by 37 · see citing works
Physics-Informed Multiagent Reinforcement Learning for Distributed Multirobot Problems
Sebastián, E., Duong, T., Atanasov, N., Montijano, E., & Sagüés, C. (2025) · IEEE Transactions on Robotics · doi:10.1109/tro.2025.3582836 · Cited by 36 · see citing works
A social path to human-like artificial intelligence
Duéñez‐Guzmán, E. A., Sadedin, S., Wang, J. X., McKee, K. R., & Leibo, J. Z. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00754-x · Cited by 39 · see citing works
Prospects for multi-agent collaboration and gaming: challenge, technology, and application
Liu, Y., Li, Z., Jiang, Z., & He, Y. (2022) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.2200055 · Cited by 32 · see citing works
Multi-agent actor centralized-critic with communication
Simões, D., Lau, N., & Reis, L. P. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.01.079 · Cited by 32 · see citing works
TRUST-REGION BASED POLICY OPTIMIZATION FOR EFFICIENT REINFORCEMENT LEARNING
Li, H. (2023) · doi:10.23860/diss-li-hepfeng-2023 · Cited by 32 · see citing works
Language evolution research in the year 2020
Nölle, J., Hartmann, S., & Tinits, P. (2020) · Language Dynamics and Change · doi:10.1163/22105832-bja10005 · Cited by 31 · see citing works
Toward Energy-Efficient Routing of Multiple AGVs with Multi-Agent Reinforcement Learning
Xian-feng, Y., Deng, Z., Shi, Y., & Shen, W. (2023) · Sensors · doi:10.3390/s23125615 · Cited by 31 · see citing works
Arena: A General Evaluation Platform and Building Toolkit for Multi-Agent Intelligence
Song, Y., Wojcicki, A., Lukasiewicz, T., Wang, J., Aryan, A., Xu, Z., Xu, M., Ding, Z., & Wu, L. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6216 · Cited by 30 · see citing works
Federated Control with Hierarchical Multi-Agent Deep Reinforcement LearningPreprint
Kumar, S., Shah, P., Hakkani-Tur, D. Z., & Heck, L. P. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.08266 · Cited by 31 · see citing works
MAGNet: Multi-agent Graph Network for Deep Multi-agent Reinforcement Learning
Malysheva, A., Kudenko, D., & Shpilman, A. (2019) · International Symposium on Problems of Redundancy in Information and Control Systems (REDUNDANCY) · doi:10.1109/redundancy48165.2019.9003345 · Cited by 30 · see citing works
Communicative Feedback in language acquisition
Nikolaus, M., & Fourtassi, A. (2022) · New Ideas in Psychology · doi:10.1016/j.newideapsych.2022.100985 · Cited by 31 · see citing works
Hierarchical Approaches for Reinforcement Learning in Parameterized Action SpacePreprint
Wei, E., Wicke, D., & Luke, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.09656 · Cited by 30 · see citing works
Fair collaborative vehicle routing: A deep multi-agent reinforcement learning approach
Mak, S., Xu, L., Pearce, T., Ostroumov, M., & Brintrup, A. M. (2023) · Transportation Research Part C Emerging Technologies · doi:10.1016/j.trc.2023.104376 · Cited by 31 · see citing works
Game-Theoretic Cooperative Lane Changing Using Data-Driven Models
Ding, G., Aghli, S., Heckman, C., & Chen, L. (2018) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8593725 · Cited by 30 · see citing works
Reinforcement learning and bandits for speech and language processing: Tutorial, review and outlook
Lin, B. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122254 · Cited by 28 · see citing works
Interpretation of Emergent Communication in Heterogeneous Collaborative Embodied Agents
Patel, S., Wani, S., Jain, U., Schwing, A. G., Lazebnik, S., Savva, M., & Chang, A. X. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01565 · Cited by 28 · see citing works
Policy Distillation and Value Matching in Multiagent Reinforcement Learning
Wadhwania, S., Kim, D.-K., Omidshafiei, S., & How, J. P. (2019) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8967849 · Cited by 28 · see citing works
Spatial Intention Maps for Multi-Agent Mobile Manipulation
Wu, J., Sun, X., Zeng, A., Song, S., Rusinkiewicz, S. M., & Funkhouser, T. A. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9561359 · Cited by 27 · see citing works
Emergence of Communication in an Interactive World with Consistent SpeakersPreprint
Bogin, B., Geva, M., & Berant, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.00549 · Cited by 27 · see citing works
Selection and Generation: Learning towards Multi-Product Advertisement Post Generation
Chan, Z., Zhang, Y., Chen, X., Gao, S., Zhang, Z., Zhao, D., & Yan, R. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.313 · Cited by 26 · see citing works
MAMBPO: Sample-efficient multi-robot reinforcement learning using learned world models
Willemsen, D., Coppola, M., & de Croon, G. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9635836 · Cited by 26 · see citing works
Learning Existing Social Conventions via Observationally Augmented Self-Play
Lerer, A. K., & Peysakhovich, A. (2019) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3306618.3314268 · Cited by 25 · see citing works
On Centralized Critics in Multi-Agent Reinforcement Learning
Lyu, X., Baisero, A., Xiao, Y., Daley, B., & Amato, C. (2023) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.14386 · Cited by 24 · see citing works
ET-HF: A novel information sharing model to improve multi-agent cooperation
Xie, S., Zhang, H., Yu, H., Li, Y., Zhang, Z., & Luo, X. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.109916 · Cited by 24 · see citing works
Si, J., Li, X., Gao, L., & Li, P. (2024) · International Journal of Production Research · doi:10.1080/00207543.2024.2335663 · Cited by 24 · see citing works
Busch, J. V. S., Latzko, V., Martin Reisslein, B., & Fitzek, F. H. P. (2020) · IEEE Open Journal of Intelligent Transportation Systems · doi:10.1109/ojits.2020.3027518 · Cited by 23 · see citing works
- Authors
- Mordatch, I., & Abbeel, P.
- Affiliations
- OpenAI (United States) (US)
- University of California, Berkeley (US)
- ORCID
- Abbeel, Pieter
- Venue
- Proceedings of the AAAI Conference on Artificial Intelligence
- Published
- 2018-04-25
- Volume
- 32
- Issue
- 1
- ISSN
- 2159-5399 (ISSN-L)
- 2374-3468
- OpenAlex
- W2602275733
- Cited by
- 440
- References
- 39
- Citations by year
- 2026: 24 · 2025: 43 · 2024: 60 · 2023: 68 · 2022: 55 · 2021: 45 · 2020: 57 · 2019: 43 · 2018: 27 · 2017: 18
- FWCI
- 397.13
- Type
- conference paper
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Proceedings of the AAAI Conference on Artificial Intelligence (publishedVersion)
- AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) (submittedVersion)
- Topics
- Social Sciences › Cultural Studies › Language and cultural evolution
- Physical Sciences › Computer Science › Artificial Intelligence › Multi-Agent Systems and Negotiation
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Keywords
- compositional language, language emergence, multi-agent systems, nonverbal communication, pointing, guiding, human-agent interaction, goal-oriented communication
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
