Decision Making in Multiagent Systems: A Survey
The DOI matches its Crossref record; 263 of the 275 references it cites were found in a registry or an index.
Abstract
Intelligent transport systems, efficient electric grids, and sensor networks for data collection and analysis are some examples of the multiagent systems (MAS) that cooperate to achieve common goals. Decision making is an integral part of intelligent agents and MAS that will allow such systems to accomplish increasingly complex tasks. In this survey, we investigate state-of-the-art work within the past five years on cooperative MAS decision making models, including Markov decision processes, game theory, swarm intelligence, and graph theoretic models. We survey algorithms that result in optimal and suboptimal policies such as reinforcement learning, dynamic programming, evolutionary computing, and neural networks. We also discuss the application of these models to robotics, wireless sensor networks, cognitive radio networks, intelligent transport systems, and smart electric grids. In addition, we define key terms in the area and discuss remaining challenges that include incorporating big data advancements to decision making, developing autonomous, scalable and computationally efficient algorithms, tackling more complex tasks, and developing standardized evaluation metrics. While recent surveys have been published on this topic, we present a broader discussion of related models and applications.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The objective is to maximize the system's overall reward, while agents work together to accomplish a common task [54].”
From Leveraging 3GPP Features and Optimization Techniques for 5G NR-V2X Resource Allocation: A Survey (2025) · cites this paper as [54]
Citations
Cites
Kennedy, J., & Eberhart, R. (2002) · Proceedings of ICNN'95 - International Conference on Neural Networks · doi:10.1109/icnn.1995.488968 · Cited by 58,192
Artificial intelligence: a modern approach
(1995) · Choice Reviews Online · doi:10.5860/choice.33-1577 · Cited by 21,974
Particle swarm optimization: An overview
Poli, R., Kennedy, J., & Blackwell, T. (2007) · Swarm Intelligence · doi:10.1007/s11721-007-0002-0 · Cited by 21,101
Artificial intelligence: A modern approach
Nilsson, N. J. (1996) · Artificial Intelligence · doi:10.1016/0004-3702(96)00007-0 · Cited by 10,583
Consensus and Cooperation in Networked Multi-Agent Systems
Olfati-Saber, R., Fax, J. A., & Murray, R. M. (2007) · Proceedings of the IEEE · doi:10.1109/jproc.2006.887293 · Cited by 10,522 · cited 1×
“Graphical models for consensus [157], formation [158], [159], and rendezvous [158] have also been investigated.”
BackgroundEquilibrium points in n -person games
Nash, J. F. (1950) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.36.1.48 · Cited by 7,560 · cited 1×
“For example, stochastic games with a finite number of players, actions and states always have a Nash equilibrium, defined as the strategy profile which maximizes each player’s utility knowing the strategy of others in the game [111].”
BackgroundAsynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132 · cited 1×
“approximate methods are adopted and include model-free RL approaches like Q-learning [54] and SARSA [55], evolutionary computing [56], [57], RNN [58]–[63], and deep RL [64], [65].”
MethodSwarm Intelligence: From Natural to Artificial Systems
Bonabeau, E., Dorigo, M., & Theraulaz, G. (1999) · Oxford University Press eBooks · doi:10.1093/oso/9780195131581.001.0001 · Cited by 6,435
Watkins, C. (1989) · OpenGrey (Institut de l'Information Scientifique et Technique) · doi:10.17863/cam.132164 · Cited by 5,297
SMITH, J. M., & PRICE, G. R. (1973) · Nature · doi:10.1038/246015a0 · Cited by 5,089 · cited 1×
“Evolutionary stable strategy is a refinement of the Nash equilibrium which requires a strategy to be stable to any perturbations that may occur to the games as they evolve [112] and was extended to stochastic games [113].”
BackgroundGame theory: analysis of conflict
(1992) · Choice Reviews Online · doi:10.5860/choice.29-2753 · Cited by 3,325
Biomimicry of bacterial foraging for distributed optimization and control
(2002) · IEEE Control Systems · doi:10.1109/mcs.2002.1004010 · Cited by 3,069 · cited 1×
“Also, swarm intelligence including PSO [252], [253] and an algorithm that mimics bacterial foraging [254] have been used to solve multiobjective optimization problems.”
MethodGraph Theoretic Methods in Multiagent Networks
Mesbahi, M., & Egerstedt, M. (2010) · Princeton University Press eBooks · doi:10.1515/9781400835355 · Cited by 2,897 · cited 1×
“Decision making in MAS have been modeled as graphs with nodes representing agents and edges representing interactions and information flow among agents [130].”
BackgroundMulti-Agent Systems: An Introduction to Distributed Artificial Intelligence
Ferber, J. (1999) · UCLA - Biblioteca de Ciencias y Tecnologia Felix Morales Bueno · Cited by 2,624
A survey of multi-agent formation control
Oh, K.-K., Park, M.-C., & Ahn, H.-S. (2015) · Automatica · doi:10.1016/j.automatica.2014.10.022 · Cited by 2,457 · cited 1×
“The following surveys discussed multiple models for one application such as formation control and coordination [21], [22], task allocation [23], [24], intrusion detection [25]–[27], and smart electric grids [28].”
BackgroundIntention, Plans, and Practical Reason.
Velleman, J. D., & Bratman, M. E. (1991) · The Philosophical Review · doi:10.2307/2185304 · Cited by 2,384
Shapley, L. S. (1953) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.39.10.1095 · Cited by 2,382
A Comprehensive Survey of Multiagent Reinforcement Learning
Busoniu, L., Babuska, R., & De Schutter, B. (2008) · IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/tsmcc.2007.913919 · Cited by 2,235 · cited 1×
“Markov decision process (Dec-POMDP) [9], swarm intelligence in robotics [10]–[14], and multiagent RL [15], [16].”
BackgroundA comprehensive survey: artificial bee colony (ABC) algorithm and applications
Karaboga, D., Gorkemli, B., Ozturk, C., & Karaboga, N. (2014) · Artificial Intelligence Review · doi:10.1007/s10462-012-9328-0 · Cited by 1,975 · cited 1×
“Multiobjective particle swarm optimization (PSO) variants [17], algorithms based on bees [18], metaheuristic algorithms [19], and artificial bee colony variants and applications [20] have also been surveyed.”
BackgroundSwarm robotics: a review from the swarm engineering perspective
Brambilla, M., Ferrante, E., Birattari, M., & Dorigo, M. (2013) · Swarm Intelligence · doi:10.1007/s11721-012-0075-2 · Cited by 1,762 · cited 1×
“While such systems exhibit desirable properties like robustness, flexibility, scalability, low complexity, inherent parallelism, and fault tolerance [11], [129], they have important limitations.”
BackgroundMultiagent systems: algorithmic, game-theoretic, and logical foundations
(2009) · Choice Reviews Online · doi:10.5860/choice.46-5662 · Cited by 1,756 · cited 6×
“They can also be viewed as a generalization of repeated games where a game from a collection of normal form games can be played at a given step [37].”
BackgroundInfluentialConstrained Markov Decision Processes: Stochastic Modeling
Altman, E. (2021) · Routledge eBooks · doi:10.1201/9781315140223 · Cited by 1,693
Weiß, G. (1999) · Cited by 1,640
Intrusion detection system: A comprehensive review
Liao, H.-J., Richard Lin, C.-H., Lin, Y.-C., & Tung, K.-Y. (2013) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2012.09.004 · Cited by 1,630
On-line Q-learning Using Connectionist Systems
Rummery, G. A. (1994) · Cited by 1,477
Intention, Plans, and Practical Reason.
McCann, H. J., & Bratman, M. E. (1991) · Noûs · doi:10.2307/2215590 · Cited by 1,472 · cited 1×
“While other models exist, such as belief-desire-intention models based on the human’s practical reasoning theory [33] and independent choice logic which combined probabilistic information with logic programming to represent knowledge [34], their MAS extensions [35], [36] have not been widely adopted in the recent work.”
BackgroundMonte-Carlo Planning in Large POMDPs
Silver, D., & Veness, J. (2010) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 1,408
Multi-Objective Particle Swarm Optimizers: A Survey of the State-of-the-Art
Coello Coello, C. A., & Reyes-Sierra, M. (2006) · International Journal of Computational Intelligence Research · doi:10.5019/j.ijcir.2006.68 · Cited by 1,365 · cited 1×
“Multiobjective particle swarm optimization (PSO) variants [17], algorithms based on bees [18], metaheuristic algorithms [19], and artificial bee colony variants and applications [20] have also been surveyed.”
BackgroundA Survey of Distributed Optimization and Control Algorithms for Electric Power Systems
Molzahn, D. K., Dorfler, F., Sandberg, H., Low, S. H., Chakrabarti, S., Baldick, R., & Lavaei, J. (2017) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2017.2720471 · Cited by 1,273 · cited 1×
“Since many controllers are based on optimization algorithms, distributed optimization is an integral part of distributed control [136].”
BackgroundCooperative Multi-Agent Learning: The State of the Art
Panait, L., & Luke, S. (2005) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-005-2631-2 · Cited by 1,237 · cited 2×
“Reference [5] used the terms team learning and concurrent”
BackgroundMethodMultiagent systems: A survey from a machine learning perspective
Stone, P., & Veloso, M. (2000) · Autonomous Robots · doi:10.1023/a:1008942012299 · Cited by 1,218 · cited 1×
“prey-predator pursuit, air traffic control, and others [4], [5].”
BackgroundDistributed Optimization Over Time-Varying Directed Graphs
Nedic, A., & Olshevsky, A. (2015) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2014.2364096 · Cited by 1,187
Multiagent Systems: Algorithmic, Game-Theoretic, and Logical Foundations
Shoham, Y., & Leyton-Brown, K. (2008) · doi:10.1017/cbo9780511811654 · Cited by 995
Swarm Intelligence in Cellular Robotic Systems
Beni, G., & Wang, J. (1993) · Robots and Biological Systems: Towards a New Bionics? · doi:10.1007/978-3-642-58069-7_38 · Cited by 974 · cited 1×
“toward a common global goal [120].”
Recent Research in Cooperative Control of Multivehicle Systems
Murray, R. M. (2007) · Journal of Dynamic Systems, Measurement, and Control · doi:10.1115/1.2766721 · Cited by 960 · cited 1×
“mobile sensor networks were surveyed in [7] who concluded that additional work in system integration, distributed embedded system verification and decision making at higher”
BackgroundFinite-time formation control for multi-agent systems
Xiao, F., Wang, L., Chen, J., & Gao, Y. (2009) · Automatica · doi:10.1016/j.automatica.2009.07.012 · Cited by 934
Nash Q-Learning for General-Sum Stochastic Games
Hu, J., Wellman, M. P., & Boutilier, C. (2003) · Cited by 923
Distributed Coordination Control of Multiagent Systems While Preserving Connectedness
Ji, M., & Egerstedt, M. (2007) · IEEE Transactions on Robotics · doi:10.1109/tro.2007.900638 · Cited by 880 · cited 1×
“Graphical models for consensus [157], formation [158], [159], and rendezvous [158] have also been investigated.”
BackgroundDistributed Continuous-Time Convex Optimization on Weight-Balanced Digraphs
Gharesifard, B., & Cortes, J. (2014) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2013.2278132 · Cited by 833 · cited 1×
“Graph theory including time-varying directed graphs [255] and weight-balanced directed graphs [256] have modeled information exchange in a distributed optimization framework.”
BackgroundHoward, R. A., & Matheson, J. E. (2005) · Decision Analysis · doi:10.1287/deca.1050.0020 · Cited by 819 · cited 1×
“a graph theoretic approach that provide a framework for decision making by adding actions and utilities to Bayesian networks [131].”
BackgroundParticle Swarm Optimization in Wireless-Sensor Networks: A Brief Survey
Kulkarni, R. V., & Venayagamoorthy, G. K. (2011) · IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/tsmcc.2010.2054080 · Cited by 732 · cited 1×
“selection [216], for security protocols [215], [217], and node positioning and localization [218], [219].”
BackgroundCognitive architectures: Research issues and challenges
Langley, P., Laird, J. E., & Rogers, S. (2009) · Cognitive Systems Research · doi:10.1016/j.cogsys.2006.07.004 · Cited by 728 · cited 1×
“decision making, problem solving, interaction, and communication [42].”
BackgroundShapley, L. S. (1953) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.39.10.1953 · Cited by 677
Dynamic programming for partially observable stochastic games
Hansen, E. A., Bernstein, D. S., & Zilberstein, S. (2004) · National Conference on Artificial Intelligence · Cited by 660
A Survey and Analysis of Multi-Robot Coordination
Yan, Z., Jouandeau, N., & Cherif, A. A. (2013) · International Journal of Advanced Robotic Systems · doi:10.5772/57313 · Cited by 647
A survey: algorithms simulating bee swarm intelligence
Karaboga, D., & Akay, B. (2009) · Artificial Intelligence Review · doi:10.1007/s10462-009-9127-4 · Cited by 646 · cited 1×
“Multiobjective particle swarm optimization (PSO) variants [17], algorithms based on bees [18], metaheuristic algorithms [19], and artificial bee colony variants and applications [20] have also been surveyed.”
BackgroundPigeon-inspired optimization: a new swarm intelligence optimizer for air robot path planning
Duan, H., & Qiao, P. (2014) · International Journal of Intelligent Computing and Cybernetics · doi:10.1108/ijicc-02-2014-0005 · Cited by 601 · cited 2×
“A pigeon inspired optimization algorithm relied on the magnetic field, sun and landmarks to achieve path planning [125].”
MethodReinforcement learning-based multi-agent system for network traffic signal control
Arel, I., Liu, C., Urbanik, T., & Kohls, A. G. (2010) · IET Intelligent Transport Systems · doi:10.1049/iet-its.2009.0070 · Cited by 592
Secondary control of microgrids based on distributed cooperative control of multi‐agent systems
Bidram, A., Davoudi, A., Lewis, F. L., & Qu, Z. (2013) · IET Generation, Transmission & Distribution · doi:10.1049/iet-gtd.2012.0576 · Cited by 541 · cited 1×
“For example, a Lyapunov-based voltage and frequency controller was designed for micro-grid systems that only requires local communication among neighbors [137] and a secondary voltage distributed controller based on input– output feedback linearization that requires sparse communication [138].”
BackgroundThe complexity of stochastic games
Condon, A. (1992) · Information and Computation · doi:10.1016/0890-5401(92)90048-k · Cited by 517 · cited 1×
“They belong to the complexity class NP ∩ co − NP [109] and can be solved using DP [108], Q-learning [110], and linear programming under certain conditions [37].”
BackgroundGame theory for cognitive radio networks: An overview
Wang, B., Wu, Y., & Liu, K. J. R. (2010) · Computer Networks · doi:10.1016/j.comnet.2010.04.004 · Cited by 497 · cited 1×
“were modeled using stochastic games [235] and spectrum sharing was modeled using game theory [236].”
MethodFast gradient-descent methods for temporal-difference learning with linear function approximation
Sutton, R. S., Maei, H. R., Precup, D., Bhatnagar, S., Silver, D., Szepesvári, C., & Wiewiora, E. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553501 · Cited by 496
Guest editorial advances in multirobot systems
Arai, T., Pagello, E., & Parker, L. E. (2002) · IEEE Transactions on Robotics and Automation · doi:10.1109/tra.2002.806024 · Cited by 491
A novel SVM-kNN-PSO ensemble method for intrusion detection system
Aburomman, A. A., & Ibne Reaz, M. B. (2016) · Applied Soft Computing · doi:10.1016/j.asoc.2015.10.011 · Cited by 490 · cited 1×
“PSO has been widely applied in combination with support vector machines [225]–[227] and linear programming [228].”
MethodA Survey of Game Theory as Applied to Network Security
Roy, S., Ellis, C., Shiva, S., Dasgupta, D., Shandilya, V., & Wu, Q. (2010) · 2010 43rd Hawaii International Conference on System Sciences · doi:10.1109/hicss.2010.35 · Cited by 482 · cited 1×
“Many models have been adopted in intrusion detection systems, as surveyed in [220].”
MethodA review of swarm robotics tasks
Bayındır, L. (2016) · Neurocomputing · doi:10.1016/j.neucom.2015.05.116 · Cited by 445 · cited 1×
“Markov decision process (Dec-POMDP) [9], swarm intelligence in robotics [10]–[14], and multiagent RL [15], [16].”
BackgroundHe, W., Qian, F., Lam, J., Chen, G., Han, Q.-L., & Kurths, J. (2015) · Automatica · doi:10.1016/j.automatica.2015.09.028 · Cited by 429
Playing games for security: an efficient exact algorithm for solving Bayesian Stackelberg games
Paruchuri, P., Pearce, J. P., Marecki, J., Tambe, M., Ordóñez, F., & Kraus, S. (2008) · doi:10.5555/1402298.1402348 · Cited by 424
Swarmanoid: A Novel Concept for the Study of Heterogeneous Robotic Swarms
Dorigo, M., Floreano, D., Gambardella, L. M., Mondada, F., Nolfi, S., Baaboura, T., Birattari, M., Bonani, M., Brambilla, M., Brutschy, A., et al. (2013) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2013.2252996 · Cited by 410
Multi-agent influence diagrams for representing and solving games
Koller, D., & Milch, B. (2003) · Games and Economic Behavior · doi:10.1016/s0899-8256(02)00544-4 · Cited by 409 · cited 1×
“erating decision rules that depend on decision rules made by other agents [134].”
BackgroundMultiagent Planning with Factored MDPs
Guestrin, C., Koller, D., & Parr, R. (2001) · Cited by 405
New inspirations in swarm intelligence: a survey
Parpinelli, R. S., & Lopes, H. S. (2011) · International Journal of Bio-Inspired Computation · doi:10.1504/ijbic.2011.038700 · Cited by 402 · cited 1×
“Multiobjective particle swarm optimization (PSO) variants [17], algorithms based on bees [18], metaheuristic algorithms [19], and artificial bee colony variants and applications [20] have also been surveyed.”
BackgroundSwarm intelligence based routing protocol for wireless sensor networks: Survey and future directions
Saleem, M., Di Caro, G. A., & Farooq, M. (2011) · Information Sciences · doi:10.1016/j.ins.2010.07.005 · Cited by 386
The independent choice logic for modelling multiple agents under uncertainty
Poole, D. (1997) · Artificial Intelligence · doi:10.1016/s0004-3702(97)00027-1 · Cited by 383 · cited 1×
“While other models exist, such as belief-desire-intention models based on the human’s practical reasoning theory [33] and independent choice logic which combined probabilistic information with logic programming to represent knowledge [34], their MAS extensions [35], [36] have not been widely adopted in the recent work.”
BackgroundMassively Parallel Methods for Deep Reinforcement Learning
Nair, A., Srinivasan, P., Blackwell, S., Alcicek, C., Fearon, R., De Maria, A., Panneershelvam, V., Suleyman, M., Beattie, C., Petersen, S., et al. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.04296 · Cited by 379 · cited 1×
“approximate methods are adopted and include model-free RL approaches like Q-learning [54] and SARSA [55], evolutionary computing [56], [57], RNN [58]–[63], and deep RL [64], [65].”
MethodThe Communicative Multiagent Team Decision Problem: Analyzing Teamwork Theories and Models
Pynadath, D. V., & Tambe, M. (2002) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1024 · Cited by 376
Research Advance in Swarm Robotics
Tan, Y., & Zheng, Z.-Y. (2013) · Defence Technology · doi:10.1016/j.dt.2013.03.001 · Cited by 359
He, W., Chen, G., Han, Q.-L., & Qian, F. (2017) · Information Sciences · doi:10.1016/j.ins.2015.06.005 · Cited by 345
Esmin, A. A. A., Coelho, R. A., & Matwin, S. (2015) · Artificial Intelligence Review · doi:10.1007/s10462-013-9400-4 · Cited by 340 · cited 1×
“Finally, certain surveys focused on a single model applied to one application such as multiagent RL for robotics [29], swarm intelligence for robot path planning [30], PSO for clustering [31], and swarm intelligence for data mining [32].”
BackgroundZhang, H., Feng, T., Yang, G.-H., & Liang, H. (2015) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2350511 · Cited by 327 · cited 1×
“Distributed consensus tracking was achieved by designing: distributed adaptive controllers in weakly connected, directed graphs [140], a distributed optimal control algorithm [141], and nonlinear distributed impulsive control (control signals are given as impulses instead of continuously) with delayed”
MethodLeader-Following Consensus of Nonlinear Multiagent Systems With Stochastic Sampling
He, W., Zhang, B., Han, Q.-L., Qian, F., Kurths, J., & Cao, J. (2016) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2015.2514119 · Cited by 310 · cited 1×
“Stochastic sampling in leader-follower consensus problems has been shown to improve scalability of MAS [143].”
BackgroundDirectional routing and scheduling for green vehicular delay tolerant networks
Zeng, Y., Xiang, K., Li, D., & Vasilakos, A. V. (2013) · Wireless Networks · doi:10.1007/s11276-012-0457-9 · Cited by 298 · cited 1×
“ery in vehicular ad hoc networks [179].”
Sequential Optimality and Coordination in Multiagent Systems
Boutilier, C. E. (1999) · Cited by 294
Parker, L. E., Rus, D., & Sukhatme, G. S. (2016) · Springer Handbooks · doi:10.1007/978-3-319-32552-1_53 · Cited by 290 · cited 1×
“Agent heterogeneity stems from diverse sensing and actuating capabilities, computing resources, cognitive algorithms, and morphology [39].”
BackgroundDistributed Optimization Based on a Multiagent System in the Presence of Communication Delays
Yang, S., Liu, Q., & Wang, J. (2017) · IEEE Transactions on Systems, Man, and Cybernetics: Systems · doi:10.1109/tsmc.2016.2531649 · Cited by 289 · cited 1×
“MAS decision making leveraged consensus and communication rules to model distributed optimization; each agent optimized part of the objective function before combining their results [248].”
Zungeru, A. M., Ang, L.-M., & Seng, K. P. (2012) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2012.03.004 · Cited by 288
Dynamic programming and influence diagrams
Tatman, J. A., & Shachter, R. D. (1990) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/21.52548 · Cited by 266 · cited 15×
“A. Influence Diagrams IDs are referred to as decision networks in [41] and are a graph theoretic approach that provide a framework for decision making by adding actions and utilities to Bayesian networks [131].”
MethodInfluentialOptimal and Approximate Q-value Functions for Decentralized POMDPs
Oliehoek, F. A., Spaan, M. T. J., & Vlassis, N. (2008) · Journal of Artificial Intelligence Research · doi:10.1613/jair.2447 · Cited by 265 · cited 1×
“Approximate solutions have been proposed based on bounded policy iteration [90], Q-value function methods [91], multiagent A∗ [81], genetic algorithms [82], DP [92]–[94], and a Bayesian learning, stick-breaking policy algorithm [95].”
MethodMulti-Agent Reinforcement Learning:a critical survey
Shoham, Y., Powers, R., & Grenager, T. (2003) · Cited by 263
Training Recurrent Networks by Evolino
Schmidhuber, J., Wierstra, D., Gagliolo, M., & Gomez, F. (2007) · Neural Computation · doi:10.1162/neco.2007.19.3.757 · Cited by 248
Distributed intelligence: overview of the field and its application in multi-robot systems
Parker, L. E. (2008) · Journal of Physical Agents (JoPha) · doi:10.14198/jopha.2008.2.1.02 · Cited by 245 · cited 1×
“exhibit different types of interactions based on agent goals, resources, and skills [43], [44].”
BackgroundA Bayesian game approach for intrusion detection in wireless ad hoc networks
Liu, Y., Comaniciu, C., & Man, H. (2006) · Proceeding from the 2006 workshop on Game theory for communications and networks - GameNets '06 · doi:10.1145/1190195.1190198 · Cited by 242 · cited 1×
“Bayesian games modeled attacker/defender games [223].”
Opportunities for multiagent systems and multiagent reinforcement learning in traffic control
Bazzan, A. L. C. (2009) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-008-9062-9 · Cited by 235 · cited 1×
“used for routing algorithms [181]–[183], adaptive broadcasting [184], adaptive data collection [185], and traffic signal control [186]–[193].”
MethodThe Complexity of Decentralized Control of Markov Decision Processes
Bernstein, D. S., Shlomo Zilberstein, B., & Immerman, N. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.3836 · Cited by 232 · cited 1×
“Finite horizon dec-MDP was proven to be worst case NEXP-complete (solvable in exponential time using a nondeterministic Turning machine), when three or more agents are considered [70].”
MethodDistributed Q-Learning for Aggregated Interference Control in Cognitive Radio Networks
Galindo-Serrano, A., & Giupponi, L. (2010) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2010.2043124 · Cited by 226
UAV Path Planning in a Dynamic Environment via Partially Observable Markov Decision Process
Ragi, S., & Chong, E. K. P. (2013) · IEEE Transactions on Aerospace and Electronic Systems · doi:10.1109/taes.2013.6621824 · Cited by 225 · cited 1×
“MDPs [150] and POMDP [151]–[155] have been used for robotics coordination including robot soccer [156].”
MethodEditorial survey: swarm intelligence for data mining
Martens, D., Baesens, B., & Fawcett, T. (2011) · Machine Learning · doi:10.1007/s10994-010-5216-5 · Cited by 219 · cited 1×
“Finally, certain surveys focused on a single model applied to one application such as multiagent RL for robotics [29], swarm intelligence for robot path planning [30], PSO for clustering [31], and swarm intelligence for data mining [32].”
BackgroundParker, L. E. (2008) · Springer Handbook of Robotics · doi:10.1007/978-3-540-30301-5_41 · Cited by 217
Singh, B., & Lobiyal, D. K. (2012) · Human-centric Computing and Information Sciences · doi:10.1186/2192-1962-2-13 · Cited by 215 · cited 1×
“selection [216], for security protocols [215], [217], and node positioning and localization [218], [219].”
BackgroundUrban traffic signal control using reinforcement learning agents
Balaji, P. G., German, X., & Srinivasan, D. (2010) · IET Intelligent Transport Systems · doi:10.1049/iet-its.2009.0096 · Cited by 213
The cooperation of swarm-bots - Physical interactions in collective robotics
Mondada, F., Gambardella, L. M., Floreano, D., Nolfi, S., Deneubourg, J., & Dorigo, M. (2005) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2005.1458313 · Cited by 212 · cited 1×
“Swarm-bots, wheeled robots that can physically connect to each other and form larger entities, accomplished coordinated motion, self-assembly, cooperative transport, goal search, and path formation [169], [170].”
BackgroundSwarm intelligence in intrusion detection: A survey
Kolias, C., Kambourakis, G., & Maragoudakis, M. (2011) · Computers & Security · doi:10.1016/j.cose.2011.08.009 · Cited by 210 · cited 2×
“Swarm intelligence is a popular approach to intrusion detection, evidenced by the recently published surveys [25]–[27].”
BackgroundNetworked distributed POMDPs: a synthesis of distributed constraint optimization and POMDPs
Nair, R., Varakantham, P. R., Tambe, M., & Yokoo, M. (2005) · Cited by 206
Traffic light control in non-stationary environments based on multi agent Q-learning
Abdoos, M., Mozayani, N., & Bazzan, A. L. C. (2011) · 2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2011.6083114 · Cited by 205
Solving Deep Memory POMDPs with Recurrent Policy Gradients
Wierstra, D., Foerster, A., Peters, J., & Schmidhuber, J. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-74690-4_71 · Cited by 205 · cited 1×
“When they are not known, heuristic search algorithms [81], genetic algorithms [82], RNN [83]–[86], and model-free RL methods were applied.”
MethodKhamis, M. A., & Gomaa, W. (2014) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2014.01.007 · Cited by 204
A review of studies in swarm robotics
Bayındır, L., & Şahin, E. (2007) · OpenMETU (Middle East Technical University) · Cited by 202
Coordinated Multi-Microgrids Optimal Control Algorithm for Smart Distribution Management System
Wu, J., & Guan, X. (2013) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2013.2269481 · Cited by 200 · cited 1×
“POSG [237], POMDP [238], [239], multiagent RL [240], [241], and swarm intelligence [28], [242]–[244]”
BackgroundMarkov Decision Processes With Applications in Wireless Sensor Networks: A Survey
Abu Alsheikh, M., Hoang, D. T., Niyato, D., Tan, H.-P., & Lin, S. (2015) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2015.2420686 · Cited by 195 · cited 1×
“ious problems in WSN including intrusion detection, sensor coverage, object detection, data exchange, topology formulation, and other problems [203].”
BackgroundFormal models and algorithms for decentralized decision making under uncertainty
Seuken, S., & Zilberstein, S. (2008) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-007-9026-5 · Cited by 190 · cited 2×
“The multiagent team decision problem [97], equivalent to Dec-POMDP when agents have perfect recall [98], extends economic team theory to robotics.”
BackgroundMethodMarkov decision processes with applications in wireless sensor networks: A survey
ALSHEIKH, A. M., Hoang, D. T., Niyato, D. T., & Tan, H.-P. (2015) · Singapore Management University Institutional Knowledge (InK) (Singapore Management University) · Cited by 189
Swarm intelligence systems for transportation engineering: Principles and applications
Teodorović, D. (2008) · Transportation Research Part C: Emerging Technologies · doi:10.1016/j.trc.2008.03.002 · Cited by 188 · cited 1×
“utilized swarm intelligence to control traffic light scheduling [194], model complex transportation systems [195], and develop routing protocols for vehicles [196], [197] and for”
MethodBarca, J. C., & Sekercioglu, Y. A. (2013) · Robotica · doi:10.1017/s026357471200032x · Cited by 187
Decentralized control of partially observable Markov decision processes
Amato, C., Chowdhary, G. V., Geramifard, A., Üre, N. K., & Kochenderfer, M. J. (2013) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc.2013.6760239 · Cited by 180
Swarm intelligence for traffic light scheduling: Application to real urban areas
García-Nieto, J., Alba, E., & Olivera, A. C. (2011) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2011.04.011 · Cited by 172 · cited 1×
“utilized swarm intelligence to control traffic light scheduling [194], model complex transportation systems [195], and develop routing protocols for vehicles [196], [197] and for”
MethodA sensor deployment approach using glowworm swarm optimization algorithm in wireless sensor networks
Liao, W.-H., Kao, Y., & Li, Y.-S. (2011) · Expert Systems with Applications · doi:10.1016/j.eswa.2011.03.053 · Cited by 170 · cited 1×
“selection [216], for security protocols [215], [217], and node positioning and localization [218], [219].”
BackgroundGet in touch: cooperative decision making based on robot-to-robot collisions
Schmickl, T., Thenius, R., Möslinger, C., Radspieler, G., Kernbach, S., Szymanski, M., & Crailsheim, K. (2008) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-008-9058-5 · Cited by 169 · cited 1×
“It has been applied to dynamic task allocation [161], distributed localization problems [162], foraging tasks [163]–[165], collision free navigation [166], [167], and communication free flocking with minimal memory requirements [168].”
Vincent, T. L. (1983) · Journal of Mechanisms, Transmissions, and Automation in Design · doi:10.1115/1.3258503 · Cited by 169 · cited 1×
“For example, game theory has formulated distributed optimization problems as games [249], [250].”
BackgroundProbabilistic online POMDP decision making for lane changes in fully automated driving
Ulbrich, S., & Maurer, M. (2013) · 16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013) · doi:10.1109/itsc.2013.6728533 · Cited by 168 · cited 1×
“POMDP was used to perform automated driving in urban traffic while dealing with sensor uncertainties [180].”
MethodMAA*: A Heuristic Search Algorithm for Solving Decentralized POMDPsPreprint
Szer, D., Charpillet, F., & Zilberstein, S. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1207.1359 · Cited by 167 · cited 2×
“Approximate solutions have been proposed based on bounded policy iteration [90], Q-value function methods [91], multiagent A∗ [81], genetic algorithms [82], DP [92]–[94], and a Bayesian learning, stick-breaking policy algorithm [95].”
MethodApproximate Solutions for Partially Observable Stochastic Games with Common Payoffs
Emery, R., Gordon, G., Schneider, J., & Thrun, S. (2004) · doi:10.1109/aamas.2004.67 · Cited by 167 · cited 1×
“observable Markovian environments [115], [117].”
Teamwork in Self-Organized Robot Colonies
Nouyan, S., Gross, R., Bonani, M., Mondada, F., & Dorigo, M. (2009) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2008.2011746 · Cited by 157 · cited 1×
“It has been applied to dynamic task allocation [161], distributed localization problems [162], foraging tasks [163]–[165], collision free navigation [166], [167], and communication free flocking with minimal memory requirements [168].”
Wu, C., Ohzahata, S., & Kato, T. (2013) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2013.2273945 · Cited by 153
Sutton, R. S., Maei, H. R., & Szepesvári, C. (2008) · neural information processing systems · Cited by 152
Transport modeling by multi-agent systems: a swarm intelligence approach
Teodorović, D. (2003) · Transportation Planning and Technology · doi:10.1080/0308106032000154593 · Cited by 145
Bounded policy iteration for decentralized POMDPs
Bernstein, D. S., Hansen, E. A., & Zilberstein, S. (2005) · Cited by 143
Influence Diagram Retrospective
Howard, R. A., & Matheson, J. E. (2005) · Decision Analysis · doi:10.1287/deca.1050.0050 · Cited by 140
Multiagent Reinforcement Learning for Multi-Robot Systems: A Survey
Yang, E., & Gu, D. (2004) · Cited by 139
Editorial: Advances in Multi-Robot Systems
Arai, T., Pagello, E., & Parker, L. E. (2002) · Cited by 135
Kuang, F., Zhang, S., Jin, Z., & Xu, W. (2014) · Soft Computing · doi:10.1007/s00500-014-1332-7 · Cited by 134 · cited 1×
“PSO has been widely applied in combination with support vector machines [225]–[227] and linear programming [228].”
MethodKang, Q., Zhou, M., An, J., & Wu, Q. (2012) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2012.2204980 · Cited by 134 · cited 1×
“POSG [237], POMDP [238], [239], multiagent RL [240], [241], and swarm intelligence [28], [242]–[244]”
BackgroundMemory-bounded dynamic programming for DEC-POMDPs
Seuken, S., & Zilberstein, S. (2007) · Cited by 132
Monte Carlo Value Iteration for Continuous-State POMDPs
Bai, H., Hsu, D., Lee, W. S., & Vien, N. A. (2010) · Springer tracts in advanced robotics · doi:10.1007/978-3-642-17452-0_11 · Cited by 125 · cited 1×
“nonlinear optimization [75], quadratically constrained linear programming [76], Monte Carlo methods [77], [78], and DP [79], [80] when state and transition models are known.”
BackgroundMulti-agent reinforcement learning for traffic signal control
Prabuchandran, K. J., Hemanth Kumar, A. N., & Bhatnagar, S. (2014) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2014.6958095 · Cited by 122
UM-PRS: An implementation of the procedural reasoning system for multirobot applications
Lee, J., Huber, M. J., Durfee, E. H., & Kenny, P. G. (1994) · NASA STI Repository (National Aeronautics and Space Administration) · Cited by 116
A novel multi-agent reinforcement learning approach for job scheduling in Grid computing
Wu, J., Xu, X., Zhang, P., & Liu, C. (2010) · Future Generation Computer Systems · doi:10.1016/j.future.2010.10.009 · Cited by 115 · cited 1×
“Adopted models include POMDP, used to minimize network bandwidth congestion and fairly allocate resource [245], Bayesian games [246] and stochastic games with multiagent RL used for job, and resource scheduling in grid computing [247].”
MethodZhang, Y. M., & Mehrjerdi, H. (2013) · International Conference on Unmanned Aircraft Systems (ICUAS) · doi:10.1109/icuas.2013.6564798 · Cited by 108 · cited 1×
“The following surveys discussed multiple models for one application such as formation control and coordination [21], [22], task allocation [23], [24], intrusion detection [25]–[27], and smart electric grids [28].”
BackgroundDucatelle, F., Di, G. A., & Gambardella, L. M. (2010) · Swarm Intelligence · doi:10.1007/s11721-010-0040-x · Cited by 108
A reinforcement learning approach to autonomous decision-making in smart electricity markets
Peters, M., Ketter, W., Saar‐Tsechansky, M., & Collins, J. E. (2013) · Machine Learning · doi:10.1007/s10994-013-5340-0 · Cited by 106 · cited 1×
“POSG [237], POMDP [238], [239], multiagent RL [240], [241], and swarm intelligence [28], [242]–[244]”
BackgroundArtificial bee colony algorithm for dynamic deployment of wireless sensor networks
Öztürk, C., Karaboğa, D., & Görkemli, B. (2012) · TURKISH JOURNAL OF ELECTRICAL ENGINEERING & COMPUTER SCIENCES · doi:10.3906/elk-1101-1030 · Cited by 104 · cited 1×
“Swarm intelligence has been used for routing in WSN [208]–[214], clustering [215], cluster head”
MethodFundamentals of Multicriteria Optimization
Stadler, W. (1988) · Springer eBooks · doi:10.1007/978-1-4899-3734-6_1 · Cited by 100 · cited 1×
“For example, game theory has formulated distributed optimization problems as games [249], [250].”
BackgroundA New Game Theoretical Resource Allocation Algorithm for Cloud Computing
Teng, F., & Magoulès, F. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-13067-0_35 · Cited by 99 · cited 1×
“Adopted models include POMDP, used to minimize network bandwidth congestion and fairly allocate resource [245], Bayesian games [246] and stochastic games with multiagent RL used for job, and resource scheduling in grid computing [247].”
MethodPlanning for robotic exploration based on forward simulation
Lauri, M., & Ritala, R. (2016) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2016.06.008 · Cited by 98 · cited 1×
“Forward simulation was used to estimate policy utilities [89].”
MethodExact Solutions to Time-Dependent MDPs
Boyan, J. A., & Littman, M. L. (2000) · Cited by 97
Coordinated Multi-Agent Reinforcement Learning in Networked Distributed POMDPs
Zhang, C., & Lesser, V. (2011) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v25i1.7886 · Cited by 97 · cited 1×
“Algorithms used to find policies for this model include multiagent RL [101], DP [102], and distributed constrained optimization [100].”
MethodImproved Memory-Bounded Dynamic Programming for Decentralized POMDPsPreprint
Seuken, S., & Zilberstein, S. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.5295 · Cited by 96 · cited 1×
“nonlinear optimization [75], quadratically constrained linear programming [76], Monte Carlo methods [77], [78], and DP [79], [80] when state and transition models are known.”
BackgroundMonte Carlo Sampling Methods for Approximating Interactive POMDPs
Doshi, P., & Gmytrasiewicz, P. J. (2009) · Journal of Artificial Intelligence Research · doi:10.1613/jair.2630 · Cited by 94 · cited 1×
“PSPACE-complete [98] and approximate solutions have been proposed based on particle filters [104], value iteration [105], and Monte Carlo sampling methods [106].”
MethodPlanning for decentralized control of multiple robots under uncertainty
Amato, C., Konidaris, G., Cruz, G., Maynor, C. A., How, J. P., & Kaelbling, L. P. (2015) · 2015 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2015.7139350 · Cited by 91
Consensus Control for a System of Underwater Swarm Robots
Joordens, M., & Jamshidi, M. M. (2010) · IEEE Systems Journal · doi:10.1109/jsyst.2010.2040225 · Cited by 91
Biologically Inspired Cooperative Routing for Wireless Mobile Sensor Networks
Iyengar, S. S., Wu, H., Balakrishnan, N., & Chang, S. Y. (2007) · IEEE Systems Journal · doi:10.1109/jsyst.2007.903101 · Cited by 90 · cited 9×
“WSN are a collection of autonomous computing and sensing devices with limited computational resources.”
MethodInfluentialPolicy Iteration for Decentralized Control of Markov Decision Processes
Bernstein, D. S., Amato, C., Hansen, E. A., & Zilberstein, S. (2009) · Journal of Artificial Intelligence Research · doi:10.1613/jair.2667 · Cited by 88 · cited 1×
“learning [48]–[50], policy search [51], and linear programming [52], [53] require models of the state transition and reward functions.”
BackgroundEvolino: hybrid neuroevolution / optimal linear search for sequence learning
Schmidhuber, J., Wierstra, D., & Gomez, F. J. (2005) · International Joint Conference on Artificial Intelligence · Cited by 87
Multi-agent role allocation: issues, approaches, and multiple perspectives
Campbell, A., & Wu, A. S. (2010) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-010-9127-4 · Cited by 87 · cited 1×
“The following surveys discussed multiple models for one application such as formation control and coordination [21], [22], task allocation [23], [24], intrusion detection [25]–[27], and smart electric grids [28].”
BackgroundGame Theoretic Control for Robot Teams
Emery, R., Gordon, G. J., Schneider, J., & Thrun, S. (2006) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2005.1570273 · Cited by 85 · cited 1×
“POSG has been applied to multiple problems in robotics [148], [149].”
MethodSpectrum management of cognitive radio using multi-agent reinforcement learning
Wu, C., Chowdhury, K. R., Felice, M. D., & Meleis, W. M. (2010) · doi:10.5555/1838194.1838199 · Cited by 85
Cao, B., Zhao, J., Lv, Z., Liu, X., Yang, S., Kang, X., & Kang, K. (2017) · IEEE Access · doi:10.1109/access.2017.2702561 · Cited by 81 · cited 1×
“Also, swarm intelligence including PSO [252], [253] and an algorithm that mimics bacterial foraging [254] have been used to solve multiobjective optimization problems.”
MethodA Stochastic Game Model for Jamming in Multi-Channel Cognitive Radio Systems
Zhu, Q., Li, H., Han, Z., & Başar, T. (2010) · International Conference on Communications · doi:10.1109/icc.2010.5502451 · Cited by 78 · cited 1×
“were modeled using stochastic games [235] and spectrum sharing was modeled using game theory [236].”
MethodOptimal Sleep-Wake Scheduling for Quickest Intrusion Detection Using Wireless Sensor Networks
Premkumar, K., & Kumar, A. (2008) · doi:10.1109/infocom.2008.198 · Cited by 75 · cited 1×
“MDP identified the network’s most vulnerable nodes based on attackers’ previous behaviors [221], [222].”
BackgroundEl-Tantawy, S., & Abdulhai, B. (2012) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2012.6338707 · Cited by 74
Sambot: A self-assembly modular robot for swarm robot
Wei, H., Cai, Y., Li, H., Li, D., & Wang, T. (2010) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2010.5509214 · Cited by 74 · cited 1×
“It has been applied to dynamic task allocation [161], distributed localization problems [162], foraging tasks [163]–[165], collision free navigation [166], [167], and communication free flocking with minimal memory requirements [168].”
Multiagent -Learning for Aloha-Like Spectrum Access in Cognitive Radio Systems
Li, H. (2010) · EURASIP Journal on Wireless Communications and Networking · doi:10.1155/2010/876216 · Cited by 72
Multiobjective Reinforcement Learning for Traffic Signal Control Using Vehicular Ad Hoc Network
Houli, D., Zhiheng, L., & Yi, Z. (2010) · EURASIP Journal on Advances in Signal Processing · doi:10.1155/2010/724035 · Cited by 72
A New Intrusion Detection Approach using PSO based Multiple Criteria Linear Programming
Hosseini, B. S. M., Amiri, B., Mirzabagheri, M., & Shi, Y. (2015) · Procedia Computer Science · doi:10.1016/j.procs.2015.07.040 · Cited by 72
Sequentially optimal repeated coalition formation under uncertainty
Chalkiadakis, G., & Boutilier, C. E. (2010) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-010-9157-y · Cited by 72
A survey on swarm intelligence based routing protocols in wireless sensor networks
Çelik, F., Zengi̇n, A., Tüncel, S., & Çobanoğlu, B. (2010) · Sakarya Üniversitesi Kurumsal Açık Akademik Arşivi · doi:10.5897/ijps.9000041 · Cited by 71
Coalition-Based Cooperative Packet Delivery under Uncertainty: A Dynamic Bayesian Coalitional Game
Akkarajitsakul, K., Hossain, E., & Niyato, D. T. (2011) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2011.251 · Cited by 68 · cited 1×
“Coalition games were generalized to problems with incomplete information through Bayesian games [177], [178].”
BackgroundManvi, S. S., Kakkasageri, M. S., & Mahapurush, C. V. (2009) · International Conference on Future Computer and Communication · doi:10.1109/icfcc.2009.122 · Cited by 68
Co-evolving recurrent neurons learn deep memory POMDPs
Gomez, F. J., & Schmidhuber, J. (2005) · Proceedings of the 7th annual conference on Genetic and evolutionary computation · doi:10.1145/1068009.1068092 · Cited by 67 · cited 3×
“These include RL, dynamic programming (DP), recurrent neural networks (RNNs), and evolutionary computing, to name a few.”
MethodEvolutionary Computation for Reinforcement Learning
Whiteson, S. (2012) · Adaptation, learning, and optimization · doi:10.1007/978-3-642-27645-3_10 · Cited by 67 · cited 1×
“approximate methods are adopted and include model-free RL approaches like Q-learning [54] and SARSA [55], evolutionary computing [56], [57], RNN [58]–[63], and deep RL [64], [65].”
MethodEfficient solutions to factored MDPs with imprecise transition probabilities
Delgado, K. V., Sanner, S., & de Barros, L. N. (2011) · Artificial Intelligence · doi:10.1016/j.artint.2011.01.001 · Cited by 66
Multiagent Reinforcement Learning Based Spectrum Sensing Policies for Cognitive Radio Networks
Lundén, J., Kulkarni, S. R., Koivunen, V., & Poor, H. V. (2013) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2013.2259797 · Cited by 65 · cited 1×
“and management problem [230]–[234].”
A differential game approach to distributed demand side management in smart grid
Zhu, Q., Han, Z., & Basar, T. (2012) · 2012 IEEE International Conference on Communications (ICC) · doi:10.1109/icc.2012.6364562 · Cited by 65 · cited 10×
“POSG modeled directional routing and scheduling of packet delivery in vehicular ad hoc networks [179].”
BackgroundInfluentialGal, Y., & Pfeffer, A. (2008) · Journal of Artificial Intelligence Research · doi:10.1613/jair.2503 · Cited by 63 · cited 1×
“A network of IDs (NIDs) is built on top of MAIDs to account for uncertainties in other agents’ decision making and hierarchy of beliefs [135].”
BackgroundKiekintveld, C. D., Marecki, J., & Tambe, M. (2011) · Adaptive Agents and Multi-Agents Systems · doi:10.5555/2034396.2034412 · Cited by 63
Policy search for multi-robot coordination under uncertainty
Amato, C., Konidaris, G., Anders, A. S., Cruz, G., How, J. P., & Kaelbling, L. P. (2016) · The International Journal of Robotics Research · doi:10.1177/0278364916679611 · Cited by 62
A Minimalist Flocking Algorithm for Swarm Robots
Möslinger, C., Schmickl, T., & Crailsheim, K. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-21314-4_47 · Cited by 60 · cited 1×
“It has been applied to dynamic task allocation [161], distributed localization problems [162], foraging tasks [163]–[165], collision free navigation [166], [167], and communication free flocking with minimal memory requirements [168].”
MethodPoint-based dynamic programming for DEC-POMDPs
Szer, D., & Charpillet, F. (2006) · Cited by 57
Brechtel, S., Gindele, T., & Dillmann, R. (2013) · International Conference on Machine Learning · Cited by 53
Distributed efficient localization in swarm robotic systems using swarm intelligence algorithms
de Sá, A. O., Nedjah, N., & Mourelle, L. D. M. (2015) · Neurocomputing · doi:10.1016/j.neucom.2015.03.099 · Cited by 53 · cited 1×
“It has been applied to dynamic task allocation [161], distributed localization problems [162], foraging tasks [163]–[165], collision free navigation [166], [167], and communication free flocking with minimal memory requirements [168].”
Applications of Reinforcement Learning to Cognitive Radio Networks
Yau, K. A., Komisarczuk, P., & Teal, P. D. (2010) · IEEE International Conference on Communications workshops · doi:10.1109/iccw.2010.5503970 · Cited by 51
A Linear Programming Approach to Nonstationary Infinite-Horizon Markov Decision Processes
Ghate, A., & Smith, R. L. (2013) · Operations Research · doi:10.1287/opre.1120.1121 · Cited by 50 · cited 1×
“learning [48]–[50], policy search [51], and linear programming [52], [53] require models of the state transition and reward functions.”
BackgroundFinite Linear Programming Approximations of Constrained Discounted Markov Decision Processes
Dufour, F., & Prieto-Rumeau, T. (2013) · SIAM Journal on Control and Optimization · doi:10.1137/120867925 · Cited by 50 · cited 1×
“learning [48]–[50], policy search [51], and linear programming [52], [53] require models of the state transition and reward functions.”
BackgroundAdaptive collective decision-making in limited robot swarms without communication
Kernbach, S., Häbe, D., Kernbach, O., Thenius, R., Radspieler, G., Kimura, T., & Schmickl, T. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364912468636 · Cited by 49 · cited 1×
“limited communication capabilities in spatial behavior problems [171].”
Backgrounddos Santos, D. S., & Bazzan, A. L. C. (2012) · Applied Soft Computing · doi:10.1016/j.asoc.2012.03.016 · Cited by 48 · cited 1×
“Swarm intelligence was used to search for the best coalitions to form [176].”
MethodMulti-agent approach to distributed ant colony optimization
Ilie, S., & Bădică, C. (2011) · Science of Computer Programming · doi:10.1016/j.scico.2011.09.001 · Cited by 48 · cited 3×
“ACO was used in IP traceback problems [229].”
MethodOptimizing Memory-Bounded Controllers for Decentralized POMDPs
Amato, C., Bernstein, D. S., & Shlomo Zilberstein, B. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.5258 · Cited by 48 · cited 1×
“nonlinear optimization [75], quadratically constrained linear programming [76], Monte Carlo methods [77], [78], and DP [79], [80] when state and transition models are known.”
BackgroundCompeting with Humans at Fantasy Football: Team Formation in Large Partially-Observable Domains
Matthews, T., Ramchurn, S. D., & Chalkiadakis, G. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v26i1.8259 · Cited by 47
Hierarchical Routing in Traffic Using Swarm-Intelligence
Tatomir, B., & Rothkrantz, L. J. M. (2006) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2006.1706747 · Cited by 47 · cited 1×
“utilized swarm intelligence to control traffic light scheduling [194], model complex transportation systems [195], and develop routing protocols for vehicles [196], [197] and for”
MethodSolving POMDPs using quadratically constrained linear programs
Amato, C., Bernstein, D. S., & Zilberstein, S. (2006) · Adaptive Agents and Multi-Agents Systems · doi:10.1145/1160633.1160694 · Cited by 46 · cited 1×
“nonlinear optimization [75], quadratically constrained linear programming [76], Monte Carlo methods [77], [78], and DP [79], [80] when state and transition models are known.”
BackgroundA Survey on Application of Swarm Intelligence Computation to Electric Power System
Bai, H., & Zhao, B. (2006) · World Congress on Intelligent Control and Automation · doi:10.1109/wcica.2006.1713441 · Cited by 44 · cited 2×
“POSG [237], POMDP [238], [239], multiagent RL [240], [241], and swarm intelligence [28], [242]–[244]”
BackgroundExtending the Applicability of POMDP Solutions to Robotic Tasks
Grady, D. K., Moll, M., & Kavraki, L. E. (2015) · IEEE Transactions on Robotics · doi:10.1109/tro.2015.2441511 · Cited by 43 · cited 1×
“MDPs [150] and POMDP [151]–[155] have been used for robotics coordination including robot soccer [156].”
MethodInteractive POMDPs: Properties and Preliminary Results
Gmytrasiewicz, P. J., & Doshi, P. (2004) · Adaptive Agents and Multi-Agents Systems · doi:10.1109/aamas.2004.154 · Cited by 43 · cited 1×
“a concurrent learning approach, generalizes POMDP to MAS by modeling other agents in the system while maintaining a belief of the system state [103].”
BackgroundGraph-based distributed cooperative navigation for a general multi-robot measurement model
Indelman, V., Gurfil, P., Rivlin, E., & Rotstein, H. (2012) · The International Journal of Robotics Research · doi:10.1177/0278364912446325 · Cited by 42 · cited 1×
“Other applications include formation control [146] and navigation [147] in MRS.”
BackgroundConstraint-based dynamic programming for decentralized POMDPs with structured interactions
Kumar, A., & Zilberstein, S. (2009) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/peqw2142 · Cited by 42
Ieong, S., & Shoham, Y. (2008) · National Conference on Artificial Intelligence · Cited by 41
Robot Path Planning using Swarm Intelligence: A Survey
SinghPal, N., & Sharma, S. (2013) · International Journal of Computer Applications · doi:10.5120/14498-2274 · Cited by 40 · cited 2×
“Finally, swarm intelligence has been applied to underwater environments [160], 3-D space [129], and robot path planning problems [30], [125].”
BackgroundMethodBiologically Inspired Approaches for Wireless Sensor Networks
Ren, H., & Meng, M. (2006) · International Conference on Mechatronics and Automation · doi:10.1109/icma.2006.257686 · Cited by 39 · cited 2×
“Swarm intelligence has been used for routing in WSN [208]–[214], clustering [215], cluster head”
BackgroundMethodNeuroevolutionary reinforcement learning for generalized helicopter control
Koppejan, R., & Whiteson, S. (2009) · Proceedings of the 11th Annual conference on Genetic and evolutionary computation · doi:10.1145/1569901.1569922 · Cited by 38
Pan, Y., Werner, H., Huang, Z., & Bartels, M. (2017) · Systems & Control Letters · doi:10.1016/j.sysconle.2017.06.002 · Cited by 38 · cited 1×
“A Lyapunov-based distributed lead-follower control system was developed that scaled to large MAS when the interaction topology is an undirected graph [139].”
MethodAn improved ant colony system algorithm for solving the IP traceback problem
Wang, P., Lin, H., & Wang, T.-S. (2015) · Information Sciences · doi:10.1016/j.ins.2015.07.006 · Cited by 37 · cited 1×
“ACO was used in IP traceback problems [229].”
MethodMulti-Agent Reinforcement Learning for Intrusion Detection: A Case Study and Evaluation
Servin, A., & Kudenko⋆, D. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-87805-6_15 · Cited by 37 · cited 1×
“Multiagent RL were used to implement distributed intrusion detection systems [224].”
MethodA survey on multi-agent reinforcement learning: Coordination problems
Choi, Y. C., & Ahn, H. (2010) · IEEE/ASME International Conference on Mechatronic and Embedded Systems and Applications · doi:10.1109/mesa.2010.5552089 · Cited by 36 · cited 1×
“MDPs [150] and POMDP [151]–[155] have been used for robotics coordination including robot soccer [156].”
MethodMonte-Carlo expectation maximization for decentralized POMDPs
Wu, F., Zilberstein, S., & Jennings, N. R. (2013) · ePrints Soton (University of Southampton) · Cited by 35
Generalized point based value iteration for interactive pomdps
Doshi, P., & Perez, D. (2008) · Cited by 35
Policy Search for Multi-Robot Coordination under Uncertainty
Amato, C., Konidaris, G., Anders, A. S., Cruz, G., How, J. P., & Kaelbling, L. P. (2015) · Robotics: Science and Systems · doi:10.15607/rss.2015.xi.007 · Cited by 34
Adaptive data collection protocol using reinforcement learning for VANETs
Soua, A., & Afifi, H. (2013) · International Wireless Communications and Mobile Computing Conference (IWCMC) · doi:10.1109/iwcmc.2013.6583700 · Cited by 34 · cited 1×
“used for routing algorithms [181]–[183], adaptive broadcasting [184], adaptive data collection [185], and traffic signal control [186]–[193].”
MethodEnsembles of Neural Networks for Robust Reinforcement Learning
Hans, A., & Udluft, S. (2010) · International Conference on Machine Learning and Applications · doi:10.1109/icmla.2010.66 · Cited by 34 · cited 1×
“approximate methods are adopted and include model-free RL approaches like Q-learning [54] and SARSA [55], evolutionary computing [56], [57], RNN [58]–[63], and deep RL [64], [65].”
MethodDistributed Cooperative Control of Multiple Nonholonomic Mobile Robots
Wang, G., Wang, C., Du, Q., Li, L., & Dong, W. (2016) · Journal of Intelligent & Robotic Systems · doi:10.1007/s10846-015-0316-x · Cited by 31 · cited 1×
“Other applications include formation control [146] and navigation [147] in MRS.”
BackgroundSmartGRID: A Fully Decentralized Grid Scheduling Framework Supported by Swarm Intelligence
Huang, Y., Brocco, A., Kuonen, P., Courant, M., & Hirsbrunner, B. (2008) · International Conference on Grid and Cooperative Computing (GCC) · doi:10.1109/gcc.2008.24 · Cited by 28 · cited 1×
“POSG [237], POMDP [238], [239], multiagent RL [240], [241], and swarm intelligence [28], [242]–[244]”
BackgroundPajarinen, J., Hottinen, A., & Peltonen, J. (2013) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2013.39 · Cited by 26 · cited 1×
“for performance optimization [204], data and memory access control [205], and sleep scheduling [206].”
Background
Research works printed but not linked: 11
- 2015, First Workshop on Cognitive Architectures
- altman, 1999, Constrained Markov Decision Processes, 7
- amato, 2013, Scalable Bayesian reinforcement learning for multiagent POMDPs, Multidisciplinary Conf on Reinforcement Learning and Decision Making, 188
- sandhu, 2004, Wireless sensor networks for commercial lighting control: Decision making with multi-agent systems, Proc of the AAAI-04 Workshop on Sensor Networks, 10, 131
- strikos, 2007, A Full Approach for Intrusion Detection in Wireless Sensor Networks
- melo, 2012, Decentralized multiagent planning for balance control in smart grids, Proc 1st Int Workshop Inf Technol Energy Appl, 923, 21
- 2016, An Internet of Things That Will Number Ten Billions
- 2015, Gartner Says 6 4 Billion Connected “Things” Will Be in Use in 2016 Up 30 Percent From 2015
- lauri, 2013, Managing power flows in microgrids using multi-agent reinforcement learning, Proc Agent Technol Energy Syst, 1
- emery, 2005, Game Theoretic Control for Robot Teams
- teodorovi?, 2005, Bee colony optimization—A cooperative learning approach to complex transportation problems, Proc 16th Mini Eur Conf 10th Meeting Adv OR AI Methods Transport (EWGT), 51
Cited by
Blockchain for AI: review and open research challenges
Salah, K., Rehman, M. H. U., Nizamuddin, N., & Al-Fuqaha, A. (2019) · IEEE Access · doi:10.1109/access.2018.2890507 · Cited by 1,001
“ing inter-layer and intralayer operability across entire systems [50].”
BackgroundCooperative Heterogeneous Multi-Robot Systems: A Survey
Rizk, Y., Awad, M., & Tunstel, E. W. (2020) · ACM Computing Surveys · doi:10.1145/3303848 · Cited by 449
Self-citationAgbehadji, I. E., Awuzie, B. O., Ngowi, A. B., & Millham, R. C. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17155330 · Cited by 248
Baccour, E., Mhaisen, N., Abdellatif, A. A., Erbad, A., Mohamed, A., Hamdi, M., & Guizani, M. (2022) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2022.3200740 · Cited by 186
“Other papers surveyed the single agent and multi-agents RL, such as [75], [76], [77], [78], [79].”
BackgroundTheoretical approaches to AI in supply chain optimization: Pathways to efficiency and resilience
Abaku, E. A., Edunjobi, T. E., & Odimarha, A. C. (2024) · International Journal of Science and Technology Research Archive · doi:10.53771/ijstra.2024.6.1.0033 · Cited by 188
“Multi-agent systems (MAS) represent another approach to decentralized decision-making and coordination, where autonomous agents interact with each other and their environment to achieve individual and collective objectives (Rizk et al ., 2018).”
Background
Metadata
- Authors
- Rizk, Y., Awad, M., & Tunstel, E. W.
- Affiliations
- American University of Beirut (LB)
- Venue
- IEEE Transactions on Cognitive and Developmental Systems
- Published
- 2018-09
- Volume
- 10
- Issue
- 3
- Pages
- 514-529
- ISSN
- 2379-8920 (ISSN-L)
- 2379-8939
- OpenAlex
- W2803208334
- Cited by
- 219
- References
- 275
- Citations by year
- 2026: 18 · 2025: 29 · 2024: 34 · 2023: 39 · 2022: 35 · 2021: 29 · 2020: 21 · 2019: 13 · 2018: 1
- FWCI
- 8.96
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Cognitive and Developmental Systems (publishedVersion)
- Funders
- National Council of Scientific Research in Lebanon
- American University of Beirut
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Multi-Agent Systems and Negotiation
- Physical Sciences › Engineering › Control and Systems Engineering › Distributed Control Multi-Agent Systems
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Keywords
- multi-agent systems, cooperative decision-making, Markov decision process, game theory, swarm intelligence, graph model, reinforcement learning, dynamic programming, evolutionary computation, artificial neural networks, robotics, wireless sensor networks, cognitive radio networks, intelligent transportation systems, big data, evaluation metrics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.