Evolution of a Complex Predator-Prey Ecosystem on Large-scale Multi-Agent Deep Reinforcement Learning
Open accessAbstract
Simulation of population dynamics is a central research theme in computational biology, which contributes to understanding the interactions between predators and preys. Conventional mathematical tools of this theme, however, are incapable of accounting for several important attributes of such systems, such as the intelligent and adaptive behavior exhibited by individual agents. This unrealistic setting is often insufficient to simulate properties of population dynamics found in the real-world. In this work, we leverage multi-agent deep reinforcement learning, and we propose a new model of large-scale predator-prey ecosystems. Using different variants of our proposed environment, we show that multi-agent simulations can exhibit key real-world dynamical properties. To obtain this behavior, we firstly define a mating mechanism such that existing agents reproduce new individuals bound by the conditions of the environment. Furthermore, we incorporate a real-time evolutionary algorithm and show that reinforcement learning enhances the evolution of the agents' physical properties such as speed, attack and resilience against attacks.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the multi-agent setting, reinforcement learning has been used to study animal behaviour in areas including predatory-prey dynamics and collective/swarm movement.
“In the multi-agent setting, it has been used to study animal behaviour in areas including predatory-prey dynamics [43,44] and collective/swarm movement [45,46].”
From Maynard Smith revisited: A multi-agent reinforcement learning approach to the coevolution of signalling behaviour (2025) · cites this paper as [44]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · see citing works
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612 · see citing works
Playing Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471 · see citing works
Behavioral decisions made under the risk of predation: a review and prospectus
Lima, S. L., & Dill, L. M. (1990) · Canadian Journal of Zoology · doi:10.1139/z90-092 · Cited by 8,196 · see citing works
Introduction to Reinforcement Learning
Sutton, R. S., & Barto, A. G. (1998) · The MIT Press eBooks · Cited by 6,855 · see citing works
Deep Reinforcement Learning with Double Q-Learning
Van Hasselt, H., Guez, A., & Silver, D. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10295 · Cited by 3,318 · see citing works
Dueling Network Architectures for Deep Reinforcement Learning
Wang, Z., Schaul, T., Hessel, M., van Hasselt, H., Lanctot, M., & de Freitas, N. (2015) · arXiv · doi:10.48550/arxiv.1511.06581 · Cited by 3,300 · see citing works
Evolution strategies – A comprehensive introduction
Beyer, H.-G., & Schwefel, H. (2002) · Natural Computing · doi:10.1023/a:1015059928466 · Cited by 2,810 · see citing works
Carmichael, R. D., & Lotka, A. J. (1926) · American Mathematical Monthly · doi:10.2307/2298330 · Cited by 2,784 · see citing works
Temporal Variation in Danger Drives Antipredator Behavior: The Predation Risk Allocation Hypothesis
Lima, S. L., & Bednekoff, P. A. (1999) · The American Naturalist · doi:10.1086/303202 · Cited by 1,488 · see citing works
Nguyen, T. T., Nguyen, N. D., & Nahavandi, S. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2020.2977374 · Cited by 1,306 · see citing works
Deep Recurrent Q-Learning for Partially Observable MDPsPreprint
Hausknecht, M., & Stone, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.06527 · Cited by 1,224 · see citing works
Multiagent cooperation and competition with deep reinforcement learning
Tampuu, A., Matiisen, T., Kodelja, D., Kuzovkin, I., Korjus, K., Aru, J., Aru, J., & Vicente, R. (2017) · PLoS ONE · doi:10.1371/journal.pone.0172395 · Cited by 901 · see citing works
Threat-sensitive predator avoidance in damselfish-trumpetfish interactions
Helfman, G. S. (1989) · Behavioral Ecology and Sociobiology · doi:10.1007/bf00300117 · Cited by 639 · see citing works
Principles of the Self-Organizing System
Ashby, W. R. (1991) · Springer eBooks · doi:10.1007/978-1-4899-0718-9_38 · Cited by 619 · see citing works
Decision theory, reinforcement learning, and the brain
Dayan, P., & Daw, N. D. (2008) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.8.4.429 · Cited by 580 · see citing works
Genetic Algorithms: Principles and Perspectives: A Guide to GA Theory
Reeves, C. R., & Rowe, J. E. (2002) · Thammasat University Digital Collections · Cited by 561 · see citing works
Turchin, P. (2013) · Princeton University Press eBooks · doi:10.1515/9781400847280 · Cited by 483 · see citing works
Incremental Evolution of Complex General Behavior
Gomez, F. J., & Miikkulainen, R. P. (1997) · Adaptive Behavior · doi:10.1177/105971239700500305 · Cited by 428 · see citing works
Genetic Algorithms—Principles and Perspectives
Reeves, C. R., & Rowe, J. E. (2002) · Operations research, computer science. Interface series · doi:10.1007/b101880 · Cited by 392 · see citing works
(1999) · doi:10.2307/1313476 · Cited by 309 · see citing works
Review of Deep Reinforcement Learning for Robot Manipulation
Nguyen, V., & La, H. M. (2019) · IEEE International Conference on Robotic Computing (IRC) · doi:10.1109/irc.2019.00120 · Cited by 275 · see citing works
Emergent Complexity via Multi-Agent CompetitionPreprint
Bansal, T., Pachocki, J., Sidor, S., Sutskever, I., & Mordatch, I. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.03748 · Cited by 266 · see citing works
MAgent: A Many-Agent Reinforcement Learning Platform for Artificial Collective Intelligence
Zheng, L., Yang, J., Cai, H., Zhou, M. Q., Zhang, W., Wang, Y.-X., & Yu, Y. H. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11371 · Cited by 160 · see citing works
Quasicycles in a spatial predator-prey model
Lugo, C. A., & McKane, A. J. (2008) · Physical Review E · doi:10.1103/physreve.78.051911 · Cited by 116 · see citing works
Deep-learning in Mobile Robotics - from Perception to Control Systems: A Survey on Why and Why not.Preprint
Tai, L., & Liu, M. (2016) · arXiv (Cornell University) · Cited by 59 · see citing works
Pineda‐Krch, M., Blok, H. J., Dieckmann, U., & Doebeli, M. (2006) · Oikos · doi:10.1111/j.2006.0030-1299.14940.x · Cited by 59 · see citing works
Fleischer, S. R., terHorst, C. P., & Li, J. (2018) · Journal of Theoretical Biology · doi:10.1016/j.jtbi.2018.08.013 · Cited by 22 · see citing works
Deep-Reinforcement Learning-Based Co-Evolution in a Predator–Prey System
Wang, X., Cheng, J. S., & Wang, L. (2019) · Entropy · doi:10.3390/e21080773 · Cited by 11 · see citing works
Co-evolution in predator prey through reinforcement learning
Olsen, M., & Fraczkowski, R. (2015) · Journal of Computational Science · doi:10.1016/j.jocs.2015.04.011 · Cited by 10 · see citing works
An Empirical Study of AI Population Dynamics with Million-agent Reinforcement Learning.
Yang, Y., Yu, L., Bai, Y., Wang, J., Zhang, W., Wen, Y., & Yu, Y. H. (2017) · Cited by 6 · see citing works
Competition Between Reinforcement Learning Methods in a Predator-Prey Grid World
Schrum, J. (2008) · Cited by 2 · see citing works
Modelling Fluctuating Populations.R. M. Nisbet , W. S. C. Gurney
Ginzburg, L. R. (1983) · The Quarterly Review of Biology · doi:10.1086/413488 · Cited by 1 · see citing works
Flagged first, then by citations
4 of the 37 references couldn’t be matched to a known paper.
Research works printed but not linked: 2
- J. Endler, "Defense against predators," Defense Against Predators, pp. 109-134, 01 1986.
- A. Tsoularis, "Reinforcement learning in predator-prey interactions," WSEAS Transactions on Biology and Biomedicine, vol. 2, pp. 141-146, April 2005.
Cited by
Sorted by citations
A survey on multi-agent reinforcement learning and its application
Ning, Z., & Xie, L.-H. (2024) · Journal of Automation and Intelligence · doi:10.1016/j.jai.2024.02.003 · Cited by 257 · see citing works
Animal behavior analysis methods using deep learning: A survey
Fazzari, E., Romano, D., Falchi, F., & Stefanini, C. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.128330 · Cited by 71 · see citing works
Metacognition as a Consequence of Competing Evolutionary Time Scales
Kuchling, F., Fields, C., & Levin, M. G. (2022) · Entropy · doi:10.3390/e24050601 · Cited by 29 · see citing works
Artificial intelligence-informed planning for the rapid response of hazard-impacted road networks
Sun, L., Shawe-Taylor, J. S., & D'Ayala, D. F. (2022) · Scientific Reports · doi:10.1038/s41598-022-19637-z · Cited by 9 · see citing works
Self-citationEcosystem Models Based on Artificial Intelligence
Strannegård, C., Engsner, N., Eisfeldt, J., Endler, J. A., Hansson, A., Lindgren, R., Mostad, P., Olsson, S., Perini, I., Reese, H., et al. (2022) · Swedish Artificial Intelligence Society Workshop (SAIS) · doi:10.1109/sais55783.2022.9833026 · Cited by 8 · see citing works
Strannegård, C., Engsner, N., Ferrari, P., Glimmerfors, H., Södergren, M. H., Karlsson, T. L., Kleve, B., & Skoglund, V. (2022) · Lecture notes in computer science · doi:10.1007/978-3-030-93758-4_28 · Cited by 3 · see citing works
AI Tool for Exploring How Economic Activities Impact Local Ecosystems
Strannegård, C., Engsner, N., Lindgren, R., Olsson, S., & Endler, J. A. (2024) · Lecture notes in networks and systems · doi:10.1007/978-3-031-47718-8_44 · Cited by 3 · see citing works
Initial Conditions Sensitivity Analysis of a Two-Species Butterfly-Effect Agent-Based Model
Berceanu, C., & Pătraşcu, M. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20614-6_4 · Cited by 2 · see citing works
Macmillan-Scott, O., & Musolesi, M. (2025) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1013302 · Cited by 1 · see citing works
Flock response to sustained asynchronous predator attacksPreprint
Mohapatra, S., & Mahapatra, P. S. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.11.14.567144
Predicting Ecosystem Resilience Using Multi-Agent Reinforcement LearningPreprint
Strannegård, C., Palak, M., Engsner, N., Stocco, A., Antonelli, A., & Silvestro, D. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.06.07.658424
- Authors
- Yamada, J., Shawe-Taylor, J., & Fountas, Z.
- Affiliations
- University College London (GB)
- Venue
- 2020 International Joint Conference on Neural Networks (IJCNN)
- Publisher
- IEEE
- Published
- 2020-07
- Pages
- 1-8
- ISSN
- 2161-4407 (ISSN-L)
- OpenAlex
- W3004630518
- Cited by
- 11
- References
- 37
- Citations by year
- 2025: 3 · 2024: 2 · 2023: 1 · 2022: 5
- FWCI
- 8.80
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- International Joint Conference on Neural Networks (IJCNN) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- UCL Discovery (University College London) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Mathematical and Theoretical Epidemiology and Ecology Models
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Simulation Techniques and Applications
- Keywords
- population dynamics, multi-agent deep reinforcement learning, agent-based modeling, adaptive behavior, agent evolution
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
