The DOI matches its Crossref record; 113 of the 113 references it cites were found in a registry or an index; 16 of the 49 sources its claims cite were checked for support.
Emergent behaviour and neural dynamics in artificial agents tracking odour plumes
Open accessAbstract
Tracking an odour plume to locate its source under variable wind and plume statistics is a complex task. Flying insects routinely accomplish such tracking, often over long distances, in pursuit of food or mates. Several aspects of this remarkable behaviour and its underlying neural circuitry have been studied experimentally. Here we take a complementary in silico approach to develop an integrated understanding of their behaviour and neural computations. Specifically, we train artificial recurrent neural network agents using deep reinforcement learning to locate the source of simulated odour plumes that mimic features of plumes in a turbulent flow. Interestingly, the agents’ emergent behaviours resemble those of flying insects, and the recurrent neural networks learn to compute task-relevant variables with distinct dynamic structures in population activity. Our analyses put forward a testable behavioural hypothesis for tracking plumes in changing wind direction, and we provide key intuitions for memory requirements and neural dynamics in odour plume tracking.
Overview
CoChat Summary
TL;DR
The paper trains recurrent neural network agents with deep reinforcement learning to track simulated turbulent odour plumes and finds emergent behaviours and neural representations resembling those of flying insects. It reports that agents track the plume centreline rather than current wind direction, use short-timescale memory for constant wind but longer memory for switching winds, and exhibit low-dimensional population dynamics organized into distinct regimes.
Key findings
- DRL-trained RNN agents successfully localize plume sources across constant, switch-once, switch-many, sparse and sparser plume configurations; their behaviour decomposes into three modules—tracking, lost and recovering—based on time since last odour encounter (tracking encounters under 0.5 s; recovering about 0.5 s; lost typically over 1 s).
- Agents track the plume centreline rather than the current wind direction: course-direction distributions align better with the centreline than with wind, and in switch-once plumes the peak is closer to ±180° relative to centreline versus ≈45° relative to wind; this trend holds across all five RNN agents.
- RNN population activity is low dimensional, with the first five to eight principal components explaining 90% of variance in the 64-dimensional population activity (Fig. 4g shows first five PCs). The networks represent head direction, time since last odour, an exponentially weighted moving average of odour concentration (window ≈0.3 s, R² = 0.91) and of odour encounters (window ≈1.9 s, R² = 0.59); classifiers using these represented features plus instantaneous sensory inputs are 10–18% more accurate than instantaneous-only classifiers and 26–51% more accurate than a majority-class classifier, with time since last encounter always among the top two most important features.
- Neural dynamics organize into overlapping regimes rather than fixed points: tracking shows funnel-like structures, lost shows quasiperiodic limit cycles, and recovering shows an amorphous transition region; the same approximate structures emerge in four of five RNN agents. Training produces unstable eigenvalues outside the unit circle, with at least one strictly real eigenvalue larger than unity for all five agents; the bulk of stimulus integration timescales lie within about 12 time steps (≈0.5 s), and RNNs outperform feedforward MLPs on every plume tracking task, with the largest gains on switching-wind and sparse tasks.
Methods
- Plume simulation: a particle-based two-dimensional model with a spatially homogeneous wind field (0.5 m s⁻¹, configurable direction), an odour source at the origin emitting puffs as a Poisson process, radial diffusion, downwind advection and random cross-wind perturbation. The arena spans [−2 m, +10 m] × [−5 m, +5 m] (120 m²), simulated at 100 iterations per second. Wind configurations include constant, switch-once (one 45° anticlockwise switch), switch-many (switches about every 3 s), sparse (0.4-fold puff birth rate) and sparser (0.5-fold radial diffusion plus sparse).
- Agents and training: actor-critic networks with a 64-unit tanh RNN receiving three-dimensional egocentric inputs (wind x, y and odour concentration) and producing continuous turn (max ±6.25π rad s⁻¹) and forward-movement (max 2.5 m s⁻¹) actions; actor and critic heads are two-layer MLPs. Agents are trained with PPO; 14 independently initialized networks are trained per architecture (RNNs and MLPs with 2, 4, 6, 8, 10 or 12 time steps of sensory history). Evaluation uses 240 episodes per agent (15 initial locations, two initial simulation timestamps, eight initial head directions) across constant, switch-once and switch-many configurations; only the top five seeds per architecture are analysed.
- Analyses: PCA on RNN population activity; random forest classifiers predicting discretized actions, with permutation importance scores over N = 30 randomized permutations; linear regression to determine moving-average window sizes for odour concentration and encounters; eigenvalue spectra of the recurrence matrix and stimulus integration timescales from the recurrence Jacobian; and performance comparison between RNNs and feedforward MLPs.
Limitations
- Stated: the plume simulator is computationally efficient but approximate and does not capture some real-plume aspects such as filamentous nature or variation of whiff duration and frequency with distance from source. The study uses vanilla recurrent units with no biomechanical body model, no spiking neurons and no biologically inspired connectivity constraints; future work could add spiking networks, connectome-derived constraints, multiple antennae, biomechanical bodies, multitask training, other sensory modalities and 3D virtual reality.
- Stated: most results describe one agent chosen at random from the top five performers of 14 trained agents, with some analyses across five selected agents; notable performance variability was observed. Behaviour-module thresholds were determined by visual inspection, and classifier-based importance estimates are approximate due to action discretization and correlations between features.
- Stated: current reverse-engineering methods that characterize RNNs using discrete dynamical features such as fixed points are not applicable to the continuous and amorphous dynamical structures encountered here; new methods are needed for comparing multiple agents at the behavioural level and for understanding training-induced unstable RNN connectivity eigenmodes.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
12 Valid · 1 Not valid · 18 Not checked · 104 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“However, most such studies are limited to one or two levels of analysis, such as behaviour 6 , computation 7,8 or neural implementation 9 .”
What the sources say
- Not checked — no full text of the sourcecites [6] van Breugel, F., Regan, W. & Lipson, H. From insects to machines. IEEE Robot. Autom. Mag. 15, 68–74 (2008). source
- Not checked — no full text of the sourcecites [7] Lochmatter T. & Martinoli, A. Theoretical analysis of three bio-inspired plume tracking algorithms. In 2009 IEEE International Conference on Robotics and Automation 2661–2668 (IEEE, 2009). source
- Supportedcites [8] Pang, R., van Breugel, F., Dickinson, M., Riffell, J. A. & Fairhall, A. History dependence in insect flight decisions during odor tracking. PLoS Comput. Biol. 14, e1005969 (2018). sourceExperimental — accuracy not yet measured
The Results (page unavailable in our copy) compare observed insect flight trajectories with simulated plume-tracking algorithms. This makes the paper an example of a study addressing behaviour and computation, the one-or-two-level role for which it is cited; it does not need to establish the broader claim about most studies. (LLM, conf=0.87)
- Partly supportedcites [9] Sun, X., Mangan, M. & Yue, S. An analysis of a ring attractor modelfor cue integration. In Conference on Biomimetic and Biohybrid Systems (eds Vouloutsi, V. et al.) 459–470 (Springer, 2018). sourceExperimental — accuracy not yet measured
The Introduction (p. 4) describes a computational analysis of a ring-attractor network for integrating orientation cues, so the paper fits the cited example of a computation-focused study. It does not study plume tracking or establish that most plume-tracking studies are limited to one or two levels of analysis. (LLM, conf=0.84)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“However, a later study 8 failed to find support for this model.”
§ Emergent behaviour and neural dynamics in artificial agents tracking odour plumes > Behavioural features
What the sources say
- Partly supportedcites [8] Pang, R., van Breugel, F., Dickinson, M., Riffell, J. A. & Fairhall, A. History dependence in insect flight decisions during odor tracking. PLoS Comput. Biol. 14, e1005969 (2018). sourceExperimental — accuracy not yet measured
The Results and Discussion report that the centerline-inferring model predicted the opposite history dependence from the observed flight trajectories, so the study did not support increased upwind movement based on an inferred plume centre line. But the paper distinguishes that model from reflexive upwind surges on plume crossing; the Results say the surge-cast model reproduced the dominant upwind component of observed turns. It therefore does not support a broad claim that insects do not perform upwind surges. (LLM, conf=0.87)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Specifically, higher odour encounter rates were associated with more frequent saccadic upwind turns 66 .”
§ Emergent behaviour and neural dynamics in artificial agents tracking odour plumes > Neural representations
What the sources say
- Supportedcites [66] Celani, A. Olfactory navigation: tempo is the key. eLife 9, e63385 (2020). source
The Figure 1 description (Abstract; no page number in our copy) says that where flies are more likely to encounter the odor, the frequency of upwind turns increases. The accompanying text describes these turns as rapid turning events. It also clarifies that odor did not increase turning frequency overall; the increase concerns upwind turns. (LLM, conf=0.91)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Second, agents must perform Bayesian inference 1 .”
§ Emergent behaviour and neural dynamics in artificial agents tracking odour plumes > Algorithms for odour localization
What the sources say
- Supportedcites [1] Reddy, G., Murthy, V. N. & Vergassola, M. Olfactory sensing and navigation in turbulent environments. Annu. Rev. Condens. Matter Phys. 13, 191–213 (2022). sourceExperimental — accuracy not yet measured
Section 5.4 describes infotaxis as maintaining a posterior distribution over the source’s location and updating that belief using Bayesian inference after an action and observation. This supports the claim about agents using infotaxis. (LLM, conf=0.92)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“However, it does not capture some aspects of real plumes, such as the filamentous nature of plumes 2 , or the variation of whiff duration and whiff frequency as a function of distance from source 69 .”
§ Emergent behaviour and neural dynamics in artificial agents tracking odour plumes > Limitations and future work
What the sources say
- Supportedcites [2] Celani, A., Villermaux, E. & Vergassola, M. Odor landscapes in turbulent environments. Phys. Rev. X 4, 041015 (2014). source
The paper’s Fig. 3 caption presents whiff-duration and blank-duration distributions at various distances from the source; blank intervals bear on how frequently whiffs occur. The Discussion and Conclusions also describes how whiff clustering changes with distance. The filamentous-plume clause is attached to a different citation, and the statement about what the citing authors’ simulator omits is their own comparison, not a claim made by this source. (LLM, conf=0.89)
- Not checked — no full text of the sourcecites [69] Villermaux, E. & Innocenti, C. On the geometry of turbulent mixing. J. Fluid Mech. 393, 123–147 (1999). source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
77 more claims
Create a free account to see all 135 claims and to verify the 2 references still open.
- Probably not a substantial claim. 31
- It may state two separate claims as one. 29
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Analogous approaches have proven successful in relating the structure of neural activity to computational functions in other domains, including vision 17–19, audition 20, olfaction 21,22, thermosensation 23, perceptual discrimination 24, facial perception 25, and navigation 26,27.”
From A virtual rodent predicts the structure of neural activity across behaviours (2024) · cites this paper as [22]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Emergent behaviour and neural dynamics in artificial agents tracking odour plumes > Agent training and evaluation
“Each trained agent is evaluated with 240 episodes at different initializations (15 initial locations, two initial simulation timestamps and eight initial head directions), and in each of the constant, switch-once and switch-many plume configurations.”
§ Emergent behaviour and neural dynamics in artificial agents tracking odour plumes > Agent training and evaluation
“For each architecture type, we proceed to analyse only the five seeds with the best performance, as measured by total number of successful episodes across the four plume configurations.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Emergent behaviour and neural dynamics in artificial agents tracking odour plumes > Plume simulation
“We implement a particle-based two-dimensional plume simulation model (Fig. 1f) that mimics both short-timescale features (intermittency, instantaneous concentrations) and long-timescale features (Gaussian time-averaged concentration, filamentous long-range puff transport, meandering plume structure) of real-world odour plumes evolving in a turbulent flow 43 .”
§ Emergent behaviour and neural dynamics in artificial agents tracking odour plumes
“f, A schematic to illustrate an agent's head direction and course direction and the wind direction, all measured with respect to the ground and anticlockwise from the x axis.”
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538 · cited 1×
“We next estimate the relative importance of each input feature by calculating its permutation importance score 47,48 , which is an estimate of the reduction in the classifier’s accuracy across several (N = 30) randomized permutations of that feature.”
MethodHuman-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · cited 1×
“6), simulated by appending historical sensory observations onto instantaneous network inputs 98 .”
BackgroundProximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625
Bengio, Y., Louradour, J., Collobert, R., & Weston, J. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553380 · Cited by 5,366 · cited 1×
“More complex training curricula 81 or alternative training algorithms using evolutionary techniques 82 might be able to mitigate the notable performance variability we observed in our agents.”
BackgroundDeep Reinforcement Learning: A Brief Survey
Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2743240 · Cited by 4,733
A Brief Survey of Deep Reinforcement Learning
Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017) · arXiv · doi:10.48550/arxiv.1708.05866 · Cited by 4,733
Conditional variable importance for random forests
Strobl, C., Boulesteix, A.-L., Kneib, T., Augustin, T., & Zeileis, A. (2008) · BMC Bioinformatics · doi:10.1186/1471-2105-9-307 · Cited by 3,254
Adler, J. (1976) · Surface Membrane Receptors · doi:10.1007/978-1-4684-2772-1_29 · Cited by 1,275
Dimensionality reduction for large-scale neural recordings
Cunningham, J. P., & Yu, B. M. (2014) · Nature Neuroscience · doi:10.1038/nn.3776 · Cited by 1,219
Kriegeskorte, N. (2015) · bioRxiv · doi:10.1101/029876 · Cited by 1,141
Research works printed but not linked: 7
- Kanitscheider, I. & Fiete, I. Training recurrent networks to generate hypotheses about how the brain solves hard navigation problems. In Proc. 31st International Conference on Neural Information Processing Systems (eds von Luxburg, U. et al.) 4532–4541 (Curran Associates Inc., 2017).
- Sutton, R. S. & Barto, A. G. Reinforcement Learning: An Introduction (MIT Press, 2018).
- Konda, V. R. & Tsitsiklis, J. N. Actor–critic algorithms. Adv. Neural Inf. Process. Syst. 12, 1008–1014 (1999).
- Baker, T. C. Upwind flight and casting flight: complementary phasic and tonic systems used for location of sex pheromone sources by male moth. In Proc. 10th International Symposium on Olfaction and Taste (ed. Doving, K. B.) 18–25 (1990).
- Stachenfeld, K. et al. Learned simulators for turbulence. In International Conference on Learning Representations (2021).
- Duncker, L., Driscoll, L., Shenoy, K. V., Sahani, M. & Sussillo, D. Organizing recurrent network dynamics by task-computation to enable continual learning. In Proc. 34th International Conference on Neural Information Processing Systems (eds Larochelle, H. et al.) 14387–14397 (Curran Associates Inc., 2020).
- Henaff, M., Szlam, A. & Le Cun, Y. Recurrent orthogonal networks and long-memory tasks. In Proc. 33rd International Conference on Machine Learning (eds Balcan, M. F. & Weinberger, K. Q.) 2034–2042 (JMLR, 2016).
Cited by
A virtual rodent predicts the structure of neural activity across behaviours
Aldarondo, D., Merel, J., Marshall, J. D., Hasenclever, L., Klibaite, U., Gellis, A., Tassa, Y., Wayne, G., Botvinick, M., & Ölveczky, B. P. (2024) · Nature · doi:10.1038/s41586-024-07633-4 · Cited by 70
Njirjak, M., Žužić, L., Babić, M., Janković, P., Otović, E., Kalafatović, D., & Mauša, G. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00928-1 · Cited by 64
Emergence of belief-like representations through reinforcement learning
Hennig, J. A., Romero Pinto, S. A., Yamaguchi, T., Linderman, S. W., Uchida, N., & Gershman, S. J. (2023) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1011067 · Cited by 41
“…the intersection of neuroscience and machine learning has shown how agents performing reinforcement learning can use RNNs to solve a variety of tasks featuring state and task uncertainty, including tasks involving motor control [30], navigation [31–33], foraging [34], and decision-making [35, 36].”
BackgroundWind gates olfaction-driven search states in free flight
Stupski, S. D., & van Breugel, F. (2024) · Current Biology · doi:10.1016/j.cub.2024.07.009 · Cited by 41
Self-citationNeural circuits for goal-directed navigation across species
Basu, J., & Nagel, K. I. (2024) · Trends in Neurosciences · doi:10.1016/j.tins.2024.09.005 · Cited by 40
Metadata
- Authors
- Singh, S. H., van Breugel, F., Rao, R. P. N., & Brunton, B. W.
- Affiliations
- University of Washington (US)
- Corresponding author
- Singh, Satpreet H.
- Venue
- Nature Machine Intelligence
- Published
- 2023-01-25
- Volume
- 5
- Issue
- 1
- Pages
- 58-70
- ISSN
- 2522-5839 (ISSN-L)
- PMC
- PMC10601839
- OpenAlex
- W4317952808
- Cited by
- 61
- References
- 90
- Citations by year
- 2026: 12 · 2025: 25 · 2024: 17 · 2023: 7
- FWCI
- 7.38
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Nature Machine Intelligence (publishedVersion, cc-by)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion, cc-by)
- Funders
- United States Department of Defense | United States Air Force | AFMC | Air Force Research Laboratory (FA8651-20-1-0002, FA9550-21-0122)
- United States Department of Defense | Defense Advanced Research Projects Agency (HR001120C0021)
- National Science Foundation (EEC-1028725)
- United States Department of Defense | United States Air Force | AFMC | Air Force Office of Scientific Research (FA9550-19-1-0386, FA9550-18-1-0114)
- Washington Research Foundation
- Topics
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurobiology and Insect Physiology Research
- Life Sciences › Agricultural and Biological Sciences › Insect Science › Insect Pheromone Research and Control
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Keywords
- artificial agents, recurrent neural networks, deep reinforcement learning, emergent behavior, neural dynamics, population activity, turbulent flow, wind direction change, insect navigation, memory requirements
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-01, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Emergent behaviour and neural dynamics in artificial agents tracking odour plumes. https://app.cochat.ai/papers/10.1038/s42256-022-00599-w
- The paper
- Singh, S. H., van Breugel, F., Rao, R. P. N., & Brunton, B. W. (2023). Emergent behaviour and neural dynamics in artificial agents tracking odour plumes. Nature Machine Intelligence. https://doi.org/10.1038/s42256-022-00599-w
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.