Reactive Environments for Active Inference Agents with RxEnvironments.jl
Open accessThe DOI matches its Crossref record; 26 of the 27 references it cites were found in a registry or an index; 12 of the 17 sources its claims cite were checked for support.
Abstract
Active Inference is a framework that emphasizes the interaction between agents and their environment. While the framework has seen significant advancements in the development of agents, the environmental models are often borrowed from reinforcement learning problems, which may not fully capture the complexity of multi-agent interactions or allow complex, conditional communication. This paper introduces Reactive Environments, a comprehensive paradigm that facilitates complex multi-agent communication. In this paradigm, both agents and environments are defined as entities encapsulated by boundaries with interfaces. This setup facilitates a robust framework for communication in nonequilibrium-Steady-State systems, allowing for complex interactions and information exchange. We present a Julia package RxEnvironments.jl, which is a specific implementation of Reactive Environments, where we utilize a Reactive Programming style for efficient implementation. The flexibility of this paradigm is demonstrated through its application to several complex, multi-agent environments. These case studies highlight the potential of Reactive Environments in modeling sophisticated systems of interacting agents.
Overview
CoChat Summary
TL;DR
The paper introduces Reactive Environments, a paradigm in which agents and environments are Entities with actuator/sensor boundaries and mutable subscriptions, and presents RxEnvironments.jl, a Julia implementation using reactive programming. It demonstrates the framework on Mountain Car, a 22-player football simulation, and a hearing-aid multi-entity setting.
Key findings
- A Reactive Environment is defined as a pair \(E=(A,S)\), where \(A\) is a set of Entities and \(S\) is a mutable set of subscriptions \((A_1,A_2)\); each Entity responds reactively to sensory impulses and can emit data through actuators.
- RxEnvironments.jl splits the classical transition function into `receive!` and `update!`, and exposes `emits` and `what_to_send` to control when and what an Entity emits; this natively supports multi-agent, multi-sensor, and conditional communication.
- Case studies show: Mountain Car emits sensor information at 2 Hz and proprioceptive feedback on action; football models 22 player Entities plus a world Entity, with random run commands sent to a random player every 0.1 seconds; the hearing aid communicates with three subscribed entities—outside world, user, and phone agent.
- Classical reinforcement learning environments can be written as particular cases of Reactive Environments.
Methods
- Conceptual and implementation paper: defines Entity, boundary, subscriptions, and Reactive Environment, then implements the paradigm in the Julia package RxEnvironments.jl, based on the Rocket.jl reactive programming library.
- No empirical dataset or benchmark is used; the framework is demonstrated through three case studies: Mountain Car, a football match simulation, and a hearing-aid environment.
- Main analysis is implementation-level, describing functions such as `receive!`, `update!`, `emits`, and `what_to_send`, plus internal triggers for regular emission; an appendix gives a Bayesian Thermostat example.
Limitations
- The paper states that the design of agents able to interact with a Reactive Environment still needs investigation, because fewer communication constraints require agents to have an internal clock, and the authors are not aware of any agent implementation built for this flexibility.
- The paper states that the football case models only running and on-ball actions, not a comprehensive football environment with full rules and dynamics.
- Evident from the text: evaluation is demonstrative case studies without quantitative performance metrics, benchmarks, or empirical comparisons of agent behavior.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
5 Valid · 7 Not valid · 2 Not checked · 21 Own results and interpretations
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“In general, we observe that there is no standardized way of defining environments for Active Inference agents.”
What the sources say
- Could not verifycites [24] Ueltzhöffer, K.: Deep active inference. Biol. Cybern. 112(6), 547–573 (2018). https://doi.org/10.1007/s00422-018-0785-7, [q-bio] sourceExperimental — accuracy not yet measured
The Introduction (p. 2) says the paper describes a mountain-car environment, and the Discussion (p. 27) compares its implementation with another. Neither passage addresses whether a standardized way to define environments for Active Inference agents exists. Searches for standardization and common environment definitions found no passage making that claim. (LLM, conf=0.74)
- Could not verifycites [17] Van de Maele, T., Dhoedt, B., Verbelen, T., Pezzulo, G.: Integrating cognitive map learning and active inference for planning in ambiguous environments (2023). https://doi.org/10.48550/arXiv.2308.08307, [cs] sourceExperimental — accuracy not yet measured
Sections 3.1 and 3.3 describe particular environments used to evaluate the agents, but do not address whether there is a standardized way to define environments for Active Inference agents. Searches for standardized environments and common benchmarks found no passage making that broader claim. (LLM, conf=0.72)
- Could not verifycites [20] Safa, A., et al.: Active Inference in Hebbian Learning Networks (2023). https://doi.org/10.48550/arXiv.2306.05053, [cs] sourceExperimental — accuracy not yet measured
Section 4.1, “Mountain Car Environment” (page unavailable), describes the OpenAI Gym environment used in this study. The Introduction and Conclusion (pages unavailable) discuss the agent architecture and results, but none of these passages addresses whether there is a standardized way to define environments for Active Inference agents. Searches for that claim and for environment standards or benchmarks found no such statement. (LLM, conf=0.73)
- Not checked — no full text of the sourcecites [5] Esaki, K., Matsumura, T., Minusa, S., Shao, Y., Yoshimura, C., Mizuno, H.: Dynamical perception-action loop formation with developmental embodiment for hierarchical active inference. In: Buckley, C.L., et al. (eds.) Active Inference, pp. 14–28. Springer, Cham (2024). https://doi.org/10.1007/978-3-031-47958-8_2 source
- Could not verifycites [10] Friston, K.J., et al.: Federated inference and belief sharing. Neurosci. Biobehav. Rev. 156, 105500 (2024). https://doi.org/10.1016/j.neubiorev.2023.105500. https://www.sciencedirect.com/science/article/pii/S0149763423004694 sourceExperimental — accuracy not yet measured
The Software note (page unavailable) says generative models change between applications, and Fig. 4 in “Active inference and belief sharing” describes how this paper specifies its simulation. Neither passage addresses whether a standardized way of defining environments for active-inference agents exists. I searched for standardization, environment specification, and conventions in our copy. (LLM, conf=0.75)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Although the realization of environments is also part of popular packages such as PyMDP [11] and the SPM-DEM toolbox [6], these packages have their primary focus on agent creation.”
What the sources say
- Supportedcites [11] Heins, C., et al.: pymdp: a Python library for active inference in discrete state spaces. J. Open Source Softw. 7(73), 4098 (2022). https://doi.org/10.21105/joss.04098. https://joss.theoj.org/papers/10.21105/joss.04098 source
The Related software packages section says pymdp is specifically designed to help users quickly build agents and plug them into environments. The Abstract also describes its environment API. The SPM-DEM toolbox has a separate citation in the citing sentence, so this verdict addresses the claim about pymdp. (LLM, conf=0.93)
- Not checked — no full text of the sourcecites [6] Friston, K.J., Trujillo-Barreto, N., Daunizeau, J.: DEM: a variational treatment of dynamic systems. Neuroimage 41(3), 849–885 (2008). https://doi.org/10.1016/j.neuroimage.2008.02.054 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In our case studies, we showed that our framework can be used to define a multitude of different environments, demonstrating the expressive power of the framework.”
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In particular, we showed that with Reactive Environments we are able to model the complex communications between agents and environments necessary to realize Active Inference simulations.”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This means that entities representing agents and environments have no constraints on their communication, allowing agent-agent subscriptions or multi-agent environments natively.”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
7 more claims
Create a free account to see all 35 claims and every source verdict.
- Probably not a substantial claim. 20
- It may state two separate claims as one. 7
- The quote only partly states this claim. 1
Citations
Cites
The free-energy principle: a unified brain theory?
Friston, K. (2010) · Nature Reviews Neuroscience · doi:10.1038/nrn2787 · Cited by 8,120 · cited 1×
“The Free Energy Principle (FEP) [7] distinguishes itself from other theories of self-organization by taking an interaction-centric perspective.”
Julia: A Fresh Approach to Numerical ComputingPreprint
Bezanson, J., Edelman, A., Karpinski, S., & Shah, V. B. (2014) · SIAM Review · doi:10.48550/arxiv.1411.1607 · Cited by 7,118 · cited 2×
“We will discuss how a reactive programming strategy addresses the flaws of current frameworks and introduce RxEnvironments.jl , a specific implementation of Reactive Environments in the Julia language [4].”
InfluentialJulia: A Fresh Approach to Numerical Computing
Bezanson, J., Edelman, A., Karpinski, S., & Shah, V. B. (2017) · SIAM Review · doi:10.1137/141000671 · Cited by 5,440
Autopoiesis: The organization of living systems, its characterization and a model
Varela, F. G., Maturana, H. R., & Uribe, R. (1974) · Biosystems · doi:10.1016/0303-2647(74)90031-8 · Cited by 1,899 · cited 1×
“Self-organizing agents maintain their existence by creating a boundary that separates their internal states from the external states of their environment [12,25,18,15].”
Action and behavior: a free-energy formulation
Friston, K. J., Daunizeau, J., Kilner, J., & Kiebel, S. J. (2010) · Biological Cybernetics · doi:10.1007/s00422-010-0364-z · Cited by 866 · cited 1×
“AIF posits that agents actively seek to minimize their free energy (a measure related to surprise or prediction error) by updating their beliefs about the environment and selecting actions that align with these beliefs [9].”
Predictions not commands: Active inference in the motor system
Adams, R. A., Shipp, S., & Friston, K. J. (2013) · Brain Structure and Function · doi:10.1007/s00429-012-0475-5 · Cited by 765 · cited 2×
“Presenting observations in this way aligns with [1].”
The Factor Graph Approach to Model-Based Signal Processing
Loeliger, H.-A., Dauwels, J., Hu, J., Korl, S., Ping, L., & Kschischang, F. R. (2007) · Proceedings of the IEEE · doi:10.1109/jproc.2007.896497 · Cited by 526 · cited 1×
“An interesting avenue is Reactive Message Passing on a Factor Graph [16,3], which employs the same reactive programming strategy to Bayesian inference that we have taken in this paper to environment design.”
Tassa, Y., Doron, Y., Muldal, A., Erez, T., Li, Y., Casas, D. D. L., Budden, D., Abdolmaleki, A., Merel, J., Lefrancq, A., et al. (2018) · arXiv · doi:10.48550/arxiv.1801.00690 · Cited by 497 · cited 1×
“A similar alternative in Python, based on the MuJoCo physics engine, is Deepmind Control Suite [21].”
The Markov blankets of life: autonomy, active inference and the free energy principle
Kirchhoff, M., Parr, T., Palacios, E., Friston, K., & Kiverstein, J. (2018) · Journal of The Royal Society Interface · doi:10.1098/rsif.2017.0792 · Cited by 473 · cited 1×
“Self-organizing agents maintain their existence by creating a boundary that separates their internal states from the external states of their environment [12,25,18,15].”
DEM: A variational treatment of dynamic systems
Friston, K. J., Trujillo-Barreto, N., & Daunizeau, J. (2008) · NeuroImage · doi:10.1016/j.neuroimage.2008.02.054 · Cited by 278 · cited 1×
“Although the realization of environments is also part of popular packages such as PyMDP [11] and the SPM-DEM toolbox [6], these packages have their primary focus on agent creation.”
Research works printed but not linked: 1
- Hewitt, C., Bishop, P., Steiger, R.: Session 8 formalisms for artificial intelligence a universal modular actor formalism for artificial intelligence. In: Advance Papers of the Conference, vol. 3, p. 235. Stanford Research Institute, Menlo Park, CA (1973)
Metadata
- Authors
- Nuijten, W. W. L., & de Vries, B.
- Affiliations
- Eindhoven University of Technology (NL)
- Corresponding author
- Nuijten, Wouter W. L.
- Venue
- Communications in computer and information science
- Publisher
- Springer Nature Switzerland
- Published
- 2024-12-31
- Pages
- 147-161
- ISSN
- 1865-0929 (ISSN-L)
- 1865-0937
- OpenAlex
- W4405888531
- Cited by
- 0
- References
- 25
- FWCI
- 0.00
- Type
- conference paper
- Language
- en
- Open access
- green
- research.tue.nl
- Licence
- other-oa
- www.springernature.com/gp/researchers/text-and-data-mining (tdm)
- www.springernature.com/gp/researchers/text-and-data-mining (vor)
- Locations
- Communications in computer and information science (publishedVersion)
- TU/e Research Portal (publishedVersion, other-oa)
- arXiv (Cornell University) (submittedVersion)
- TU/e Research Portal (submittedVersion, other-oa)
- arXiv (Cornell University)
- Funders
- Nederlandse Organisatie voor Wetenschappelijk Onderzoek
- Ministerie van Economische Zaken en Klimaat
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Embodied and Extended Cognition
- Physical Sciences › Computer Science › Artificial Intelligence › Multi-Agent Systems and Negotiation
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Keywords
- active inference, reactive environments, multi-agent communication, agent-environment interaction, reactive programming, Julia package, nonequilibrium steady states
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-27, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-27 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Reactive Environments for Active Inference Agents with RxEnvironments.jl. https://app.cochat.ai/papers/10.1007/978-3-031-77138-5_10
- The paper
- Nuijten, W. W. L., & de Vries, B. (2025). Reactive Environments for Active Inference Agents with RxEnvironments.jl. Communications in computer and information science. https://doi.org/10.1007/978-3-031-77138-5_10
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.