CoChat
Sign Up
PreprintCited by 2Embodied and Extended CognitionBayesian Modeling and Causal InferenceGaussian Processes and Bayesian Inference

pyhgf: A neural network library for predictive coding

Open access
Legrand, Nicolas, Weber, Lilian, Waade, Peter Thestrup, Daugaard, Anna Hedvig Møller, Khodadadi, Mojtaba, Mikuš, Nace, Mathys, Chris·2024·arXiv
DOI 10.48550/arxiv.2410.09206arXiv 2410.09206PMC PMC13318038
Open this paper in CoChat — free accountOpen PDF

Abstract

Bayesian models of cognition have gained considerable traction in computational neuroscience and psychiatry. Their scopes are now expected to expand rapidly to artificial intelligence, providing general inference frameworks to support embodied, adaptable, and energy-efficient autonomous agents. A central theory in this domain is predictive coding, which posits that learning and behaviour are driven by hierarchical probabilistic inferences about the causes of sensory inputs. Biological realism constrains these networks to rely on simple local computations in the form of precision-weighted predictions and prediction errors. This can make this framework highly efficient, but its implementation comes with unique challenges on the software development side. Embedding such models in standard neural network libraries often becomes limiting, as these libraries' compilation and differentiation backends can force a conceptual separation between optimization algorithms and the systems being optimized. This critically departs from other biological principles such as self-monitoring, self-organisation, cellular growth and functional plasticity. In this paper, we introduce \texttt{pyhgf}: a Python package backed by JAX and Rust for creating, manipulating and sampling dynamic networks for predictive coding. We improve over other frameworks by enclosing the network components as transparent, modular and malleable variables in the message-passing steps. The resulting graphs can implement arbitrary computational complexities as beliefs propagation. But the transparency of core variables can also translate into inference processes that leverage self-organisation principles, and express structure learning, meta-learning or causal discovery as the consequence of network structural adaptation to surprising inputs. The code, tutorials and documentation are hosted at: https://github.com/ilabcode/pyhgf.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2410.09206

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The gHGF's modular message-passing structure yields update equations that are derived analytically, are local by construction, and require no automatic-differentiation scaffolding.

    “its modular message-passing structure yields update equations that are derived analytically, are local by construction, require no automatic-differentiation scaffolding”

    From Closed-form predictive coding via hierarchical Gaussian filters (2026) · cites this paper as Legrand et al., 2024

    The Introduction and §2.1 describe analytic, closed-form update equations and local computations on adjacent nodes. The claim that no automatic-differentiation scaffolding is required is not established by those passages; §2.2 says pyhgf leverages JAX automatic differentiation for optimisation and inference, though it does not say the update equations themselves require it. Our copy provides no page numbers for these sections. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • The gHGF's update equations open the expressivity of network structures and dynamics that can be supported.

    “open the expressivity of network structures and dynamics that can be supported [Legrand et al., 2024]”

    From Closed-form predictive coding via hierarchical Gaussian filters (2026) · cites this paper as Legrand et al., 2024

  • The authors provide an implementation of the modular HGF in Python and Julia that allows users to design HGF structures for simulation and empirical parameter estimation.

    “In two open-source projects, we provide such an implementation (in Python and Julia, Legrand et al., 2024; Thestrup Waade et al., in prep), which allows users to flexibly design their own HGF structures that can be used for simulation and empirical parameter estimation.”

    From The generalized Hierarchical Gaussian Filter (2026) · cites this paper as Legrand et al., 2024

  • The authors provide implementations in two open-source projects available as part of the TAPAS software collection.

    “We provide such implementations in two open-source projects (Legrand et al., 2024; Thestrup Waade et al., in prep) which are available as part of the TAPAS software collection (https://github.com/ComputationalPsychiatry Frässle et al., 2021).”

    From The generalized Hierarchical Gaussian Filter (2026) · cites this paper as Legrand et al., 2024

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 56

  1. [1]Ji W, Kording KP. Bayesian models of perception and action. London, England: MIT Press. 2023.
  2. [2]Huys QJM, Maia TV, Frank MJ. Computational psychiatry as a bridge from neuroscience to clinical applications. Nat Neurosci. 2016; 19(3): 404– 13. doi: 10.1038/nn.4238 26906507 PMC5443409 · DOI · Paper page
  3. [3]Sandhu TR, Xiao B, Lawson RP. Transdiagnostic computations of uncertainty: towards a new lens on intolerance of uncertainty. Neurosci Biobehav Rev. 2023; 148: 105123. doi: 10.1016/j.neubiorev.2023.105123 36914079 · DOI · Paper page
  4. [4]Friston K. Computational psychiatry: from synapses to sentience. Mol Psychiatry. 2023; 28(1): 256– 68. doi: 10.1038/s41380-022-01743-z 36056173 PMC7614021 · DOI
  5. [5]Mathys C, Weber L. Hierarchical Gaussian filtering of sufficient statistic time series for active inference. Communications in Computer and Information Science. Springer; 2020. p. 52– 8. 10.1007/978-3-030-64919-7_7 · DOI · Paper page
  6. [6]Da Costa L, Lanillos P, Sajid N, Friston KJ, Khan S. How active inference could help revolutionise robotics. Entropy. 2022; 24(3): 361. doi: 10.3390/e24030361 35327872 PMC8946999 · DOI
  7. [7]Rao RP, Ballard DH. Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nat Neurosci. 1999; 2(1): 79– 87. doi: 10.1038/4580 10195184 · DOI · Paper page
  8. [8]Friston K. A theory of cortical responses. Philos Trans R Soc Lond B Biol Sci. 2005; 360(1456): 815– 36. doi: 10.1098/rstb.2005.1622 15937014 PMC1569488 · DOI · Paper page
  9. [9]Friston K. Hierarchical models in the brain. PLoS Comput Biol. 2008; 4(11):e1000211. doi: 10.1371/journal.pcbi.1000211 18989391 PMC2570625 · DOI · Paper page
  10. [10]Mikulasch FA, Rudelt L, Wibral M, Priesemann V. Where is the error? Hierarchical predictive coding through dendritic error computation. Trends Neurosci. 2023; 46(1): 45– 59. doi: 10.1016/j.tins.2022.09.007 36577388 · DOI
  11. [11]Ororbia A, Kifer D. The neural coding framework for learning generative models. Nat Commun. 2022; 13(1): 2064. doi: 10.1038/s41467-022-29632-7 35440589 PMC9018730 · DOI
  12. [12]Rumelhart DE, Hinton GE, Williams RJ. Learning representations by back-propagating errors. Nature. 1986; 323(6088): 533– 6. doi: 10.1038/323533a0 · DOI · Paper page
  13. [13]Millidge B, Salvatori T, Song Y, Bogacz R, Lukasiewicz T. Predictive coding: Towards a future of deep learning beyond backpropagation? 2022. 10.48550/arXiv.2202.09467 · DOI · Paper page
  14. [14]Song Y, Millidge B, Salvatori T, Lukasiewicz T, Xu Z, Bogacz R. Inferring neural activity before plasticity as a foundation for learning beyond backpropagation. Nat Neurosci. 2024; 27(2): 348– 58. doi: 10.1038/s41593-023-01514-1 38172438 PMC7615830 · DOI · Paper page
  15. [16]Paszke A, Gross S, Massa F, Lerer A, Bradbury J, Chanan G, et al. PyTorch: An Imperative Style, High-Performance Deep Learning Library. 2019. https://arxiv.org/abs/1912.01703 · arXiv
  16. [17]Fey M, Lenssen JE. Fast Graph Representation Learning with PyTorch Geometric. 2019. https://arxiv.org/abs/1903.02428 · arXiv
  17. [19]Daunizeau J, den Ouden HEM, Pessiglione M, Kiebel SJ, Stephan KE, Friston KJ. Observing the observer (I): meta-bayesian models of learning and decision-making. PLoS One. 2010; 5(12):e15554. doi: 10.1371/journal.pone.0015554 21179480 PMC3001878 · DOI · Paper page
  18. [20]Betancourt M. A conceptual introduction to Hamiltonian Monte Carlo. 2017. 10.48550/arXiv.1701.02434 · DOI
  19. [21]Salvatori T, Mali A, Buckley CL, Lukasiewicz T, Rao RPN, Friston K, et al. A survey on neuro-mimetic deep learning via predictive coding. Neural Netw. 2026; 195: 108161. doi: 10.1016/j.neunet.2025.108161 41161207 · DOI · Paper page
  20. [22]van Zwol B, Jefferson R, van den Broek EL. Predictive coding networks and inference learning: tutorial and survey. ACM Computing Surveys. 2026. doi: 10.1145/3797870 · DOI
  21. [23]Millidge B, Seth A, Buckley CL. Predictive coding: a theoretical and experimental review. 2022. https://arxiv.org/abs/2107.12979 · arXiv
  22. [24]Innocenti F, Kinghorn P, Yun-Farmbrough W, Varona MDL, Singh R, Buckley CL. Flexible inference for predictive coding networks in JAX. 2024. https://arxiv.org/abs/2412.03676 · arXiv
  23. [25]Pinchetti L, Qi C, Lokshyn O, Olivers G, Emde C, Tang M, et al. Benchmarking predictive coding networks – made simple. 2024. 10.48550/arXiv.2407.01163 · DOI · Paper page
  24. [26]Millidge B, Tang M, Osanlouy M, Harper NS, Bogacz R. Predictive coding networks for temporal prediction. PLoS Comput Biol. 2024; 20(4):e1011183. doi: 10.1371/journal.pcbi.1011183 38557984 PMC11008833 · DOI · Paper page
  25. [27]Mathys C, Daunizeau J, Friston KJ, Stephan KE. A bayesian foundation for individual learning under uncertainty. Front Hum Neurosci. 2011; 5: 39. doi: 10.3389/fnhum.2011.00039 21629826 PMC3096853 · DOI · Paper page
  26. [28]Mathys CD, Lomakina EI, Daunizeau J, Iglesias S, Brodersen KH, Friston KJ, et al. Uncertainty in perception and the Hierarchical Gaussian Filter. Front Hum Neurosci. 2014; 8: 825. doi: 10.3389/fnhum.2014.00825 25477800 PMC4237059 · DOI · Paper page
  27. [29]Weber LA, Waade PT, Legrand N, Møller AH, Stephan KE, Mathys C. The generalised Hierarchical Gaussian Filter. 2026. doi: 10.7554/elife.110174.1 · DOI · Paper page
  28. [30]Corlett PR, Horga G, Fletcher PC, Alderson-Day B, Schmack K, Powers AR. Hallucinations and strong priors. Trends in Cognitive Sciences. 2019; 23(2): 114– 27. doi: 10.1016/j.tics.2018.12.001 30583945 PMC6368358 · DOI · Paper page
  29. [31]Reed EJ, Uddenberg S, Suthaharan P, Mathys CD, Taylor JR, Groman SM, et al. Paranoia as a deficit in non-social belief updating. Elife. 2020; 9:e56345. doi: 10.7554/eLife.56345 32452769 PMC7326495 · DOI
  30. [32]Powers AR, Mathys C, Corlett PR. Pavlovian conditioning-induced hallucinations result from overweighting of perceptual priors. Science. 2017; 357(6351): 596– 600. doi: 10.1126/science.aan3458 28798131 PMC5802347 · DOI · Paper page
  31. [33]Lawson RP, Mathys C, Rees G. Adults with autism overestimate the volatility of the sensory environment. Nat Neurosci. 2017; 20(9): 1293– 9. doi: 10.1038/nn.4615 28758996 PMC5578436 · DOI · Paper page
  32. [34]Frässle S, Aponte EA, Bollmann S, Brodersen KH, Do CT, Harrison OK. TAPAS: An open-source software package for translational neuromodeling and computational psychiatry. Frontiers in Psychiatry. 2021; 12. doi: 10.3389/fpsyt.2021.680811 PMC8206497 34149484 · DOI · Paper page
  33. [36]Matsakis ND, Klock II KS. The rust language. Ada Lett. 2014; 34(3): 103– 4. doi: 10.1145/2692956.2663188 · DOI
  34. [39]Abril-Pla O, Andreani V, Carroll C, Dong L, Fonnesbeck CJ, Kochurov M, et al. PyMC: a modern and comprehensive probabilistic programming framework in Python. PeerJ Comput Sci. 2023; 9:e1516. doi: 10.7717/peerj-cs.1516 37705656 PMC10495961 · DOI
  35. [40]Kumar R, Carroll C, Hartikainen A, Martin O. ArviZ a unified library for exploratory analysis of Bayesian models in Python. JOSS. 2019; 4(33): 1143. doi: 10.21105/joss.01143 · DOI
  36. [41]Kidger P, Garcia C. Equinox: neural networks in JAX via callable PyTrees and filtered transformations. arXiv. 2021. https://arxiv.org/abs/2111.00254 · arXiv
  37. [42]De Domenico M. More is different in real-world multilayer networks. Nat Phys. 2023; 19(9): 1247– 62. doi: 10.1038/s41567-023-02132-1 · DOI
  38. [43]Oliver BM. Efficient Coding. Bell System Technical Journal. 1952; 31(4): 724– 50. doi: 10.1002/j.1538-7305.1952.tb01403.x · DOI
  39. [44]Hoffman MW, Shahriari B, Aslanides J, Barth-Maron G, Momchev N, Sinopalnikov D. Acme: a research framework for distributed reinforcement learning. 2020. https://arxiv.org/abs/2006.00979 · arXiv
  40. [45]Hoffman MD, Gelman A. The No-U-turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo. J Mach Learn Res. 2014; 15(1): 1593– 623.
  41. [46]Iglesias S, Kasper L, Harrison SJ, Manka R, Mathys C, Stephan KE. Cholinergic and dopaminergic effects on prediction error and uncertainty responses during sensory associative learning. Neuroimage. 2021; 226: 117590. doi: 10.1016/j.neuroimage.2020.117590 33285332 · DOI
  42. [47]Vehtari A, Gelman A, Gabry J. Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC. Stat Comput. 2016; 27(5): 1413– 32. doi: 10.1007/s11222-016-9696-4 · DOI
  43. [48]Salvatori T, Pinchetti L, M’Charrak A, Millidge B, Lukasiewicz T. Causal inference via predictive coding. 2023. https://arxiv.org/abs/2306.15479 · arXiv
  44. [49]Salvatori T, Pinchetti L, Millidge B, Song Y, Bao T, Bogacz R. Learning on arbitrary graph topologies via predictive coding. 2022. https://arxiv.org/abs/2201.13180 PMC7614467 37090087 · arXiv
  45. [50]Friston KJ, Parr T, de Vries B. The graphical brain: Belief propagation and active inference. Netw Neurosci. 2017; 1(4): 381– 414. doi: 10.1162/NETN_a_00018 29417960 PMC5798592 · DOI · Paper page
  46. [51]Senoz I, de Vries B. Online message passing-based inference in the hierarchical Gaussian filter. In: 2020 IEEE International Symposium on Information Theory (ISIT), 2020. 10.1109/isit44484.2020.9173980 · DOI
  47. [52]Senoz I, Podusenko A, Akbayrak S, Mathys C, de Vries B. The Switching Hierarchical Gaussian Filter. In: 2021 IEEE International Symposium on Information Theory (ISIT), 2021. 10.1109/isit45174.2021.9518229 · DOI
  48. [53]Choksi B, Mozafari M, O’May CB, Ador B, Alamia A, VanRullen R. Predify: augmenting deep neural networks with brain-inspired predictive coding dynamics. 2021. https://arxiv.org/abs/2106.02749 · arXiv
  49. [54]Lotter W, Kreiman G, Cox D. Deep predictive coding networks for video prediction and unsupervised learning. 2017. https://arxiv.org/abs/1605.08104 · arXiv
  50. [55]Hashim IC, Senden M, Goebel R. PrediRep: modeling hierarchical predictive coding with an unsupervised deep learning network. Neural Netw. 2025; 185: 107246. doi: 10.1016/j.neunet.2025.107246 39946763 · DOI
  51. [56]Najarro E, Sudhakaran S, Risi S. Towards self-assembling artificial neural networks through neural developmental programs. 2023. https://arxiv.org/abs/2307.08197 · arXiv
  52. [57]Kudithipudi D, Aguilar-Simon M, Babb J, Bazhenov M, Blackiston D, Bongard J, et al. Biological underpinnings for lifelong learning machines. Nat Mach Intell. 2022; 4(3): 196– 210. doi: 10.1038/s42256-022-00452-0 · DOI · Paper page
  53. [58]Goddu MK, Gopnik A. The development of human causal learning and reasoning. Nat Rev Psychol. 2024; 3(5): 319– 39. doi: 10.1038/s44159-024-00300-5 · DOI
  54. [59]Deng Z, Jiang J, Long G, Zhang C. Causal reinforcement learning: a survey. 2023. 10.48550/arXiv.2307.01452 · DOI
  55. [60]Chen J, Bornstein AM. The causal structure and computational value of narratives. Trends Cogn Sci. 2024; 28(8): 769– 81. doi: 10.1016/j.tics.2024.04.003 38734531 PMC11305923 · DOI
  56. [61]Binz M, Dasgupta I, Jagadish AK, Botvinick M, Wang JX, Schulz E. Meta-learned models of cognition. Behav Brain Sci. 2023; 47:e147. doi: 10.1017/S0140525X23003266 37994495 · DOI · Paper page

Cited by

Sorted by citations

Authors
Legrand, N., Weber, L., Waade, P. T., Daugaard, A. H. M., Khodadadi, M., Mikuš, N., & Mathys, C.
Affiliations
  • Aarhus University (DK)
  • University of Oxford (GB)
  • Scuola Internazionale Superiore di Studi Avanzati (IT)
Corresponding author
Legrand, Nicolas
ORCID
Venue
arXiv
Publisher
arXiv
Published
2024
Volume
22
Issue
6
Pages
e1014340-e1014340
ISSN
1553-734X (ISSN-L)
DOI
10.48550/arxiv.2410.09206
arXiv
2410.09206
PMC
PMC13318038
OpenAlex
W4403564456
Cited by
0
References
0
FWCI
0.00
Type
journal article
Language
en
Open access
Licence
Locations
  • PLoS Computational Biology (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • PubMed Central (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Funders
  • Aarhus Universitet (AUFF-E-2019-7-10)
  • Aarhus Universitets Forskningsfond (CF21-0439, AUFF‐E‐2019‐7‐10)
  • Carlsbergfondet (CF21‐0439)
Topics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Embodied and Extended Cognition
  • Physical Sciences › Computer Science › Artificial Intelligence › Bayesian Modeling and Causal Inference
  • Physical Sciences › Computer Science › Artificial Intelligence › Gaussian Processes and Bayesian Inference
Keywords
predictive coding, Bayesian models of cognition, message passing, belief propagation, differentiable programming, JAX, Rust, dynamic Bayesian networks, hierarchical Gaussian filter, Bayesian filtering, structure learning, meta-learning, causal discovery, self-organization, neural network libraries
MeSH terms
Algorithms, Bayes Theorem, Cognition, Humans, Models, Neurological, Software, Neural Networks, Computer, Computational Biology

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review