CoChat
Sign Up
PreprintCited by 2Neural Networks and ApplicationsAdvanced Memory and Neural ComputingEmbodied and Extended Cognition

The DOI matches its DataCite record.

ePC: Fast and Deep Predictive Coding in Digital Simulation

Open access
Goemaere, Cédric, Oliviers, Gaspard, Bogacz, Rafal, Demeester, Thomas·2025·arXiv
DOI 10.48550/arxiv.2505.20137arXiv 2505.20137
Open this paper in CoChat — free accountOpen PDF

Abstract

Predictive Coding (PC) offers a brain-inspired alternative to backpropagation for neural network training, described as a physical system minimizing its internal energy. However, in practice, PC is predominantly digitally simulated, requiring excessive amounts of compute while struggling to scale to deeper architectures. This paper reformulates PC to overcome this hardware-algorithm mismatch. First, we uncover how the canonical state-based formulation of PC (sPC) is, by design, deeply inefficient in digital simulation, inevitably resulting in exponential signal decay that stalls the entire minimization process. Then, to overcome this fundamental limitation, we introduce error-based PC (ePC), a novel reparameterization of PC which does not suffer from signal decay. Though no longer biologically plausible, ePC numerically computes exact PC weights gradients and runs orders of magnitude faster than sPC. Experiments across multiple architectures and datasets demonstrate that ePC matches backpropagation's performance even for deeper models where sPC struggles. Besides practical improvements, our work provides theoretical insight into PC dynamics and establishes a foundation for scaling PC-based learning to deeper architectures on digital hardware and beyond.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2505.20137

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Proposed solutions to predictive coding's scaling problem still require automatic differentiation or global signal propagation.

    “Solutions have been proposed to circumvent this problem [Qi et al., 2025, Innocenti et al., 2025, Goemaere et al., 2025], but they still require automatic differentiation or global signal propagation.”

    From Closed-form predictive coding via hierarchical Gaussian filters (2026) · cites this paper as Goemaere et al., 2025

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

No references could be resolved yet.

Research works printed but not linked: 31

  1. Alonso, N., Krichmar, J., & Neftci, E. (2024). Understand- ing and improving optimization in predictive coding net- works. Proceedings of the AAAI Conference on Artificial Intelligence, 38(10), 10812-10820.
  2. Bogacz, R. (2017). A tutorial on the free-energy framework for modelling perception and learning. Journal of Mathe- matical Psychology, 76, 198-211.
  3. Cohen, G., Afshar, S., Tapson, J., & Van Schaik, A. (2017). EMNIST: Extending MNIST to handwritten letters. Inter- national Joint Conference on Neural Networks (IJCNN), 2921-2926.
  4. Friston, K., & Kiebel, S. (2009). Predictive coding under the free-energy principle. Philosophical transactions of the Royal Society B: Biological sciences, 364(1521), 1211- 1221.
  5. Ha, M. H., Kim, H., Sung, Y., Jo, Y., Kang, M. S., & Lee, S. W. (2026). Stable and scalable deep predictive coding networks with meta-prediction errors. The Fourteenth International Conference on Learning Representations.
  6. He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 770-778.
  7. Hendrycks, D., & Gimpel, K. (2016). Gaussian error linear units (GELUs). arXiv preprint arXiv:1606.08415. · arXiv
  8. Hooker, S. (2021). The hardware lottery. Communications of the ACM, 64(12), 58-65.
  9. Hu, W., Xiao, L., & Pennington, J. (2020). Provable bene- fit of orthogonal initialization in optimizing deep linear networks. International Conference on Learning Repre- sentations.
  10. Innocenti, F., Achour, E. M., & Buckley, C. (2025). µPC: Scaling predictive coding to 100+ layer networks. The Thirty-ninth Annual Conference on Neural Information Processing Systems.
  11. Innocenti, F., Achour, E. M., Singh, R., & Buckley, C. (2024a). Only strict saddles in the energy landscape of predictive coding networks? The Thirty-eighth Annual Conference on Neural Information Processing Systems.
  12. Innocenti, F., Kinghorn, P., Yun-Farmbrough, W., Varona, M. D. L., Singh, R., & Buckley, C. L. (2024b). JPC: Flexible inference for predictive coding networks in jax. arXiv preprint arXiv:2412.03676. · arXiv
  13. Kingma, D. P., & Ba, J. (2014). Adam: A method for stochas- tic optimization. arXiv preprint arXiv:1412.6980. · arXiv
  14. Kingma, D. P., & Welling, M. (2013). Auto-encoding varia- tional Bayes. arXiv preprint arXiv:1312.6114. · arXiv
  15. Lee, K., Pennartz, C. M. A., & Mejias, J. F. (2025). Cor- tical networks with multiple interneuron types generate oscillatory patterns during predictive coding. PLOS Com- putational Biology, 21(9), 1-25.
  16. Loshchilov, I., & Hutter, F. (2019). Decoupled weight de- cay regularization. International Conference on Learning Representations.
  17. Meulemans, A., Zucchet, N., Kobayashi, S., Von Oswald, J., & Sacramento, J. (2022). The least-control principle for local learning at equilibrium. Advances in Neural Information Processing Systems, 35, 33603-33617.
  18. Millidge, B., Salvatori, T., Song, Y., Bogacz, R., & Lukasiewicz, T. (2022a). Predictive coding: Towards a fu- ture of deep learning beyond backpropagation? Proceed- ings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI-22, 5538-5545.
  19. Millidge, B., Seth, A., & Buckley, C. L. (2021). Pre- dictive coding: A theoretical and experimental review. arXiv:2107.12979. · arXiv
  20. Millidge, B., Tang, M., Osanlouy, M., Harper, N. S., & Bo- gacz, R. (2024). Predictive coding networks for temporal prediction. PLOS Computational Biology, 20(4), 1-31.
  21. Millidge, B., Tschantz, A., & Buckley, C. L. (2022b). Pre- dictive coding approximates backprop along arbitrary computation graphs. Neural Computation, 34(6), 1329- 1368.
  22. Oliviers, G., Bogacz, R., & Meulemans, A. (2024). Learning probability distributions of sensory inputs with Monte Carlo predictive coding. bioRxiv, 2024-02.
  23. Pinchetti, L., Qi, C., Lokshyn, O., Emde, C., M'Charrak, A., Tang, M., Frieder, S., Menzat, B., Oliviers, G., Bogacz, R., Lukasiewicz, T., & Salvatori, T. (2025). Benchmarking predictive coding networks -made simple. The Thirteenth International Conference on Learning Representations.
  24. Pinchetti, L., Salvatori, T., Millidge, B., Song, Y., Yordanov, Y., & Lukasiewicz, T. (2022). Predictive coding beyond Gaussian distributions. 36th Conference on Neural Infor- mation Processing Systems.
  25. Salvatori, T., Song, Y., Millidge, B., Xu, Z., Sha, L., Emde, C., Bogacz, R., & Lukasiewicz, T. (2024). Incremental predictive coding: A parallel and fully automatic learning algorithm. International Conference on Learning Repre- sentations.
  26. Simonyan, K., & Zisserman, A. (2014). Very deep convolu- tional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556. · arXiv
  27. Song, Y., Lukasiewicz, T., Xu, Z., & Bogacz, R. (2020). Can the brain do backpropagation? -Exact implemen- tation of backpropagation in predictive coding networks. Advances in Neural Information Processing Systems, 33.
  28. Tschantz, A., Millidge, B., Seth, A. K., & Buckley, C. L. (2023). Hybrid predictive coding: Inferring, fast and slow. PLOS Computational Biology, 19(8), 1-31.
  29. Xiao, H., Rasul, K., & Vollgraf, R. (2017). Fashion-MNIST: A novel image dataset for benchmarking machine learn- ing algorithms. arXiv preprint arXiv:1708.07747. · arXiv
  30. Zahid, U., Guo, Q., & Fountas, Z. (2023). Predictive cod- ing as a neuromorphic alternative to backpropagation: A critical evaluation. Neural Computation, 35(12), 1881- 1909.
  31. Zahid, U., Guo, Q., & Fountas, Z. (2024). Sample as you infer: Predictive coding with Langevin dynamics. Forty- first International Conference on Machine Learning.

Cited by

Sorted by citations

Metadata

Authors
Goemaere, C., Oliviers, G., Bogacz, R., & Demeester, T.
Affiliations
  • Ghent University (BE)
  • University of Oxford (GB)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2025
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2505.20137
arXiv
2505.20137
OpenAlex
W4414588327
Cited by
2
References
0
Citations by year
2026: 2
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • Ghent University Academic Bibliography (Ghent University) (submittedVersion, other-oa)
  • arXiv (Cornell University)
Funders
Universiteit Gent
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
  • Physical Sciences › Engineering › Electrical and Electronic Engineering › Advanced Memory and Neural Computing
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Embodied and Extended Cognition
Keywords
predictive coding, brain-inspired learning, neural network training, gradient computation, signal decay, deep learning architectures, backpropagation alternatives

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review