Memory-Limited Partially Observable Stochastic Control and Its Mean-Field Control Approach
Open accessAbstract
Control problems with incomplete information and memory limitation appear in many practical situations. Although partially observable stochastic control (POSC) is a conventional theoretical framework that considers the optimal control problem with incomplete information, it cannot consider memory limitation. Furthermore, POSC cannot be solved in practice except in special cases. In order to address these issues, we propose an alternative theoretical framework, memory-limited POSC (ML-POSC). ML-POSC directly considers memory limitation as well as incomplete information, and it can be solved in practice by employing the technique of mean-field control theory. ML-POSC can generalize the linear-quadratic-Gaussian (LQG) problem to include memory limitation. Because estimation and control are not clearly separated in the LQG problem with memory limitation, the Riccati equation is modified to the partially observable Riccati equation, which improves estimation as well as control. Furthermore, we demonstrate the effectiveness of ML-POSC for a non-LQG problem by comparing it with the local LQG approximation.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Planning and acting in partially observable stochastic domains
Kaelbling, L. P., Littman, M. L., & Cassandra, A. R. (1998) · Artificial Intelligence · doi:10.1016/s0004-3702(98)00023-x · Cited by 4,149 · see citing works
Controlled Markov Processes and Viscosity Solutions
Fleming, W. H., & Soner, H. M. (2005) · Springer eBooks · doi:10.1007/0-387-31071-1 · Cited by 2,794 · see citing works
Yong, J., & Zhou, X. Y. (1999) · doi:10.1007/978-1-4612-1466-3 · Cited by 1,243 · see citing works
A Concise Introduction to Decentralized POMDPs
Oliehoek, F. A., & Amato, C. (2016) · SpringerBriefs in Intelligent Systems · doi:10.1007/978-3-319-28929-8 · Cited by 968 · see citing works
A survey of convergence results on particle filtering methods for practitioners
Crisan, D., & Doucet, A. (2002) · IEEE Transactions on Signal Processing · doi:10.1109/78.984773 · Cited by 919 · see citing works
Mean Field Games and Mean Field Type Control Theory
Bensoussan, A., Frehse, J., & Yam, S. C. P. (2013) · SpringerBriefs in mathematics · doi:10.1007/978-1-4614-8508-7 · Cited by 789 · see citing works
Stochastic Control of Partially Observable Systems
Bensoussan, A. (1992) · Cambridge University Press eBooks · doi:10.1017/cbo9780511526503 · Cited by 674 · see citing works
Fundamentals of Stochastic Filtering
Bain, A., & Crisan, D. (2008) · Applications of mathematics · doi:10.1007/978-0-387-76896-0 · Cited by 659 · see citing works
Numerical Methods for Stochastic Control Problems in Continuous Time
Kushner, H. J., & Dupuis, P. G. (1992) · Springer eBooks · doi:10.1007/978-1-4684-0441-8 · Cited by 632 · see citing works
Probabilistic Theory of Mean Field Games with Applications II
Carmona, R. A., & Delarue, F. (2018) · Probability theory and stochastic modelling · doi:10.1007/978-3-319-56436-4 · Cited by 575 · see citing works
Probabilistic Theory of Mean Field Games with Applications I
Carmona, R. A., & Delarue, F. (2018) · Probability theory and stochastic modelling · doi:10.1007/978-3-319-58920-6 · Cited by 516 · see citing works
Decentralized Stochastic Control with Partial History Sharing: A Common Information Approach
Nayyar, A., Mahajan, A., & Teneketzis, D. (2013) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2013.2239000 · Cited by 279 · see citing works
Ruthotto, L., Osher, S., Li, W., Nurbekyan, L., & Fung, S. W. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1922204117 · Cited by 205 · see citing works
Learning Finite-State Controllers for Partially Observable EnvironmentsPreprint
Meuleau, N., Peshkin, L., Kim, K.-E., & Kaelbling, L. P. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.6721 · Cited by 187 · see citing works
Li, W., & Todorov, E. V. (2007) · International Journal of Control · doi:10.1080/00207170701364913 · Cited by 176 · see citing works
Stochastic Optimal Control in Infinite Dimension
Fabbri, G., Gozzi, F., & Święch, A. (2017) · Probability theory and stochastic modelling · doi:10.1007/978-3-319-53067-3 · Cited by 167 · see citing works
Solving POMDPs by Searching the Space of Finite PoliciesPreprint
Meuleau, N., Kim, K.-E., Kaelbling, L. P., & Cassandra, A. R. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.6720 · Cited by 146 · see citing works
Bounded policy iteration for decentralized POMDPs
Bernstein, D. S., Hansen, E. A., & Zilberstein, S. (2005) · Cited by 142 · see citing works
The Master equation in mean field theory
Bensoussan, A., Frehse, J., & Yam, S. C. P. (2015) · Journal de Mathématiques Pures et Appliquées · doi:10.1016/j.matpur.2014.11.005 · Cited by 138 · see citing works
Bellman equation and viscosity solutions for mean-field stochastic control problem
Pham, H., & Wei, X. (2017) · ESAIM Control Optimisation and Calculus of Variations · doi:10.1051/cocv/2017019 · Cited by 126 · see citing works
A Fully Discrete Semi-Lagrangian Scheme for a First Order Mean Field Game Problem
Carlini, E., & Silva, F. J. (2014) · SIAM Journal on Numerical Analysis · doi:10.1137/120902987 · Cited by 103 · see citing works
A survey of numerical methods for nonlinear filtering problems
Budhiraja, A., Chen, L., & Lee, C. (2006) · Physica D Nonlinear Phenomena · doi:10.1016/j.physd.2006.08.015 · Cited by 97 · see citing works
Policy Iteration for Decentralized Control of Markov Decision Processes
Bernstein, D. S., Amato, C., Hansen, E. A., & Shlomo Zilberstein, B. (2009) · Journal of Artificial Intelligence Research · doi:10.1613/jair.2667 · Cited by 88 · see citing works
Hamilton-Jacobi Equations: Approximations, Numerical Analysis and Applications
Achdou, Y., Barles, G., Ishii, H., et al. (2013) · Lecture notes in mathematics · doi:10.1007/978-3-642-36433-4 · Cited by 83 · see citing works
On the interpretation of the Master Equation
Bensoussan, A., Frehse, J., & Yam, S. C. P. (2017) · Stochastic Processes and their Applications · doi:10.1016/j.spa.2016.10.004 · Cited by 76 · see citing works
Lin, A. T., Fung, S. W., Li, W., Nurbekyan, L., & Osher, S. (2020) · arXiv (Cornell University) · doi:10.1073/pnas.2024713118 · Cited by 72 · see citing works
Mean Field Games and Applications: Numerical Aspects
Achdou, Y., & Laurière, M. (2020) · Lecture notes in mathematics · doi:10.1007/978-3-030-59837-2_4 · Cited by 72 · see citing works
An Improved Policy Iteration Algorithm for Partially Observable MDPs
Hansen, E. A. (1997) · Cited by 68 · see citing works
Stochastic Control Theory: Dynamic Programming Principle
Nisio, M. (2015) · Probability Theory and Stochastic Modelling · doi:10.1007/978-4-431-55123-2 · Cited by 67 · see citing works
Finite Difference Methods for Mean Field Games
Achdou, Y. (2013) · Lecture notes in mathematics · doi:10.1007/978-3-642-36433-4_1 · Cited by 61 · see citing works
Numerical methods for mean field games and mean field type control
Laurière, M. (2021) · Proceedings of symposia in applied mathematics · doi:10.1090/psapm/078/06 · Cited by 57 · see citing works
Estimation and Control of Dynamical Systems
Bensoussan, A. (2018) · Interdisciplinary Applied Mathematics · doi:10.1007/978-3-319-75456-7 · Cited by 51 · see citing works
Achdou, Y., Cardaliaguet, P., Delarue, F., et al. (2020) · Lecture notes in mathematics · doi:10.1007/978-3-030-59837-2 · Cited by 50 · see citing works
Optimizing Memory-Bounded Controllers for Decentralized POMDPsPreprint
Amato, C., Bernstein, D. S., & Zilberstein, S. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.5258 · Cited by 48 · see citing works
Finite-State Controllers Based on Mealy Machines for Centralized and Decentralized POMDPs
Amato, C., Bonet, B., & Zilberstein, S. (2010) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v24i1.7748 · Cited by 43 · see citing works
Charalambous, C. D., & Ahmed, N. U. (2016) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2016.2575818 · Cited by 39 · see citing works
An Iterative Optimal Control and Estimation Design for Nonlinear Stochastic System
Li, W., & Todorov, E. (2006) · Proceedings of the IEEE Conference on Decision & Control, including the Symposium on Adaptive Processes · doi:10.1109/cdc.2006.377485 · Cited by 32 · see citing works
Dynamic Programming for Mean-Field Type Control
Laurière, M., & Pironneau, O. (2015) · Journal of Optimization Theory and Applications · doi:10.1007/s10957-015-0785-x · Cited by 27 · see citing works
Connection between the Bacterial Chemotactic Network and Optimal Filtering
Nakamura, K., & Kobayashi, T. J. (2021) · Physical Review Letters · doi:10.1103/physrevlett.126.128102 · Cited by 19 · see citing works
Self-citationChemotaxis emerges as the optimal solution to cooperative search games
Pezzotta, A., Adorisio, M., & Celani, A. (2018) · Physical Review E · doi:10.1103/physreve.98.042401 · Cited by 17 · see citing works
Optimal sensing and control of run-and-tumble chemotaxis
Nakamura, K., & Kobayashi, T. J. (2022) · Physical Review Research · doi:10.1103/physrevresearch.4.013120 · Cited by 17 · see citing works
Self-citationMinimum-information LQG control part I: Memoryless controllers
Fox, R., & Tishby, N. (2016) · Proceedings of the IEEE Conference on Decision & Control, including the Symposium on Adaptive Processes · doi:10.1109/cdc.2016.7799131 · Cited by 16 · see citing works
Charalambous, C. D., & Ahmed, N. U. (2017) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2017.2758838 · Cited by 13 · see citing works
Semi-Lagrangian schemes for mean field game models
Carlini, E., & Silva, F. J. (2013) · Proceedings of the IEEE Conference on Decision & Control, including the Symposium on Adaptive Processes · doi:10.1109/cdc.2013.6760358 · Cited by 11 · see citing works
Minimum-information LQG control Part II: Retentive controllers
Fox, R., & Tishby, N. (2016) · Proceedings of the IEEE Conference on Decision & Control, including the Symposium on Adaptive Processes · doi:10.1109/cdc.2016.7799130 · Cited by 11 · see citing works
Tottori, T., & Kobayashi, T. J. (2023) · Entropy · doi:10.3390/e25020208 · Cited by 11 · see citing works
Self-citationMean field approach to stochastic control with partial information
Bensoussan, A., & Yam, S. C. P. (2021) · ESAIM Control Optimisation and Calculus of Variations · doi:10.1051/cocv/2021085 · Cited by 10 · see citing works
Optimal collision avoidance in swarms of active Brownian particles
Borra, F., Cencini, M., & Celani, A. (2021) · Journal of Statistical Mechanics Theory and Experiment · doi:10.1088/1742-5468/ac12c6 · Cited by 10 · see citing works
A semi-Lagrangian scheme for a degenerate second order mean field game system
Carlini, E., & Silva, F. J. (2015) · Discrete and Continuous Dynamical Systems · doi:10.3934/dcds.2015.35.4269 · Cited by 5 · see citing works
Forward and Backward Bellman Equations Improve the Efficiency of the EM Algorithm for DEC-POMDP
Tottori, T., & Kobayashi, T. J. (2021) · Entropy · doi:10.3390/e23050551 · Cited by 3 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 2
- [17]Hansen E.A. Solving POMDPs by Searching in Policy Space Proceedings of the Fourteenth Conference on Uncertainty in Artificial Intelligence Madison, WI, USA 24–26 July 1998 211 219
- [21]Poupart P. Boutilier C. Bounded Finite State Controllers Advances in Neural Information Processing Systems MIT Press Cambridge, MA, USA 2003 Volume 16
Cited by
Sorted by citations
Schnyder, S. K., Molina, J. J., Yamamoto, R., & Turner, M. S. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2409362122 · Cited by 12 · see citing works
Tottori, T., & Kobayashi, T. J. (2023) · Entropy · doi:10.3390/e25020208 · Cited by 11 · see citing works
Self-citationDecentralized Stochastic Control with Finite-Dimensional Memories: A Memory Limitation Approach
Tottori, T., & Kobayashi, T. J. (2023) · Entropy · doi:10.3390/e25050791 · Cited by 7 · see citing works
Self-citationTottori, T., & Kobayashi, T. J. (2025) · Physical Review Research · doi:10.1103/gvl6-cvby · Cited by 6 · see citing works
Self-citationResource limitations induce phase transitions in biological information processing
Tottori, T., & Kobayashi, T. J. (2025) · Physical Review Research · doi:10.1103/nz4j-tyv1 · Cited by 3 · see citing works
Self-citationTheoretical Analysis of Resource-Induced Phase Transitions in Estimation Strategies
Tottori, T., & Kobayashi, T. J. (2026) · Physical Review Letters · doi:10.1103/5ynb-7k4v
Self-citation
- Authors
- Tottori, T., & Kobayashi, T. J.
- Affiliations
- The University of Tokyo (JP)
- Institute of Industrial Science, The University of Tokyo (JP)
- Corresponding author
- Tottori, Takehiro
- Venue
- Entropy
- Publisher
- MDPI AG
- Published
- 2022-11-03
- Volume
- 24
- Issue
- 11
- Pages
- 1599
- ISSN
- 1099-4300 (ISSN-L)
- PMC
- PMC9689590
- OpenAlex
- W4308609397
- Cited by
- 11
- References
- 50
- Citations by year
- 2026: 1 · 2025: 4 · 2024: 1 · 2023: 5
- FWCI
- 1.83
- Type
- journal article
- Language
- en
- Open access
- gold
- dx.doi.org
- Licence
- Locations
- Entropy (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- arXiv (Cornell University) (cc-by)
- Funders
- JSPS Research Fellowship (21J20436, 19H05799, JPMJCR2011)
- JSPS KAKENHI (21J20436, 19H05799, JPMJCR2011)
- JST CREST (21J20436, 19H05799, JPMJCR2011)
- Topics
- Social Sciences › Economics, Econometrics and Finance › Finance › Stochastic processes and financial applications
- Physical Sciences › Engineering › Control and Systems Engineering › Advanced Control Systems Optimization
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Adaptive Dynamic Programming Control
- Keywords
- mean-field control, incomplete information, memory constraints, LQG control
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
