Projective simulation for artificial intelligence
Open accessThe DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Abstract
We propose a model of a learning agent whose interaction with the environment is governed by a simulation-based projection, which allows the agent to project itself into future situations before it takes real action. Projective simulation is based on a random walk through a network of clips, which are elementary patches of episodic memory. The network of clips changes dynamically, both due to new perceptual input and due to certain compositional principles of the simulation process. During simulation, the clips are screened for specific features which trigger factual action of the agent. The scheme is different from other, computational, notions of simulation, and it provides a new element in an embodied cognitive science approach to intelligent action and learning. Our model provides a natural route for generalization to quantum-mechanical operation and connects the fields of reinforcement learning and quantum computation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Godsil, C., & Royle, G. (2001) · Graduate Texts in Mathematics · doi:10.1007/978-1-4613-0163-9 · Cited by 8,154
Simulating physics with computers
Feynman, R. P. (1982) · International Journal of Theoretical Physics · doi:10.1007/bf02650179 · Cited by 7,691
Cognitive maps in rats and men
Tolman, E. C. (1948) · Psychological Review · doi:10.1037/h0061626 · Cited by 6,641
Quantum theory, the Church–Turing principle and the universal quantum computer
Deutsch, D. (1985) · Proceedings of the Royal Society of London. A. Mathematical and Physical Sciences · doi:10.1098/rspa.1985.0070 · Cited by 4,597
Quantum Mechanics Helps in Searching for a Needle in a Haystack
Grover, L. K. (1997) · Physical Review Letters · doi:10.1103/physrevlett.79.325 · Cited by 4,306
(2007) · Cognitive Psychology · doi:10.4135/9781446212967.n15 · Cited by 3,749
Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning
Sutton, R. S., Precup, D., & Singh, S. (1999) · Artificial Intelligence · doi:10.1016/s0004-3702(99)00052-1 · Cited by 3,119
Self-improving reactive agents based on reinforcement learning, planning and teaching
Lin, L.-J. (1992) · Machine Learning · doi:10.1007/bf00992699 · Cited by 1,564 · cited 7×
“This concerns in particular the notion of experience replay by Lin [17] and recent ideas by Sutton et al. [21].”
BackgroundMethodResultInfluentialQuantum random walks: An introductory overview
Kempe, J. (2003) · Contemporary Physics · doi:10.1080/00107151031000110776 · Cited by 1,413
An open-system quantum simulator with trapped ions
Barreiro, J. T., Müller, M., Schindler, P., Nigg, D., Monz, T., Chwalla, M., Hennrich, M., Roos, C. F., Zoller, P., & Blatt, R. (2011) · Nature · doi:10.1038/nature09801 · Cited by 1,100 · cited 1×
“Note that quantum random walk processes similar to (26), with engineered quantum many-body interactions, have recently been realized in the context of dissipation-driven quantum simulation with trapped ions [44].”
BackgroundWeimer, H., Müller, M., Lesanovsky, I., Zoller, P., & Büchler, H. P. (2010) · Nature Physics · doi:10.1038/nphys1614 · Cited by 867
Episodic Simulation of Future Events: Concepts, Data, and Applications
Schacter, D. L., Addis, D. R., & Buckner, R. L. (2008) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1440.001 · Cited by 840
Aharonov, D., Ambainis, A., Kempe, J., & Vazirani, U. (2001) · Proceedings of the thirty-third annual ACM symposium on Theory of computing · doi:10.1145/380752.380758 · Cited by 654
Model-based learning for mobile robot navigation from the dynamical systems perspective
Tani, J. (1996) · IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) · doi:10.1109/3477.499793 · Cited by 335
Quantum walks, quantum gates, and quantum computers
Hines, A. P., & Stamp, P. C. E. (2007) · Physical Review A · doi:10.1103/physreva.75.062321 · Cited by 54 · cited 2×
“The Hamiltonian used in (21) can be generalized to so-called composite walks [42] that include further degrees of freedom associated with a given transition, which could be used to include the emotion tags (see Section IV) into the quantum mode, as well as to implement discrete quantum walks using…”
BackgroundMethod
Cited by
Machine learning & artificial intelligence in the quantum domain: a review of recent progress
Dunjko, V., & Briegel, H. J. (2018) · Reports on Progress in Physics · doi:10.1088/1361-6633/aab406 · Cited by 1,055
Self-citationDiscovering Physical Concepts with Neural Networks
Iten, R., Metger, T., Wilming, H., del Rio, L., & Renner, R. (2020) · Physical Review Letters · doi:10.1103/physrevlett.124.010508 · Cited by 458
Quantum-Enhanced Machine Learning
Dunjko, V., Taylor, J. M., & Briegel, H. J. (2016) · Physical Review Letters · doi:10.1103/physrevlett.117.130501 · Cited by 460
“Q-Learning [14, 26], Policy iteration [34] or the more recent Projective Simulation [35, 36] model), together with the maze environment (with a unique winning path) do form luck-favoring pairs for all histories, so Theorem 1 applies.”
BackgroundSelf-citationVariational Quantum Circuits for Deep Reinforcement Learning
Chen, S. Y.-C., Yang, C.-H. H., Qi, J., Chen, P.-Y., Ma, X., & Goan, H.-S. (2020) · IEEE Access · doi:10.1109/access.2020.3010470 · Cited by 452
Harnessing Disordered-Ensemble Quantum Dynamics for Machine Learning
Fujii, K., & Nakajima, K. (2017) · Physical Review Applied · doi:10.1103/physrevapplied.8.024030 · Cited by 337
Metadata
- Authors
- Briegel, H. J., & De las Cuevas, G.
- Venue
- Scientific Reports
- Published
- 2012-05-15
- Volume
- 2
- Issue
- 1
- Pages
- 400-400
- ISSN
- 2045-2322 (ISSN-L)
- OpenAlex
- W2079905842
- Cited by
- 151
- References
- 15
- FWCI
- 5.07
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.