Advances in quantum reinforcement learning
Open accessThe DOI matches its Crossref record; 26 of the 28 references it cites were found in a registry or an index.
Abstract
In recent times, there has been much interest in quantum enhancements of machine learning, specifically in the context of data mining and analysis. Reinforcement learning, an interactive form of learning, is, in turn, vital in artificial intelligence-type applications. Also in this case, quantum mechanics was shown to be useful, in certain instances. Here, we elucidate these results, and show that quantum enhancements can be achieved in a new setting: the setting of learning models which learn how to improve themselves - that is, those that metalearn. While not all learning models meta-learn, all non-trivial models have the potential of being "lifted", enhanced, to meta-learning models. Our results show that also such models can be quantum-enhanced to make even better learners. In parallel, we address one of the bottlenecks of current quantum reinforcement learning approaches: the need for so-called oracularized variants of task environments. Here we elaborate on a method which realizes these variants, with minimal changes in the setting, and with no corruption of the operative specification of the environments. This result may be important in near-term experimental demonstrations of quantum reinforcement learning.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · cited 2×
“stering tasks, more generally referred to as supervised and unsupervised learning, form two out of three canonical branches of ML. These learning modes are complemented by reinforcement learning (RL) [5]. RL is central to some of the most celebrated recent successes in ML (e.g. a computer beating a human champion in t…”
BackgroundReinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612 · cited 2×
“Exactly this is done in [6, 8], for the purpose of improving computational complexity, with great success, as the improvement can be exponential.”
BackgroundArtificial intelligence: A modern approach
Nilsson, N. J. (1996) · Artificial Intelligence · doi:10.1016/0004-3702(96)00007-0 · Cited by 10,583
Barto, A. G. (1998) · IFAC Proceedings Volumes · doi:10.1016/s1474-6670(17)38315-5 · Cited by 2,976
Quantum Support Vector Machine for Big Data Classification
Rebentrost, P., Mohseni, M., & Lloyd, S. (2014) · Physical Review Letters · doi:10.1103/physrevlett.113.130503 · Cited by 1,951
Quantum amplitude amplification and estimation
Brassard, G., Høyer, P., Mosca, M., & Tapp, A. (2002) · Contemporary Mathematics · doi:10.1090/conm/305/05215 · Cited by 1,680 · cited 2×
“With access to the instatiations, the agent can execute amplitude amplification [15] to obtain an example of a rewarding sequence of actions more efficiently.”
MethodThe Lack of A Priori Distinctions Between Learning Algorithms
Wolpert, D. H. (1996) · Neural Computation · doi:10.1162/neco.1996.8.7.1341 · Cited by 1,647
Giovannetti, V., Lloyd, S., & Maccone, L. (2008) · Physical Review Letters · doi:10.1103/physrevlett.100.160501 · Cited by 935 · cited 1×
“Such a memory can easily be represented as a reversible process, and consequently as a quantum operation, a unitary map, performing | i (cid:105) | 0 qMem → | i (cid:105) | x i . can be implemented In [11] it was shown how such a unitary efficiently.”
BackgroundQuantum algorithms for supervised and unsupervised machine learning
Lloyd, S., Mohseni, M., & Rebentrost, P. (2013) · arXiv · doi:10.48550/arxiv.1307.0411 · Cited by 542 · cited 1×
“For example, quantum algorithms which can be used for efficient classification and clustering [1], [2], [3], [4], have been proposed.”
MethodQuantum Amplitude Amplification and Estimation
Gilles Brassard, A., Høyer, P., Mosca, M., & Tapp, A. (2000) · doi:10.48550/arxiv.quant-ph/0005055 · Cited by 514 · cited 2×
“With access to the instatiations, the agent can execute amplitude amplification [15] to obtain an example of a rewarding sequence of actions more efficiently.”
MethodQuantum Machine Learning: What Quantum Computing Means to Data Mining
Wittek, P. (2014) · Cited by 481
Quantum-Enhanced Machine Learning
Dunjko, V., Taylor, J. M., & Briegel, H. J. (2016) · Physical Review Letters · doi:10.1103/physrevlett.117.130501 · Cited by 460 · cited 8×
“In [12] it was shown that a quantum analog, which allows for provable learning improvements, does exist.”
BackgroundInfluentialSelf-citationChiribella, G., D’Ariano, G. M., & Perinotti, P. (2008) · Physical Review Letters · doi:10.1103/physrevlett.101.060401 · Cited by 381 · cited 1×
“In the most general setting, both the learner and the environment are maps with memory (often called quantum combs [14]), and classical RL corresponds to the case where the maps are themselves classical.”
BackgroundA Quantum Algorithm for Finding the Minimum
Durr, C., & Hoyer, P. (1996) · arXiv · doi:10.48550/arxiv.quant-ph/9607014 · Cited by 359 · cited 3×
“Since here we wish to maintain the presentation model-independent (albeit, at the cost of its efficiency) we approach this, for now, using the generic black-box, discrete optimization techniques such as [18].”
MethodPrediction by linear regression on a quantum computer
Schuld, M., Sinayskiy, I., & Petruccione, F. (2016) · Physical Review A · doi:10.1103/physreva.94.022342 · Cited by 274 · cited 1×
“Having access to such an object often allows for the efficient encoding of data into amplitudes, which is at the basis of many exponential quantum improvements [4], [21], or directly with the purpose of quadratic improvements [3].”
BackgroundQuantum algorithms for topological and geometric analysis of data
Lloyd, S., Garnerone, S., & Zanardi, P. (2016) · Nature Communications · doi:10.1038/ncomms10138 · Cited by 241 · cited 1×
“Note, in such settings, overall improvements are only attainable assuming such a memory has already been “loaded” with the data-set off-line, barring few exceptions [22].”
BackgroundQuantum speed-up for unsupervised learning
Aïmeur, E., Brassard, G., & Gambs, S. (2013) · Machine Learning · doi:10.1007/s10994-012-5316-5 · Cited by 214 · cited 2×
“object often allows for the efficient encoding of data into amplitudes, which is at the basis of many exponential quantum improvements [4], [21], or directly with the purpose of quadratic improvements [3]. The memory can be thought of as a black-box oracle and quantum-parallel access allows for query-complexity improve…”
BackgroundMethodQuantum gradient descent and Newton’s method for constrained polynomial optimization
Rebentrost, P., Schuld, M., Wossnig, L., Petruccione, F., & Lloyd, S. (2019) · New Journal of Physics · doi:10.1088/1367-2630/ab2a9e · Cited by 196 · cited 3×
“In this case one can, for instance, use “quantum-improved” gradient descent methods [20], or use “completely quantum” gradient descent techniques [19].”
BackgroundMethodQuantum Speedup for Active Learning Agents
Paparo, G. D., Dunjko, V., Makmal, A., Martin-Delgado, M. A., & Briegel, H. J. (2014) · Physical Review X · doi:10.1103/physrevx.4.031002 · Cited by 192 · cited 1×
“Few works do exist: for instance in [8] it was suggested QIP may yield new ways to evaluate policies, and in [9] it was proven that the computational complexity of a particular model, Projective Simulation [10], can be quadratically reduced.”
BackgroundSelf-citationProjective simulation for artificial intelligence
Briegel, H. J., & De las Cuevas, G. (2012) · Scientific Reports · doi:10.1038/srep00400 · Cited by 151 · cited 3×
“Few works do exist: for instance in [8] it was suggested QIP may yield new ways to evaluate policies, and in [9] it was proven that the computational complexity of a particular model, Projective Simulation [10], can be quadratically reduced.”
BackgroundSelf-citationFast Quantum Algorithm for Numerical Gradient Estimation
Jordan, S. P. (2005) · Physical Review Letters · doi:10.1103/physrevlett.95.050501 · Cited by 117 · cited 2×
“In this case one can, for instance, use “quantum-improved” gradient descent methods [20], or use “completely quantum” gradient descent techniques [19].”
BackgroundMethodThe Bayesian Learner is Optimal for Noisy Binary Search (and Pretty Good for Quantum as Well)
Ben-Or, M., & Hassidim, A. (2008) · Annual Symposium on Foundations of Computer Science · doi:10.1109/focs.2008.58 · Cited by 85
Quantum Reinforcement Learning
Dong, D., Chen, C., & Chen, Z. (2005) · Lecture notes in computer science · doi:10.1007/11539117_97 · Cited by 44
Meta-learning within Projective Simulation
Makmal, A., Melnikov, A. A., Dunjko, V., & Briegel, H. J. (2016) · IEEE Access · doi:10.1109/access.2016.2556579 · Cited by 38 · cited 3×
“For example, in [17], the meta-parameter – the “forgetting” parameter, but the details are not relevant here – of the PS model is, is optimized.”
BackgroundMethodResultSelf-citationFramework for learning agents in quantum environmentsPreprint
Dunjko, V., Taylor, J. M., & Briegel, H. J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.08482 · Cited by 31 · cited 6×
“6Note, the capacity that the environment can be reversed implies either a deterministic environment, or that, in the register hijacking phase [13], the agent can temporarily access the register purifying the random choices of the environment.”
BackgroundInfluentialSelf-citation
Cited by
Noisy intermediate-scale quantum algorithms
Bharti, K., Cervera-Lierta, A., Kyaw, T. H., Haug, T., Alperin-Lea, S., Anand, A., Degroote, M., Heimonen, H., Kottmann, J. S., Menke, T., et al. (2022) · Reviews of Modern Physics · doi:10.1103/revmodphys.94.015004 · Cited by 2,231
Machine learning & artificial intelligence in the quantum domain: a review of recent progress
Dunjko, V., & Briegel, H. J. (2018) · Reports on Progress in Physics · doi:10.1088/1361-6633/aab406 · Cited by 1,055
Self-citationQuantum Machine Learning for 6G Communication Networks: State-of-the-Art and Vision for the Future
Nawaz, S. J., Sharma, S. K., Wyne, S., Patwary, M. N., & Asaduzzaman, M. (2019) · IEEE Access · doi:10.1109/access.2019.2909490 · Cited by 658
“Advances in quantum-powered reinforcement learning have been discussed in [175]; where a solution to the bottleneck of required oracularized variants of task environments, has also been proposed.”
BackgroundVariational Quantum Circuits for Deep Reinforcement Learning
Chen, S. Y.-C., Yang, C.-H. H., Qi, J., Chen, P.-Y., Ma, X., & Goan, H.-S. (2020) · IEEE Access · doi:10.1109/access.2020.3010470 · Cited by 452
“A general review of the field quantum machine learning can refer to [35] and [36].”
BackgroundThe prospects of quantum computing in computational molecular biology
Outeiral, C., Strahm, M., Shi, J., Morris, G. M., Benjamin, S. C., & Deane, C. M. (2021) · WIREs Computational Molecular Science · doi:10.1002/wcms.1481 · Cited by 232
“Reinforcement learning O N ð Þ O ffiffiffiffi N p No [106,107]”
Background
Metadata
- Authors
- Dunjko, V., Taylor, J. M., & Briegel, H. J.
- Affiliations
- Max Planck Institute of Quantum Optics (DE)
- National Institute of Standards and Technology (US)
- University of Konstanz (DE)
- Venue
- 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC)
- Publisher
- IEEE
- Published
- 2017-10
- Pages
- 282-287
- ISSN
- 1062-922X (ISSN-L)
- OpenAlex
- W2773252567
- Cited by
- 81
- References
- 28
- Citations by year
- 2026: 3 · 2025: 13 · 2024: 15 · 2023: 11 · 2022: 9 · 2021: 15 · 2020: 7 · 2019: 6 · 2018: 2
- FWCI
- 1.87
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Max Planck Digital Library (submittedVersion)
- MPG.PuRe (Max Planck Society) (submittedVersion)
- arXiv (Cornell University)
- Bulletin of the American Physical Society
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Quantum Computing Algorithms and Architecture
- Physical Sciences › Physics and Astronomy › Atomic and Molecular Physics, and Optics › Quantum Information and Cryptography
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Keywords
- quantum reinforcement learning, meta-learning, quantum machine learning, task environment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.