Glassy Chimeras Could Be Blind to Quantum Speedup: Designing Better Benchmarks for Quantum Annealing Machines
Open accessThe DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Abstract
Recent benchmarking of the computational speedup of quantum ``annealing'' machines of the D-Wave-2 type shows that they do not perform faster than a standard desktop computer. A timely theoretical study of the computational tests used in the benchmarking explains why that may be the case.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Spin Glass Theory and Beyond: An Introduction to the Replica Method and Its Applications
Mezard, M., Parisi, G., & Virasoro, M. (1986) · World Scientific Lecture Notes in Physics · doi:10.1142/0271 · Cited by 3,653
Ising formulations of many NP problems
Lucas, A. (2014) · Frontiers in Physics · doi:10.3389/fphy.2014.00005 · Cited by 2,512
Quantum annealing in the transverse Ising model
Kadowaki, T., & Nishimori, H. (1998) · Physical Review E · doi:10.1103/physreve.58.5355 · Cited by 2,339
Quantum annealing with manufactured spins
Johnson, M. W., Amin, M. H. S., Gildert, S., Lanting, T., Hamze, F., Dickson, N., Harris, R., Berkley, A. J., Johansson, J., Bunyk, P., et al. (2011) · Nature · doi:10.1038/nature10012 · Cited by 1,891
Self-citationInfinite Number of Order Parameters for Spin-Glasses
Parisi, G. (1979) · Physical Review Letters · doi:10.1103/physrevlett.43.1754 · Cited by 1,140
The order parameter for spin glasses: a function on the interval 0-1
Parisi, G. (1980) · Journal of Physics A: Mathematical and General · doi:10.1088/0305-4470/13/3/042 · Cited by 846
Statistical Mechanics of Phase Transitions
Yeomans, J. M. (1992) · doi:10.1093/oso/9780198517290.001.0001 · Cited by 809
Colloquium : Quantum annealing and analog quantum computation
Das, A., & Chakrabarti, B. K. (2008) · Reviews of Modern Physics · doi:10.1103/revmodphys.80.1061 · Cited by 700 · cited 1×
“In analogy to simulated annealing [2] where thermal fluctuations are adiabatically quenched to minimize a cost function, this machine is based on the quantum annealing optimization method [3–11] where quantum fluctuations replace thermal ones.”
MethodDefining and detecting quantum speedup
Rønnow, T. F., Wang, Z., Job, J., Boixo, S., Isakov, S. V., Wecker, D., Martinis, J. M., Lidar, D. A., & Troyer, M. (2014) · Science · doi:10.1126/science.1252319 · Cited by 601
Quantum Annealing of a Disordered Magnet
Brooke, J., Bitko, D., F., T., Rosenbaum, & Aeppli, G. (1999) · Science · doi:10.1126/science.284.5415.779 · Cited by 518
Cited by
Defining and detecting quantum speedup
Rønnow, T. F., Wang, Z., Job, J., Boixo, S., Isakov, S. V., Wecker, D., Martinis, J. M., Lidar, D. A., & Troyer, M. (2014) · Science · doi:10.1126/science.1252319 · Cited by 601
Perspectives of quantum annealing: methods and implementations
Hauke, P., Katzgraber, H. G., Lechner, W., Nishimori, H., & Oliver, W. D. (2020) · Reports on Progress in Physics · doi:10.1088/1361-6633/ab85b8 · Cited by 507
Self-citationPhysics-Inspired Optimization for Quadratic Unconstrained Problems Using a Digital Annealer
Aramon, M., Rosenberg, G., Valiante, E., Miyazawa, T., Tamura, H., & Katzgraber, H. G. (2019) · Frontiers in Physics · doi:10.3389/fphy.2019.00048 · Cited by 442
“As such, spin glasses have been used extensively to benchmark algorithms on off-the-shelf CPUs [43, 44], novel computing technologies such as quantum annealers [18, 20, 45, 46], and coherent Ising machines [32].”
Self-citationWhat is the Computational Value of Finite-Range Tunneling?
Denchev, V. S., Boixo, S., Isakov, S. V., Ding, N., Babbush, R., Smelyanskiy, V., Martinis, J., & Neven, H. (2016) · Physical Review X · doi:10.1103/physrevx.6.031015 · Cited by 399
Quantum annealing for industry applications: introduction and review
Yarkoni, S., Raponi, E., Bäck, T., & Schmitt, S. (2022) · Reports on Progress in Physics · doi:10.1088/1361-6633/ac8c54 · Cited by 341
“Due to their relatively sparse connectivity, Chimera topology QPUs have been found to not be conducive to quantum speedups and advantages for problems formulated directly on Chimera graphs (also known as native problems) [95].”
Hamerly, R., Inagaki, T., McMahon, P. L., Venturelli, D., Marandi, A., Onodera, T., Ng, E., Langrock, C., Inaba, K., Honjo, T., et al. (2019) · Science Advances · doi:10.1126/sciadv.aau0823 · Cited by 328
Phase transitions in a programmable quantum spin glass simulator
Harris, R., Sato, Y., Berkley, A. J., Reis, M., Altomare, F., Amin, M. H., Boothby, K., Bunyk, P., Deng, C., Enderud, C., et al. (2018) · Science · doi:10.1126/science.aat2025 · Cited by 259
A quantum annealing architecture with all-to-all connectivity from local interactions
Lechner, W., Hauke, P., & Zoller, P. (2015) · Science Advances · doi:10.1126/sciadv.1500838 · Cited by 256
Quantum Optimization of Fully Connected Spin Glasses
Venturelli, D., Mandrà, S., Knysh, S., O’Gorman, B., Biswas, R., & Smelyanskiy, V. (2015) · Physical Review X · doi:10.1103/physrevx.5.031040 · Cited by 209
Demonstration of a Scaling Advantage for a Quantum Annealer over Simulated Annealing
Albash, T., & Lidar, D. A. (2018) · Physical Review X · doi:10.1103/physrevx.8.031016 · Cited by 238
Next-Generation Topology of D-Wave Quantum Processors
Boothby, K., Bunyk, P., Raymond, J., & Roy, A. (2020) · arXiv · doi:10.48550/arxiv.2003.00133 · Cited by 125
Benchmarking Quantum Annealing Controls with Portfolio Optimization
Grant, E., Humble, T. S., & Stump, B. (2021) · Physical Review Applied · doi:10.1103/physrevapplied.15.014012 · Cited by 90
“Several experimental efforts have realized quantum annealers [10–12], and application benchmarking of these systems has shown QA is capable of finding the correct result with varying probability of success [13–19].”
BackgroundA deceptive step towards quantum speedup detection
Mandrà, S., & Katzgraber, H. G. (2018) · Quantum Science and Technology · doi:10.1088/2058-9565/aac8b2 · Cited by 82
“From a physical point of view, the DCL problems have another important property which makes them interesting in their own right: By continuously changing the scaling parameter λ, it is possible to modify the critical spin-glass temperature from Tc = 0 (λ ∼ 1) [12], to Tc > 0 (λ 1) [38].”
BackgroundSelf-citationGenome assembly using quantum and quantum-inspired annealing
Boev, A. S., Rakitko, A. S., Usmanov, S. R., Kobzeva, A. N., Popov, I. V., Ilinsky, V. V., Kiktenko, E. O., & Fedorov, A. K. (2021) · Scientific Reports · doi:10.1038/s41598-021-88321-5 · Cited by 63
Towards practical applications in quantum computational biology
Fedorov, A. K., & Gelfand, M. S. (2021) · Nature Computational Science · doi:10.1038/s43588-021-00024-z · Cited by 62
Readiness of Quantum Optimization Machines for Industrial Applications
Perdomo-Ortiz, A., Feldman, A., Ozaeta, A., Isakov, S. V., Zhu, Z., O’Gorman, B., Katzgraber, H. G., Diedrich, A., Neven, H., de Kleer, J., et al. (2019) · Physical Review Applied · doi:10.1103/physrevapplied.12.014004 · Cited by 62
Self-citationQuinton, F. A., Myhr, P. A. S., Barani, M., Crespo del Granado, P., & Zhang, H. (2025) · Scientific Reports · doi:10.1038/s41598-025-96220-2 · Cited by 46
Hybrid Quantum Annealing for Larger-than-QPU Lattice-structured Problems
Raymond, J., Stevanovic, R., Bernoudy, W., Boothby, K., McGeoch, C. C., Berkley, A. J., Farré, P., Pasvolsky, J., & King, A. D. (2023) · ACM Transactions on Quantum Computing · doi:10.1145/3579368 · Cited by 39
“Some classical heuristics have demonstrated strong performance beyond annealing in lower dimensions, including 3D [38, 41], and additional methods have been proposed for leveraging structure in QPU architectures, in particular the cellular-level structure of Chimera graphs [4, 16, 36, 37].”
Algorithm engineering for a quantum annealing platform
King, A. D., & McGeoch, C. C. (2014) · arXiv · doi:10.48550/arxiv.1410.2628 · Cited by 39
The pitfalls of planar spin-glass benchmarks: raising the bar for quantum annealers (again)
Mandrà, S., Katzgraber, H. G., & Thomas, C. (2017) · Quantum Science and Technology · doi:10.1088/2058-9565/aa7877 · Cited by 35
Self-citationFair sampling of ground-state configurations of binary optimization problems
Zhu, Z., Ochoa, A. J., & Katzgraber, H. G. (2019) · Physical Review E · doi:10.1103/physreve.99.063314 · Cited by 17
Self-citation
Metadata
- Authors
- Katzgraber, H. G., Hamze, F., & Andrist, R. S.
- Venue
- Physical Review X
- Publisher
- American Physical Society (APS)
- Published
- 2014-04-10
- Volume
- 4
- Issue
- 2
- ISSN
- 2160-3308 (ISSN-L)
- OpenAlex
- W2026087412
- Cited by
- 140
- References
- 12
- FWCI
- 20.47
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.