The hardware lottery
Open accessAbstract
After decades of incentivizing the isolation of hardware, software, and algorithm development, the catalysts for closer collaboration are changing the paradigm.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The success of AI models can depend on the compatibility of their design with available computational architectures (the 'hardware lottery'), further contributing to convergent trends.
“Additionally, the “hardware lottery” (Hooker, 2021) suggests that the success of AI models can also depend on the compatibility of their design with available computational architectures, further contributing to convergent trends.”
From The Platonic Representation Hypothesis (2024) · cites this paper as Hooker, 2021
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Backpropagation Applied to Handwritten Zip Code Recognition
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.4.541 · Cited by 11,891 · see citing works
McClelland, J. L., McNaughton, B. L., & O'Reilly, R. C. (1995) · Psychological Review · doi:10.1037/0033-295x.102.3.419 · Cited by 5,256 · see citing works
Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning
Szegedy, C., Ioffe, S., Vanhoucke, V., & Alemi, A. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11231 · Cited by 4,636 · see citing works
Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem
McCloskey, M., & Cohen, N. J. (1989) · Psychology of Learning and Motivation · doi:10.1016/s0079-7421(08)60536-8 · Cited by 3,770 · see citing works
All-optical machine learning using diffractive deep neural networks
Lin, X., Rivenson, Y., Yardimci, N. T., Veli, M., Luo, Y., Jarrahi, M., & Ozcan, A. (2018) · Science · doi:10.1126/science.aat8084 · Cited by 2,568 · see citing works
When and where do we apply what we learn?: A taxonomy for far transfer
Barnett, S. M., & Ceci, S. J. (2002) · Psychological Bulletin · doi:10.1037/0033-2909.128.4.612 · Cited by 2,076 · cited 1× · see citing works
“Our brains are able to fine-tune and retain skills across our lifetime (Benna & Fusi, 2016; Bremner et al., 2013; Stein et al., 2004; Tani & Press, 2016; Gallistel & King, 2009; Tulving, 2002; Barnett & Ceci, 2002).”
BackgroundAutomated empirical optimizations of software and the ATLAS project
Clint Whaley, R., Petitet, A., & Dongarra, J. J. (2001) · Parallel Computing · doi:10.1016/s0167-8191(00)00087-9 · Cited by 1,317 · see citing works
Equivalent-accuracy accelerated neural-network training using analogue memory
Ambrogio, S., Narayanan, P., Tsai, H., Shelby, R. M., Boybat, I., di Nolfo, C., Sidler, S., Giordano, M., Bodini, M., Farinha, N. C. P., et al. (2018) · Nature · doi:10.1038/s41586-018-0180-5 · Cited by 1,151 · see citing works
Validating quantum computers using randomized model circuits
Cross, A. W., Bishop, L. S., Sheldon, S., Nation, P. D., & Gambetta, J. M. (2019) · Physical Review A · doi:10.1103/physreva.100.032328 · Cited by 704 · see citing works
The Computational Limits of Deep Learning
Thompson, N. C., Greenewald, K., Lee, K., & Manso, G. F. (2020) · arXiv · doi:10.48550/arxiv.2007.05558 · Cited by 538 · see citing works
The State of Sparsity in Deep Neural Networks
Gale, T., Elsen, E., & Hooker, S. (2019) · arXiv · doi:10.48550/arxiv.1902.09574 · Cited by 440 · see citing works
Self-citationTaylor expansion of the accumulated rounding error
Linnainmaa, S. (1976) · BIT · doi:10.1007/bf01931367 · Cited by 315 · see citing works
Graphics processing unit (GPU) programming strategies and trends in GPU computing
Brodtkorb, A. R., Hagen, T. R., & Sætra, M. L. (2013) · Journal of Parallel and Distributed Computing · doi:10.1016/j.jpdc.2012.04.003 · Cited by 283 · see citing works
Flagged first, then by citations
Research works printed but not linked: 31
- Moravec, H. When will computer hardware match the human brain. Journal of Transhumanism 1 (1998).
- Olukotun, K. Beyond parallel programming with domain specific languages. SIGPLAN Not. 49, 8 (Feb. 2014), 179-180. https://doi.org/10.1145/ 2692916.2557966 · DOI
- Prabhakar, R., et al. Plasticine: A reconfigurable architecture for parallel patterns. In 2017 ACM/ IEEE 44 th Annual Intern. Symposium on Computer Architecture (ISCA). 389-402.
- Reddi, V., et al. MLPerf inference benchmark. In 2020 ACM/IEEE 47 th Annual Intern. Symposium on Computer Architecture (2020), 446-459.
- Rumelhart, D., Hinton, G., and Williams, R. Learning representations by back-propagating errors. MIT Press (1988), 696-699.
- Sabour, S., Frost, N., and Hinton, G. Dynamic routing between capsules. (2017), 3856-3866. http://papers. nips.cc/paper/6975-dynamic-routing-between- capsules.pdf
- Shalf, J. The future of computing beyond Moore's law. Philosophical Transactions of the Royal Society A, 378 (2020).
- Singh, V., Perdigones, A., Garcia, J., Cañas, I., and Mazarrón, F. Analyzing worldwide research in hardware architecture, 1997-2011. Commun. ACM 58 (January 2015), 76-85. https://doi.org/10.1145/ 2688498.2688499. · DOI
- Steinbuch, K. and Piske, U.A.W. Learning matrices and their applications. IEEE Transactions on Electronic Computers EC-12, 6 (1963), 846-862.
- Stroop, J. Studies of interference in serial verbal reactions. J. of Experimental Psychology 18, 6 (1935), 643. https://doi.org/10.1037/h0054651 · DOI · Paper page
- Tolstoy, L. and Bartlett, R. Anna Karenina. Oxford University Press (2016), https://books.google.com/ books?id=1DooDwAAQBAJ
- Von Neumann, J., Churchland, P.M., and Churchland, P.S. The Computer and the Brain. Yale University Press (2000), https://books.google.com/ books?id=Q30MqJjRv1gC
- Warden, P. and Situnayake, D. TinyML: Machine Learning With TensorFlow Lite on Arduino and Ultra-Low-Power Microcontrollers. (2019), O'Reilly Media, Inc. https://books.google.com/ books?id=sB3mxQEACAAJ
- Barham, P. and Isard, M. Machine learning systems are stuck in a rut. In Proceedings of the Workshop on Hot Topics in Operating Systems (HotOS '19), (Bertinoro, Italy), ACM, New York, NY, USA, 177-183. https://doi.org/10.1145/3317550.3321441 · DOI
- Bi, G. and Poo, M. Synaptic modifications in cultured hippocampal neurons: Dependence on spike timing, synaptic strength, and postsynaptic cell type. Journal of Neuroscience 18, 24 (1998), 10464-10472. https://doi.org/10.1523/ JNEUROSCI.18-24-10464.1998 arXiv:https://www. jneurosci.org/content/18/24/10464.full.pdf · DOI
- Bubic, A., Cramon, D., and Schubotz, R. Prediction, cognition, and the brain. Frontiers in Human Neuroscience 4 (2010), 25. https://doi.org/10.3389/ fnhum.2010.00025 · DOI
- Chellapilla, K., Puri, S., and Simard, P. High performance convolutional neural networks for document processing. Tenth International Workshop on Frontiers in Handwriting Recognition (2006).
- Coates, A., Huval, B., Wang, T., Wu, D., Catanzaro, B., and Andrew, N. Deep learning with COTS HPC systems. In Proceedings of the 30 th Intern. Conf. on Machine Learning (2013), Sanjoy Dasgupta and David McAllester (Eds.). PMLR, Atlanta, GA, USA, 1337- 1345. http://proceedings.mlr.press/v28/coates13.html
- Dean, J. The deep learning revolution and its implications for computer architecture and chip design. IEEE International Solid-State Circuits Conference (2020), 8-14.
- Dennard, R., Gaensslen, F., Yu, H., Rideout, V., Bassous, E., and LeBlanc, A. Design of ion implanted MOSFET's with very small physical dimensions. IEEE Journal of Solid-State Circuits 9, 5 (1974), 256-268.
- Dongarra, J., Gates, M., Kurzak, J., Luszczek, P., and Tsai, Y. Autotuning numerical dense linear algebra for batched computation with GPU hardware accelerators. In Proceedings of the IEEE 106, 11 (2018), 2040-2055.
- Fukushima, K. and Miyake, S. Neocognitron: A new algorithm for pattern recognition tolerant of deformations and shifts in position. Pattern Recognition 15, 6 (1982), 455-469. http:// www.sciencedirect.com/science/article/ pii/0031320382900243
- Hauck, S. and DeHon, A. Reconfigurable Computing: The Theory and Practice of FPGA-Based Computation. (2017), Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.
- Horowitz, M. Computing's energy problem (and what we can do about it). In 2014 IEEE International Solid- State Circuits Conference Digest of Technical Papers (ISSCC). 10-14.
- Jouppi, N., et al. In-datacenter performance analysis of a tensor processing unit. SIGARCH Comput. Archit. News 45, 2 (June 2017), 1-12. https://doi. org/10.1145/3140659.3080246 · DOI
- Kuhn, T. The Structure of Scientific Revolutions. (1962), University of Chicago Press, Chicago.
- Kurzweil, R. The Age of Intelligent Machines. (1990), MIT Press, Cambridge, MA, USA.
- Larus, J. Spending Moore's dividend. Commun. ACM 52, 5 (May 2009), 62-69. https://doi. org/10.1145/1506409.1506425 · DOI
- Lee, H., Brown, K., Sujeeth, A., Chafi, H., Rompf, T., Odersky, M., and Olukotun, K. Implementing domain-specific languages for heterogeneous parallel computing. IEEE Micro 31, 5 (2011), 42-53.
- Mirhoseini, A., et al. A graph placement methodology for fast chip design. Nature 594 (June 9, 2021), 207-212, https://www.nature.com/articles/s41586- 021-03544-w.
- Moore, D. The Anna Karenina Principle applied to ecological risk assessments of multiple stressors. Human and Ecological Risk Assessment: An International Journal 7, 2 (2001), 231-237. https://doi. org/10.1080/20018091094349 · DOI
Cited by
Sorted by citations
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676 · see citing works
Recent Advances in Natural Language Processing via Large Pre-trained Language Models: A Survey
Min, B., Ross, H., Sulem, E., Veyseh, A. P. B., Nguyen, T. H., Sainz, O., Agirre, E., Heintz, I., & Roth, D. (2024) · ACM Computing Surveys · doi:10.1145/3605943 · Cited by 1,387 · see citing works
FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
Dao, T., Fu, D. Y., Ermon, S., Rudra, A., & Ré, C. (2022) · arXiv · doi:10.48550/arxiv.2205.14135 · Cited by 1,214 · see citing works
“However, even though structured matrices are efficient in theory, they have not seen wide adoption since it is hard to translate their efficiency to wall-clock speedup since dense unconstrained matrix multiply has very optimize implementation, a phenomenon known as the hardware lottery [39].”
BackgroundTowards Continual Reinforcement Learning: A Review and Perspectives
Khetarpal, K., Riemer, M., Rish, I., & Precup, D. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13673 · Cited by 224 · see citing works
A Practical Survey on Faster and Lighter Transformers
Fournier, Q., Caron, G. M., & Aloise, D. (2023) · ACM Computing Surveys · doi:10.1145/3586074 · Cited by 145 · see citing works
The Platonic Representation Hypothesis
Huh, M., Cheung, B., Wang, T., & Isola, P. (2024) · arXiv · doi:10.48550/arxiv.2405.07987 · Cited by 35 · see citing works
“Additionally, the “hardware lottery” (Hooker, 2021) suggests that the success of AI models can also depend on the compatibility of their design with available computational architectures, further contributing to convergent trends.”
BackgroundAI’s Epistemic Harm: Reinforcement Learning, Collective Bias, and the New AI Culture War
Humphreys, D. (2025) · Philosophy & Technology · doi:10.1007/s13347-025-00928-y · Cited by 26 · see citing works
“Hooker ( 2021 ) introduced the con - cept of a hardware lottery where a design solution is chosen not because it is intrinsically better, but because it is favoured by existing hardware.”
Implicit Language Models are RNNs: Balancing Parallelization and Expressivity
Schöne, M., Rahmani, B., Kremer, H., Falck, F., Ballani, H., & Gladrow, J. (2025) · arXiv · doi:10.48550/arxiv.2502.07827
“Emerging hardware promising rapid computation of fixed-points of neural networks (Brunner et al., 2025) may tilt the hardware lottery (Hooker, 2020) in favor of such implicit models, making this an opportune moment to explore their potential.”
- Authors
- Hooker, S.
- Affiliations
- Google (United States) (US)
- Corresponding author
- Hooker, Sara
- ORCID
- Hooker, Sara
- Venue
- Communications of the ACM
- Published
- 2021-11-19
- Volume
- 64
- Issue
- 12
- Pages
- 58-65
- ISSN
- 0001-0782 (ISSN-L)
- 1557-7317
- OpenAlex
- W3087547017
- Cited by
- 211
- References
- 13
- Citations by year
- 2026: 40 · 2025: 41 · 2024: 26 · 2023: 42 · 2022: 32 · 2021: 20 · 2020: 10
- FWCI
- 19.11
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Communications of the ACM (publishedVersion, cc-by-nc)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Hardware and Architecture › Embedded Systems Design Techniques
- Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
- Social Sciences › Business, Management and Accounting › Strategy and Management › Digital Platforms and Economics
- Keywords
- hardware-software co-design, research incentives, interdisciplinary collaboration, hardware development
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
