The DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddings
Open accessAbstract
In response to innovations in machine learning (ML) models, production workloads changed radically and rapidly. TPU v4 is the fifth Google domain specific architecture (DSA) and its third supercomputer for such ML models. Optical circuit switches (OCSes) dynamically reconfigure its interconnect topology to improve scale, availability, utilization, modularity, deployment, security, power, and performance; users can pick a twisted 3D torus topology if desired. Much cheaper, lower power, and faster than Infiniband, OCSes and underlying optical components are <5% of system cost and <3% of system power. Each TPU v4 includes SparseCores, dataflow processors that accelerate models that rely on embeddings by 5x--7x yet use only 5% of die area and power. Deployed since 2020, TPU v4 outperforms TPU v3 by 2.1x and improves performance/Watt by 2.7x. The TPU v4 supercomputer is 4x larger at 4096 chips and thus nearly 10x faster overall, which along with OCS flexibility and availability allows a large language model to train at an average of ~60% of peak FLOPS/second. For similar sized systems, it is ~4.3x--4.5x faster than the Graphcore IPU Bow and is 1.2x--1.7x faster and uses 1.3x--1.9x less power than the Nvidia A100. TPU v4s inside the energy-optimized warehouse scale computers of Google Cloud use ~2--6x less energy and produce ~20x less CO2e than contemporary DSAs in typical on-premise data centers.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Principles and Practices of Interconnection Networks
Dally, W. J., & Towles, B. (2004) · Cited by 3,260
Self-citationWilliams, S., Waterman, A., & Patterson, D. A. (2009) · Communications of the ACM · doi:10.1145/1498765.1498785 · Cited by 2,798
Self-citation1.1 Computing's energy problem (and what we can do about it)
Horowitz, M. (2014) · 2014 IEEE International Solid-State Circuits Conference Digest of Technical Papers (ISSCC) · doi:10.1109/isscc.2014.6757323 · Cited by 2,069
The Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale Machines
Barroso, L. A., & Hölzle, U. (2009) · Synthesis lectures on computer architecture · doi:10.2200/s00193ed1v01y200905cac006 · Cited by 1,714
Schwartz, R., Dodge, J., Smith, N. A., & Etzioni, O. (2020) · Communications of the ACM · doi:10.1145/3381831 · Cited by 1,647
Recalibrating global data center energy-use estimates
Masanet, E., Shehabi, A., Lei, N., Smith, S., & Koomey, J. (2020) · Science · doi:10.1126/science.aba3758 · Cited by 1,545
Anton, a special-purpose machine for molecular dynamics simulation
Shaw, D. E., Deneroff, M. M., Dror, R. O., Kuskin, J. S., Larson, R. H., Salmon, J. K., Young, C., Batson, B., Bowers, K. J., Chao, J. C., et al. (2008) · Communications of the ACM · doi:10.1145/1364782.1364802 · Cited by 817
Self-citationLaMDA: Language Models for Dialog Applications
Thoppilan, R., De Freitas, D., Hall, J., Shazeer, N., Kulshreshtha, A., Cheng, H.-T., Jin, A., Bos, T., Baker, L., Du, Y., et al. (2022) · arXiv · doi:10.48550/arxiv.2201.08239 · Cited by 778
Cited by
The Computational Limits of Deep Learning
Thompson, N. C., Greenewald, K., Lee, K., & Manso, G. F. (2020) · arXiv · doi:10.48550/arxiv.2007.05558 · Cited by 538
A review of green artificial intelligence: Towards a more sustainable future
Bolón-Canedo, V., Morán-Fernández, L., Cancela, B., & Alonso-Betanzos, A. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128096 · Cited by 486
Neuromorphic computing at scale
Kudithipudi, D., Schuman, C., Vineyard, C. M., Pandit, T., Merkel, C., Kubendran, R., Aimone, J. B., Orchard, G., Mayr, C., Benosman, R., et al. (2025) · Nature · doi:10.1038/s41586-024-08253-8 · Cited by 357
Self-citationDong, P., Zhuang, J., Yang, Z., Ji, S., Li, Y., Xu, D., Huang, H., Hu, J., Jones, A. K., Shi, Y., et al. (2024) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2024.3443692 · Cited by 35
“In the cloud , to guarantee the quality of service, deep learning-based inference for the cloud services in Microsoft Bing Search [21], Microsoft Azure Cloud [22], [23], and Google Cloud [24], [25], [26], all have a single-digit millisecond latency budget to process.”
A Survey: Collaborative Hardware and Software Design in the Era of Large Language Models
Guo, C., Cheng, F., Du, Z., Kiessling, J., Ku, J., Li, S., Li, Z., Ma, M., Molom-Ochir, T., Morris, B., et al. (2025) · IEEE Circuits and Systems Magazine · doi:10.1109/mcas.2024.3476008 · Cited by 32
Self-citation
Metadata
- Authors
- Jouppi, N., Kurian, G., Li, S., Ma, P., Nagarajan, R., Nai, L., Patil, N., Subramanian, S., Swing, A., Towles, B., Young, C., Zhou, X., Zhou, Z., & Patterson, D. A.
- Venue
- Proceedings of the 50th Annual International Symposium on Computer Architecture
- Publisher
- ACM
- Published
- 2023-06-17
- Pages
- 1-14
- ISSN
- 1063-6897 (ISSN-L)
- OpenAlex
- W4380874786
- Cited by
- 508
- References
- 17
- FWCI
- 94.19
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddings. https://app.cochat.ai/papers/10.1145/3579371.3589350
- The paper
- Jouppi, N., Kurian, G., Li, S., Ma, P., Nagarajan, R., Nai, L., Patil, N., Subramanian, S., Swing, A., Towles, B., Young, C., Zhou, X., Zhou, Z., & Patterson, D. A. (2023). TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddings. Proceedings of the 50th Annual International Symposium on Computer Architecture. https://doi.org/10.1145/3579371.3589350
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
