The DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
Ten Lessons From Three Generations Shaped Google’s TPUv4i : Industrial Product
Abstract
Google deployed several TPU generations since 2015, teaching us lessons that changed our views: semi-conductor technology advances unequally; compiler compatibility trumps binary compatibility, especially for VLIW domain-specific architectures (DSA); target total cost of ownership vs initial cost; support multi-tenancy; deep neural networks (DNN) grow 1.5X annually; DNN advances evolve workloads; some inference tasks require floating point; inference DSAs need air-cooling; apps limit latency, not batch size; and backwards ML compatibility helps deploy DNNs quickly. These lessons molded TPUv4i, an inference DSA deployed since 2020.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Mastering the game of Go without human knowledge
Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al. (2017) · Nature · doi:10.1038/nature24270 · Cited by 9,084
In-Datacenter Performance Analysis of a Tensor Processing Unit
Jouppi, N. P., Young, C., Patil, N., Patterson, D., Agrawal, G., Bajwa, R., Bates, S., Bhatia, S., Boden, N., Borchers, A., et al. (2017) · Proceedings of the 44th Annual International Symposium on Computer Architecture · doi:10.1145/3079856.3080246 · Cited by 4,537
Self-citationWilliams, S., Waterman, A., & Patterson, D. A. (2009) · Communications of the ACM · doi:10.1145/1498765.1498785 · Cited by 2,798
Self-citation1.1 Computing's energy problem (and what we can do about it)
Horowitz, M. (2014) · 2014 IEEE International Solid-State Circuits Conference Digest of Technical Papers (ISSCC) · doi:10.1109/isscc.2014.6757323 · Cited by 2,069
Dean, J., & Barroso, L. A. (2013) · Communications of the ACM · doi:10.1145/2408776.2408794 · Cited by 1,852 · cited 1×
“Trying to maximize average perf/TDP, such as temporarily running faster, can lead to worse tail latencies [6].”
Ho, R., Mai, K. W., & Horowitz, M. A. (2001) · Proceedings of the IEEE · doi:10.1109/5.920580 · Cited by 1,438
In-Datacenter Performance Analysis of a Tensor Processing Unit
Jouppi, N. P., Young, C., Patil, N., Patterson, D., Agrawal, G., Bajwa, R., Bates, S., Bhatia, S., Boden, N., Borchers, A., et al. (2017) · ACM SIGARCH Computer Architecture News · doi:10.1145/3140659.3080246 · Cited by 1,240
Self-citationA new golden age for computer architecture
Hennessy, J. L., & Patterson, D. A. (2019) · Communications of the ACM · doi:10.1145/3282307 · Cited by 690
Self-citationEmbedded Computing: A VLIW Approach to Architecture, Compilers and Tools
Fisher, J. A., Faraboschi, P., & Young, C. (2004) · Cited by 327
Self-citationA domain-specific supercomputer for training deep neural networks
Jouppi, N. P., Yoon, D. H., Kurian, G., Li, S., Patil, N., Laudon, J., Young, C., & Patterson, D. (2020) · Communications of the ACM · doi:10.1145/3360307 · Cited by 258
Self-citationDomain-specific hardware accelerators
Dally, W. J., Turakhia, Y., & Han, S. (2020) · Communications of the ACM · doi:10.1145/3361682 · Cited by 246
Park, J., Naumov, M., Basu, P., Deng, S., Kalaiah, A., Khudia, D. S., Law, J., Malani, P., Malevich, A., Nadathur, S., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.09886 · Cited by 213
Fine-grained DRAM: energy-efficient DRAM for extreme bandwidth systems
O'Connor, M., Chatterjee, N., Lee, D., Wilson, J., Agrawal, A., Keckler, S. W., & Dally, W. J. (2017) · Proceedings of the 50th Annual IEEE/ACM International Symposium on Microarchitecture · doi:10.1145/3123939.3124545 · Cited by 199
DeepRank: A New Deep Architecture for Relevance Ranking in Information Retrieval
Pang, L., Lan, Y., Guo, J., Xu, J., Xu, J., & Cheng, X. (2017) · Proceedings of the 2017 ACM on Conference on Information and Knowledge Management · doi:10.1145/3132847.3132914 · Cited by 197
Value-Aware Quantization for Training and Inference of Neural Networks
Park, E., Yoo, S., & Vajda, P. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01225-0_36 · Cited by 181
A Study of BFLOAT16 for Deep Learning Training
Kalamkar, D., Mudigere, D., Mellempudi, N., Das, D., Banerjee, K., Avancha, S., Vooturi, D. T., Jammalamadaka, N., Huang, J., Yuen, H., et al. (2019) · arXiv · doi:10.48550/arxiv.1905.12322 · Cited by 169 · cited 1×
“Bfloat16 is a better match to DNNs than IEEE 754 fp16 [24], so the MXU was changed to become the first hardware to support it, with 16K MAC units (½ the side of the systolic array, so ¼ of the size).”
Aspelund, K. (2006) · Cited by 166
A domain-specific architecture for deep neural networks
Jouppi, N. P., Young, C., Patil, N., & Patterson, D. (2018) · Communications of the ACM · doi:10.1145/3154484 · Cited by 166
Self-citationThe Design Process for Google's Training Chips: TPUv2 and TPUv3
Norrie, T., Patil, N. P., Yoon, D. H., Kurian, G. T., Li, S., Laudon, J. P., Young, C., Jouppi, N. P., & Patterson, D. A. (2021) · IEEE Micro · doi:10.1109/mm.2021.3058217 · Cited by 164
Self-citationThe Datacenter As a Computer: Designing Warehouse-scale Machines
Barroso, L. A., Hölzle, U., Ranganathan, P., & Martonosi, M. (2018) · Cited by 99
The Datacenter as a Computer: Designing Warehouse-Scale Machines, Third Edition
Barroso, L. A., Hölzle, U., & Ranganathan, P. (2018) · Synthesis lectures on computer architecture · doi:10.2200/s00874ed3v01y201809cac046 · Cited by 93
The Accelerator Wall: Limits of Chip Specialization
Fuchs, A., & Wentzlaff, D. (2019) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca.2019.00023 · Cited by 74
Dissecting the NVidia Turing T4 GPU via Microbenchmarking
Jia, Z., Maggioni, M., Smith, J., & Scarpazza, D. P. (2019) · arXiv · doi:10.48550/arxiv.1903.07486 · Cited by 69
Optimizing Data-Center TCO with Scale-Out Processors
Grot, B., Hardy, D., Lotfi-Kamran, P., Falsafi, B., Nicopoulos, C. A., & Sazeides, Y. (2012) · IEEE Micro · doi:10.1109/mm.2012.71 · Cited by 42
Embedded Deep Inference in Practice: Case for Model Partitioning
Dey, S., Mukherjee, A., Pal, A., & Balamuralidhar, P. (2019) · doi:10.1145/3362743.3362964 · Cited by 24
INFaaS: A Model-less Inference Serving SystemPreprint
Romero, F., Li, Q., Yadwadkar, N. J., & Kozyrakis, C. (2019) · arXiv (Cornell University) · Cited by 7
Cited by
Competition-level code generation with AlphaCode
Li, Y., Choi, D., Chung, J., Kushman, N., Schrittwieser, J., Leblond, R., Eccles, T., Keeling, J., Gimeno, F., Dal Lago, A., et al. (2022) · Science · doi:10.1126/science.abq1158 · Cited by 1,043
Competition-Level Code Generation with AlphaCode
Li, Y., Choi, D., Chung, J., Kushman, N., Schrittwieser, J., Leblond, R., Eccles, T., Keeling, J., Gimeno, F., Lago, A. D., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.07814 · Cited by 1,043
Neural inference at the frontier of energy, space, and time
Modha, D. S., Akopyan, F., Andreopoulos, A., Appuswamy, R., Arthur, J. V., Cassidy, A. S., Datta, P., DeBole, M. V., Esser, S. K., Otero, C. O., et al. (2023) · Science · doi:10.1126/science.adh1174 · Cited by 115
DFX: A Low-latency Multi-FPGA Appliance for Accelerating Transformer-based Text Generation
Hong, S., Moon, S., Kim, J., Lee, S., Kim, M., Lee, D., & Kim, J.-Y. (2022) · 2022 55th IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1109/micro56248.2022.00051 · Cited by 107
The rising costs of training frontier AI models
Cottier, B., Rahman, R., Fattorini, L., Maslej, N., Besiroglu, T., & Owen, D. (2024) · arXiv · doi:10.48550/arxiv.2405.21015 · Cited by 27
“For TPUs, the server scale-up was based on data from Table 1 of [26].”
Method
Metadata
- Authors
- Jouppi, N. P., Hyun Yoon, D., Ashcraft, M., Gottscho, M., Jablin, T. B., Kurian, G., Laudon, J., Li, S., Ma, P., Ma, X., Norrie, T., Patil, N., Prasad, S., Young, C., Zhou, Z., & Patterson, D.
- Venue
- 2021 ACM/IEEE 48th Annual International Symposium on Computer Architecture (ISCA)
- Publisher
- IEEE
- Published
- 2021-06
- Pages
- 1-14
- OpenAlex
- W3190062760
- Cited by
- 360
- References
- 26
- FWCI
- 39.56
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.