PreprintCited by 72Parallel Computing and Optimization TechniquesComputer Graphics and Visualization TechniquesNumerical Methods and Algorithms

The DOI matches its DataCite record.

Dissecting the NVidia Turing T4 GPU via Microbenchmarking

Open access
Jia, Zhe, Maggioni, Marco, Smith, Jeffrey, Scarpazza, Daniele Paolo·2019·arXiv

Abstract

In 2019, the rapid rate at which GPU manufacturers refresh their designs, coupled with their reluctance to disclose microarchitectural details, is still a hurdle for those software designers who want to extract the highest possible performance. Last year, these very reasons motivated us to dissect the Volta GPU architecture using microbenchmarks. The introduction in August 2018 of Turing, NVidia's latest architecture, pressed us to update our study. In this report, we examine Turing and compare it quantitatively against previous NVidia GPU generations. Specifically, we study the T4 GPU: a low-power board aiming at inference applications. We describe its improvements against its inference-oriented predecessor: the P4 GPU based on the Pascal architecture. Both T4 and P4 GPUs achieve significantly higher frequency-per-Watt figures than their full-size counterparts. We study the performance of the T4's TensorCores, finding a much higher throughput on low-precision operands than on the P4 GPU. We reveal that Turing introduces new instructions that express matrix math more succinctly. We map Turing's instruction space, finding the same encoding as Volta, and additional instructions. We reveal that the Turing TU104 chip has the same memory hierarchy depth as the Volta GV100; cache levels sizes on the TU104 are frequently twice as large as those found on the Pascal GP104. We benchmark each constituent of the T4 memory hierarchy and find substantial overall performance improvements over its P4 predecessor. We studied how clock throttling affects compute-intensive workloads that hit power or thermal limits. Many of our findings are novel, published here for the first time. All of them can guide high-performance software developers get closer to the GPU's peak performance.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1903.07486

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Still being checked.

Research works printed but not linked: 6

  1. [1][1] Z. Jia, M. Maggioni, B. Staiger, and D. P. Scarpazza, 'Dissecting the NVidia Volta GPU architecture via microbenchmarking,' in 2018 GPU Technology Conference , 2018. [Online]. Available: http://on-demand.gputechconf.com/gtc/2018/presentation/ s8122-dissecting-the-volta-gpu-architecture-through-microbenchmarking. pdf -
  2. [2][2] --, 'Dissecting the NVidia Volta GPU architecture via microbenchmarking,' 2018. [Online]. Available: https://arxiv.org/abs/1804.06826 - · arXiv
  3. [5][5] C. Wang, Z. Jia, and K. Chen, 'Tuning performance on Kepler GPUs: An introduction to Kepler assembler and its usage in CNN optimization,' in GPU Technology Conference Presentation , vol. 6173, 2015. -
  4. [7][7] H. Wong, M. M. Papadopoulou, M. Sadooghi-Alvandi, and A. Moshovos, 'Demystifying GPU microarchitecture through microbenchmarking,' in 2010 IEEE International Symposium on Performance Analysis of Systems Software (ISPASS) , March 2010, pp. 235-246. -
  5. [8][8] X. Zhang, G. Tan, S. Xue, J. Li, K. Zhou, and M. Chen, 'Understanding the GPU microarchitecture to achieve bare-metal performance tuning,' in Proceedings of the 22nd ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming , ser. PPoPP '17. New York, NY, USA: ACM, 2017, pp. 31-43. -
  6. [9][9] X. Mei and X. Chu, 'Dissecting GPU memory hierarchy through microbenchmarking,' IEEE Transactions on Parallel and Distributed Systems , vol. 28, no. 1, pp. 72-86, Jan 2017. -

Cited by

Sorted by citations

Metadata

Authors
Jia, Z., Maggioni, M., Smith, J., & Scarpazza, D. P.
Venue
arXiv
Publisher
arXiv
Published
2019
ISSN
2331-8422 (ISSN-L)
OpenAlex
W2920798074
Cited by
69
References
0
Citations by year
2025: 3 · 2024: 9 · 2023: 8 · 2022: 8 · 2021: 24 · 2020: 14 · 2019: 3
Type
posted content
Language
en
Open access
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
  • Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Computer Graphics and Visualization Techniques
  • Physical Sciences › Computer Science › Computational Theory and Mathematics › Numerical Methods and Algorithms
Keywords
microarchitecture, low-precision arithmetic, memory hierarchy, cache size, power efficiency, GPU inference

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review