CoChat

AI and ML Accelerator Survey and Trends

Open access
Reuther, Albert, Michaleas, Peter, Jones, Michael, Gadepally, Vijay, Samsi, Siddharth, Kepner, Jeremy · 2022 · 2022 IEEE High Performance Extreme Computing Conference (HPEC)

The DOI matches its Crossref record; 61 of the 61 references it cites were found in a registry or an index.

Abstract

This paper updates the survey of AI accelerators and processors from past three years. This paper collects and summarizes the current commercial accelerators that have been publicly announced with peak performance and power consumption numbers. The performance and power values are plotted on a scatter graph, and a number of dimensions and observations from the trends on this plot are again discussed and analyzed. Two new trends plots based on accelerator release dates are included in this year's paper, along with the additional trends of some neuromorphic, photonic, and memristor-based inference accelerators.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/hpec55821.2022.9926331

Cites

61 of 61 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 116

  1. Gadepally, 2019, AI Enabling Technologies
  2. Liao, 2001, Neural Networks in Hardware: A Survey
  3. Rogers, 2021, An Academic’s Attempt to Clear the Fog of the Machine Learning Accelerator War - SIGARCH
  4. Roos, 2019, FPGA Acceleration Card Delivers on Bandwidth, Speed, and Flexibility
  5. 2018, aiWare3 Hardware IP Helps Drive Autonomous Vehicles To Production
  6. Merritt, 2019, Startup Accelerates AI at the Sensor
  7. Peng, 2019, Alibaba’s New AI Chip Can Process Nearly 80K Images Per Second
  8. Clarke, 2018, Indo-US Startup Preps Agent-based AI Processor
  9. Hamilton, 2018, AWS Inferentia Machine Learning Processor
  10. Evangelist, 2020, Deep dive into Amazon Inferentia: A Custom-Built Chip to Enhance ML and AI
  11. 2017, Taking a Deeper Look at AMD Radeon Instinct GPUs for Deep Learning
  12. Smith, 2018, AMD Announces Radeon Instinct MI60 & MI50 Accelerators Powered By 7nm Vega
  13. Schor, 2020, Arm Ethos is for Ubiquitous AI At the Edge
  14. Ward-Foxton, 2022, Axelera Demos AI Test Chip After Taping Out in Four Months
  15. Merritt, 2018, Baidu Accelerator Rises in AI
  16. Duckett, 2018, Baidu Creates Kunlun Silicon for AI
  17. Wheeler, 2019, Bitmain SoC Brings AI to the Edge
  18. Demler, 2020, Blaize Ignites Edge-AI Performance
  19. Wu, 2018, Chinese AI Chip Maker Cambricon Unveils New Cloud-Based Smart Chip
  20. Cutress, 2018, Cambricon, Maker of Hauwei’s Kirin NPU IP, Build a Big AI Chip and PCIe Card
  21. Gwennap, 2019, Kendryte Embeds AI for Surveillance
  22. Hock, 2019, Introducing the Cerebras CS-1, the Industry’s Fastest Artificial Intelligence Computer
  23. Trader, 2021, Cerebras Doubles AI Performance with Second-Gen 7nm Wafer Scale Engine
  24. 2019, Cornami Achieves Unprecedented Performance at Lowest Power Dissipation for Deep Neural Networks
  25. Clarke, 2019, GlobalFoundries Aids Launch of Chinese AI Startup
  26. Mehta, Performance Estimation and Benchmarks for Real-World Edge Inference Applications
  27. 2019, Edge TPU
  28. Teich, 2018, Tearing Apart Google’s TPU 3.0 AI Coprocessor
  29. Jouppi, Ten Lessons From Three Generations Shaped Google’s TPUv4i, 1
  30. Peckham, 2022, Google Cloud’s New TPU v4 ML Hub Packs 9 Exaflops of AI
  31. Gwennap, 2020, Groq Rocks Neural Networks
  32. Lacey, 2017, Preliminary IPU Benchmarks
  33. 2020, Dell DSS8440 Graphcore IPU Server
  34. Ward-Foxton, 2020, Graphcore Takes on Nvidia with Second-Gen AI Accelerator
  35. Tyson, 2022, Graphcore Bow IPU Introduces TSMC 3D Wafer-on-Wafer Processor
  36. 2020, GAP Application Processors
  37. Turley, 2020, GAP9 for ML at the Edge EEJournal
  38. Hemsoth, 2020, Groq Shares Recipe for TSP Nodes, Systems
  39. Ward-Foxton, 2019, Gyrfalcon Unveils Fourth AI Accelerator Chip - EE Times
  40. 2020, SolidRun, Gyrfalcon Develop Arm-based Edge Op-timized AI Inference Server
  41. Gwennap, 2019, Habana Offers Gaudi for AI Training
  42. Gwennap, 2019, Habana Wins Cigar for AI Inference
  43. Ward-Foxton, 2019, Details of Hailo AI Edge Accelerator Emerge
  44. 2020, Horizon Robotics Journey2 Automotive AI Processor Series
  45. 2020, Ascend 310 AI Processor
  46. 2020, Ascend 910 AI Processor
  47. Hemsoth, 2018, Intel FPGA Architecture Focuses on Deep Learning Inference
  48. Hruska, 2017, New Movidius Myriad X VPU Packs a Custom Neural Compute Engine
  49. De Gelas, 2019, Intel’s Xeon Cascade Lake vs. NVIDIA Turing: An Analysis in AI
  50. 2020, Intel Xeon Platinum 8180
  51. 2020, Intel Xeon Platinum 8280
  52. Dupont de Dinechin, Kalray’s MPPA® Manycore Processor: At the Heart of Intelligent Systems
  53. Clarke, 2020, NXP, Kalray Demo Coolidge Parallel Processor in ’BlueBox
  54. Ward-Foxton, 2020, Kneron’s Next-Gen Edge AI Chip Gets $40m Boost
  55. 2021, Kneron Attracts Strategic Investors
  56. 2020, Maxim Debuts Homegrown AI Accelerator in Latest ULP SoC
  57. Jani, 2021, Maxim Showcases Efficient Custom AI
  58. Morgan, 2017, Drilling Into Microsoft’s BrainWave Soft Deep Learning Chip
  59. Ward-Foxton, 2021, Mythic Resizes its AI Chip
  60. Hemsoth, 2018, A Mythic Approach to Deep Learning Inference
  61. Fick, 2018, Mythic @ Hot Chips 2018
  62. Freund, 2019, NovuMind: An Early Entrant in AI Silicon
  63. Yoshida, 2018, NovuMind’s AI Chip Sparks Controversy
  64. Morgan, 2021, Nvidia Rounds Out “Ampere” Lineup With Two New Accelerators
  65. Krashinsky, 2020, NVIDIA Ampere Architecture In-Depth
  66. Need, 2017, CoRR, abs/1706.0
  67. Cutress, 2018, NVIDIA’s DGX-2: Sixteen Tesla V100s, 30TB of NVMe, Only $400K
  68. Campa, 2020, Defining AI Innovation with NVIDIA DGX A100
  69. Smith, 2022, NVIDIA Hopper GPU Architecture and H100 Accelerator Announced: Working Smarter and Harder
  70. 2019, NVIDIA Gives Jetson AGX Xavier a Trim, Announces Nano-Sized Jetson Xavier NX
  71. Funk, 2022, NVIDIA Jetson AGX Orin: The Next-Gen Platform That Will Power Our AI Robot Overlords Unveiled
  72. 2022, Jetson AGX Orin for Next-Gen Robotics
  73. Franklin, 2017, NVIDIA Jetson TX2 Delivers Twice the Intelligence to the Edge
  74. Hill, 2022, NVIDIA Unveils Ampere-Infused DRIVE AGX For Autonomous Cars, Isaac Robotics Platform With BMW Partnership
  75. NVIDIA Tesla P100
  76. Smith, 2018, 16GB NVIDIA Tesla V100 Gets Reprieve; Remains in Production
  77. Kilgariff, 2018, NVIDIA Turing Architecture In-Depth
  78. 2019, NVIDIA Tesla V100 Tensor Core GPU
  79. McGregor, 2020, Perceive Exits Stealth With Super Efficient Machine Learning Chip For Smarter Devices
  80. Schor, 2017, The 2,048-core PEZY-SC2 Sets a Green500 Record
  81. 2020, MN-Core
  82. Cutress, 2019, Preferred Networks: A 500 W Custom PCIe Card using 3000 mm2 Silicon
  83. Firu, 2019, Quadric Edge Supercomputer
  84. Ward-Foxton, 2020, Qualcomm Cloud AI 100 Promises Impressive Performance per Watt for Near-Edge AI
  85. McGrath, 2019, Qualcomm Targets AI Inferencing in the Cloud
  86. 2018, Rockchip Released Its First AI Processor RK3399Pro NPU Performance Up to 2.4TOPs
  87. Gwennap, 2020, Machine Learning Moves to the Edge
  88. McGrath, 2018, Tech Heavyweights Back AI Chip Startup
  89. Merritt, 2018, Startup Rolls AI Chips for Audio
  90. Shilov, 2022, Tachyum Teases 128-Core CPU: 5.7 GHz, 950W, 16 DDR5 Channels
  91. Gwennap, 2020, Tenstorrent Scales AI Performance: Architecture Leads in Data-Center Power Efficiency
  92. 2020, FSD Chip - Tesla
  93. Ward-Foxton, 2021, TI’s First Automotive SoC with an AI Accelerator Launches
  94. 2021, TDA4VM Jacinto Processors for ADAS and Autonomous Vehicles
  95. Demler, 2020, TI Jacinto Accelerates Level 3 ADAS
  96. Merritt, 2019, Samsung, Toshiba Detail AI Chips
  97. Gwennap, 2020, Untether Delivers At-Memory AI
  98. Hilson, 2022, Startup Tachyum Offers Universal Processor for Evaluation
  99. Toon, 2020, Introducing 2nd Generation IPU Systems for AI at Scale
  100. Lunden, 2020, Graphcore Unveils New GC200 Chip and the Expandable M2000 IPU Machine That Runs on Them
  101. Ward-Foxton, 2020, SambaNova Emerges From Stealth With Record-Breaking AI System
  102. Prabhakar, SambaNova SN10 RDU: Accelerating Software 2.0 with Dataflow, 1
  103. Peckham, 2022, Intel’s Habana Labs Unveils Gaudi2, Greco AI Processors
  104. Morgan, 2022, Intel Pits New Gaudi2 AI Training Engine Against Nvidia GPUs
  105. Martin, 2022, Samsung, Others Test Esperanto’s 1,000-Core RISC-V AI Chip
  106. Freund, 2021, Esperanto Launches AI Accelerator with over 1000 RISCV Cores
  107. Peckham, 2021, Enter Dojo: Tesla Reveals Design for Modular Supercomputer & D1 Chip
  108. Shilov, 2021, Via Shutters Centaur Technology Site, Sells Off Equipment
  109. Burt, 2022, Chip Makers Press For Standardized FP8 Format For AI
  110. Ward-Foxton, 2021, Innatera Unveils Neuromorphic AI Chip to Accelerate Spiking Networks
  111. Levy, 2021, Innatera’s Spiking Neural Processor
  112. Ward-Foxton, 2020, Optical Compute Promises Game-Changing AI Performance
  113. 2021, Optical Chip Solves Hardest Math Problems Faster than GPUs
  114. Wilson, 2020, The Multiply and Fourier Transform Unit: A Micro-Scale Optical Processor
  115. Schneider, 2019, A Neural-Net Based on Light Could Best Digital Computers
  116. Choi, 2022, Photonic Chip Performs Image Recognition at the Speed of Light

Cited by

See all 162

Metadata

Authors
Reuther, A., Michaleas, P., Jones, M., Gadepally, V., Samsi, S., & Kepner, J.
Affiliations
MIT Lincoln Laboratory (US)
Venue
2022 IEEE High Performance Extreme Computing Conference (HPEC)
Publisher
IEEE
Published
2022-09-19
Pages
1-10
OpenAlex
W4308090791
Cited by
109
References
58
Citations by year
2026: 25 · 2025: 29 · 2024: 30 · 2023: 23 · 2022: 2
FWCI
16.89
Type
conference paper
Language
en
Open access
Locations
  • IEEE High Performance Extreme Computing Conference (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Engineering › Electrical and Electronic Engineering › Advanced Memory and Neural Computing
Keywords
AI accelerators, machine learning accelerators, peak performance, power consumption, power-performance tradeoff, neuromorphic processors

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.