The DOI matches its Crossref record; 79 of the 86 references it cites were found in a registry or an index; 2 of the 15 sources its claims cite were checked for support.
EQ-ViT: Algorithm-Hardware Co-Design for End-to-End Acceleration of Real-Time Vision Transformer Inference on Versal ACAP Architecture
Open accessAbstract
While Vision Transformers (ViTs) have shown consistent progress in computer vision, deploying them for real-time decision-making scenarios (< 1 ms) is challenging. Current computing platforms like CPUs, GPUs, or FPGA-based solutions struggle to meet this deterministic low-latency real-time requirement, even with quantized ViT models. Some approaches use pruning or sparsity to reduce model size and latency, but this often results in accuracy loss. To address the aforementioned constraints, in this work, we propose EQ-ViT, an end-to-end acceleration framework with novel algorithm and architecture co-design features to enable real-time ViT acceleration on AMD Versal Adaptive Compute Acceleration Platform (ACAP). The contributions are four-fold. First, we perform in-depth kernel-level performance profiling & analysis and explain the bottlenecks for existing acceleration solutions on GPU, FPGA, and ACAP. Second, on the hardware level, we introduce a new spatial and heterogeneous accelerator architecture, EQ-ViT architecture. This architecture leverages the heterogeneous features of ACAP, where both FPGA and artificial intelligence engines (AIEs) coexist on the same system-on-chip (SoC). Third, On the algorithm level, we create a comprehensive quantization-aware training strategy, EQ-ViT algorithm. This strategy concurrently quantizes both weights and activations into 8-bit integers, aiming to improve accuracy rather than compromise it during quantization. Notably, the method also quantizes nonlinear functions for efficient hardware implementation. Fourth, we design EQ-ViT automation framework to implement the EQ-ViT architecture for four different ViT applications on the AMD Versal ACAP VCK190 board, achieving accuracy improvement with 2.4%, and average speedups of 315.0x, 3.39x, 3.38x, 14.92x, 59.5x, 13.1x over computing solutions of Intel Xeon 8375C vCPU, Nvidia A10G, A100, Jetson AGX Orin GPUs, and AMD ZCU102, U250 FPGAs. The energy efficiency gains are 62.2x, 15.33x, 12.82x, 13.31x, 13.5x, 21.9x.
Overview
CoChat Summary
TL;DR
EQ-ViT is an algorithm-hardware co-design framework for real-time Vision Transformer inference on AMD Versal ACAP, combining INT8 quantization-aware training with heterogeneous AIE/FPGA spatial accelerators. It reports sub-1 ms latency on VCK190 for four ViT applications, with accuracy improvements over baselines and large speedups/energy-efficiency gains over CPU, GPU, and FPGA solutions.
Key findings
- On AMD Versal ACAP VCK190, EQ-ViT achieves latencies of 0.56 ms, 0.46 ms, 0.89 ms, and 0.61 ms for four ViT applications under a 1 ms latency budget with batch size 6.
- Average speedups are 315.0x, 3.39x, 3.38x, 14.93x, 59.5x, and 13.1x over Intel Xeon 8375C vCPU, Nvidia A10G, A100, Jetson AGX Orin GPUs, and AMD ZCU102, U250 FPGAs; energy-efficiency gains are 62.2x, 15.33x, 12.82x, 13.31x, 13.5x, and 21.9x.
- EQ-ViT improves accuracy up to 2.4% over baseline, and over Q-ViT by 0.4%–0.9% on ImageNet, 0.3%–0.4% on Cifar-100, and 0.2% on Cifar-10; Cifar-100 and Cifar-10 gains over baseline are up to 1.4% and 0.8%, respectively.
- Hardware-efficient nonlinear quantization saves Softmax DSP and LUT by 7.0x and 14.48x, LUTMem by 18834x, and total RAM by 9.19x; GeLU reduces REG and LUT by 137x and 142x. The MIP model predicts on-board latency with less than 4% error, and all four cases use over 98.5% AIE.
Methods
- Algorithm-hardware co-design: activation-aware QAT quantizes weights and activations to INT8 and quantizes nonlinear functions; hardware maps MM/BMM kernels to AIEs and non-MM kernels to PL, with on-chip data forwarding, multiple spatial accelerators, and fine-grained pipelining.
- Data and training: experiments use ImageNet-1k, Cifar-100, and Cifar-10 with DeiT variants and LV-ViT; training is in PyTorch on 4 NVIDIA V100 GPUs, with outlier-predictable training following Q-ViT with DGD.
- Evaluation: implemented on AMD Versal ACAP VCK190 and compared against CPU, GPU, and FPGA solutions; analysis includes latency, energy efficiency, accuracy versus PTQ/QAT methods, resource utilization, and MIP model versus on-board implementation.
Limitations
- The paper states that if a model cannot fit on a single board, partitioning it onto multiple devices is left as future work.
- The A8W4 results on AMD Versal VEK280 are described as estimated, not measured on hardware.
- Evident from the text: evaluation is limited to four ViT applications and primarily one ACAP board, and the accuracy advantage over full-precision models is partly attributed to DGD distillation integrated into the training setting.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 Valid · 4 Not valid · 14 Not checked · 78 Own results and interpretations
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“HeatViT and Auto-ViT-Acc fail to consider the inherent data distribution within ViTs, resulting in inconsistencies between the quantization strategy and the data distribution.”
§ EQ-ViT: Algorithm-Hardware Co-Design for End-to-End Acceleration of Real-Time Vision Transformer Inference on Versal ACAP Architecture > Background and Related Works > Transformer Accelerators on Hardware
What the sources say
- Could not verifycites [30] sourceExperimental — accuracy not yet measured
Our copy of HeatViT describes its quantization only as uniform 8-bit fixed-point on weights and activations applied after token pruning (Experimental Setup, Sec. VII-A) plus a regularization effect on quantization error introduced through polynomial approximation of GELU/Softmax (Sec. V-D, V-E, Eq. 15-17). No passage in our copy discusses the inherent/data distribution of activations inside ViTs or admits any inconsistency between its quantization strategy and that distribution; searching 'data distribution', 'activation distribution', 'quantization strategy' and 'limitation' returned nothing on point. The internal characteristics HeatViT does analyze (token redundancy across attention heads and transformer blocks, Sec. III-A, Fig. 5-6) concern pruning, not quantization, so they neither support nor contradict the claim; the criticism is the citing authors' own assessment. Auto-ViT-Acc is a separate work and is not addressed here. (LLM, conf=0.68)
- Disputedcites [41] sourceExperimental — accuracy not yet measured
The Proposed ViT Quantization Algorithm (p. 3) says Auto-ViT-Acc assigns quantization schemes based on the distribution of ViT weights: rows with smaller variance receive power-of-two quantization, while others receive fixed-point quantization. That contradicts the claim that Auto-ViT-Acc fails to consider data distribution when choosing a quantization strategy. This source does not establish the claim about HeatViT. (LLM, conf=0.87)
Checked by a reasoning model — may be wrong.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Notably, EQ-ViT also surpasses Q-ViT accuracy under the same training conditions.”
§ EQ-ViT: Algorithm-Hardware Co-Design for End-to-End Acceleration of Real-Time Vision Transformer Inference on Versal ACAP Architecture > Experiments > Inference Accuracy Comparisons
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Weight: Data follows a standard normal distribution (Figure 8 (a)).”
§ EQ-ViT: Algorithm-Hardware Co-Design for End-to-End Acceleration of Real-Time Vision Transformer Inference on Versal ACAP Architecture > EQ-ViT Algorithm > Discovery of Data Distribution within ViTs
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Activation: Two key features impact the quantization strategy, long-tail distribution and channel-wise outliers.”
§ EQ-ViT: Algorithm-Hardware Co-Design for End-to-End Acceleration of Real-Time Vision Transformer Inference on Versal ACAP Architecture > EQ-ViT Algorithm > Discovery of Data Distribution within ViTs
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We propose EQ-ViT architecture and our implemented EQ-ViT design on AMD Versal ACAP VCK190 achieves a latency as low as 0.56 ms, which has 3.2x latency improvement over A10G GPU and 13.1x over U250 FPGA.”
§ EQ-ViT: Algorithm-Hardware Co-Design for End-to-End Acceleration of Real-Time Vision Transformer Inference on Versal ACAP Architecture > Introduction > Low-latency real-time application scenarios.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
40 more claims
Create a free account to see all 96 claims and to verify the 3 references still open.
- It may state two separate claims as one. 45
- The quote only partly states this claim. 4
- Probably not a substantial claim. 7
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ EQ-ViT: Algorithm-Hardware Co-Design for End-to-End Acceleration of Real-Time Vision Transformer Inference on Versal ACAP Architecture > EQ-ViT Algorithm > Nonlinear Operations Quantization
“(12) AttnQ=log2Q(Attn∣b)=clip(⎿−log2Σl=1L2X^l+X^i⏋,0,2b−1),”
§ EQ-ViT: Algorithm-Hardware Co-Design for End-to-End Acceleration of Real-Time Vision Transformer Inference on Versal ACAP Architecture > EQ-ViT Algorithm > Activation-aware QAT
“(8) AttnQ=log2Q(Attn∣b)=clip(⎿−log2(Attn)⏋,0,2b−1),”
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 1×
“This sequence is then passed through an L -layer Transformer encoder [45].”
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 1×
“Application and Training Framework Setup: Our experiments are conducted on the ImageNet-1k [66], Cifar-100, and Cifar-10 [67] datasets in PyTorch 3.8.”
InfluentialAn Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472 · cited 3×
“…(ViTs) [1], [2], [3] have shown remarkable versatility in a broad range of application domains, including computer vision (e.g., image classification [1], [3], object detection [4], [5], image processing [6], and video understanding [7]), and in complex scenarios that involve the multimodal data.”
BackgroundSwin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534 · cited 2×
“…(ViTs) [1], [2], [3] have shown remarkable versatility in a broad range of application domains, including computer vision (e.g., image classification [1], [3], object detection [4], [5], image processing [6], and video understanding [7]), and in complex scenarios that involve the multimodal data.”
BackgroundLearning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,395
End-to-End Object Detection with Transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_13 · Cited by 18,285 · cited 1×
“…(ViTs) [1], [2], [3] have shown remarkable versatility in a broad range of application domains, including computer vision (e.g., image classification [1], [3], object detection [4], [5], image processing [6], and video understanding [7]), and in complex scenarios that involve the multimodal data.”
Deformable DETR: Deformable Transformers for End-to-End Object Detection
Zhu, X., Su, W., Lu, L., Li, B., Wang, X., & Dai, J. (2020) · arXiv · doi:10.48550/arxiv.2010.04159 · Cited by 6,027
Swin-Unet: Unet-like pure transformer for medical image segmentation
Cao, H., Wang, Y., Chen, J., Jiang, D., Zhang, X., Tian, Q., & Wang, M. (2023) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-25066-8_9 · Cited by 5,640
Self-citationPyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions
Wang, W., Xie, E., Li, X., Fan, D.-P., Song, K., Liang, D., Lu, T., Luo, P., & Shao, L. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00061 · Cited by 5,259 · cited 1×
“Many networks [1], [8], [9], [10] use ViTs as the backbone [8], [9] and show superior transferability to various downstream tasks with minor fine tuning.”
Background
Research works printed but not linked: 11
- Rafie, 2021, Autonomous vehicles drive AI advances for edge computing.
- 2023, Colliding Particles Not Cars: CERN’S Machine Learning Could Help Self-Driving Cars
- Vanholder, Efficient inference with tensorRT, 1, 1
- 2024, Nvidia Nsight systems
- Nvidia TensorRT documentation
- contributors, 2022, 68–95–99.7 rule
- System management interface SMI | NVIDIA developer
- ViT Github
- 2023, Board Evaluation and Management Tool
- NVIDIA H100 tensor core GPU architecture
- Ma, Rammer: Enabling holistic deep learning compiler optimizations with {rTasks}, 881
Cited by
Saha, S., & Xu, L. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.130417 · Cited by 116
ARIES: An Agile MLIR-Based Compilation Flow for Reconfigurable Devices with AI Engines
Zhuang, J., Xiang, S., Chen, H., Zhang, N., Yang, Z., Mao, T., Zhang, Z., & Zhou, P. (2025) · ACM/SIGDA International Symposium on Field-Programmable Gate Arrays (FPGA) · doi:10.1145/3706628.3708870 · Cited by 23
Self-citationART: Customizing Accelerators for DNN-Enabled Real-Time Safety-Critical Systems
Ji, S., Chen, X., Zhuang, J., Zhang, W., Yang, Z., Schultz, S., Song, Y., Hu, J., Jones, A. K., Dong, Z., et al. (2025) · ACM Great Lakes Symposium on VLSI (GLSVLSI) · doi:10.1145/3716368.3735215 · Cited by 8
Self-citationGAMA: High-Performance GEMM Acceleration on AMD Versal ML-Optimized AI Engines
Mhatre, K. M., Taka, E., & Arora, A. (2025) · International Conference on Field Programmable Logic and Applications (FPL) · doi:10.1109/fpl68686.2025.00051 · Cited by 6
“Since the introduction of AI engines by AMD [2], a wide variety of workloads have been deployed on them [7], [9], [11], [13], [17]–[19], [23].”
High-Precision and Efficiency Hardware Implementation for GELU via Its Internal Symmetry
Huang, J., Wu, Y., Zhuang, M., & Zhou, J. (2025) · Electronics · doi:10.3390/electronics14091825 · Cited by 5
Metadata
- Authors
- Dong, P., Zhuang, J., Yang, Z., Ji, S., Li, Y., Xu, D., Huang, H., Hu, J., Jones, A. K., Shi, Y., Wang, Y., & Zhou, P.
- Affiliations
- Northeastern University (US)
- Brown University (US)
- North Carolina State University (US)
- University of Maryland, College Park (US)
- University of Pittsburgh (US)
- Syracuse University (US)
- University of Notre Dame (US)
- ORCID
- Venue
- IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
- Published
- 2024-11
- Volume
- 43
- Issue
- 11
- Pages
- 3949-3960
- ISSN
- 0278-0070 (ISSN-L)
- 1937-4151
- PMC
- PMC12327521
- OpenAlex
- W4404102506
- Semantic Scholar
- 1ec9b653475287e95fdaef2f5247f82a8376c56c
- Cited by
- 35
- References
- 86
- Citations by year
- 2026: 17 · 2025: 16 · 2024: 2
- FWCI
- 4.88
- Type
- journal article
- Language
- en
- Open access
- green
- pmc.ncbi.nlm.nih.gov
- Licence
- Locations
- IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- Funders
- NSF (2213701, 2217003, 2133267, 2122320, 2324864, 2324937, 2328972)
- NIH (R01EB033387)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Hardware and Architecture › Embedded Systems Design Techniques
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Embedded Systems and FPGA Applications
- Keywords
- algorithm-hardware co-design, FPGA acceleration, energy efficiency, system-on-chip
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: EQ-ViT: Algorithm-Hardware Co-Design for End-to-End Acceleration of Real-Time Vision Transformer Inference on Versal ACAP Architecture. https://app.cochat.ai/papers/10.1109/tcad.2024.3443692
- The paper
- Dong, P., Zhuang, J., Yang, Z., Ji, S., Li, Y., Xu, D., Huang, H., Hu, J., Jones, A. K., Shi, Y., Wang, Y., & Zhou, P. (2024). EQ-ViT: Algorithm-Hardware Co-Design for End-to-End Acceleration of Real-Time Vision Transformer Inference on Versal ACAP Architecture. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems. https://doi.org/10.1109/tcad.2024.3443692
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.