Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index; 4 of the 23 sources its claims cite were checked for support.

Review of large YOLOv8 and RT-DETR energy efficiency on edge devices for real-time detection

Open access
Suchý, Ivan, Turčaník, Michal · 2026 · Scientific Reports

Abstract

Deploying large object detectors on embedded edge platforms is governed by a joint trade-off among detection accuracy, end-to-end throughput, and total system power. This paper benchmarks the large variants YOLOv8l and RT-DETR-l across heterogeneous deployment runtimes on two edge platforms: Raspberry Pi 5 with CPU and NPU offload, and Nvidia Jetson Orin NX with GPU acceleration. Accuracy is evaluated on COCO val2017 using mAP50-95, while throughput and energy efficiency (FPS/W) are measured on a realistic end-to-end video pipeline that includes decoding, preprocessing, inference, and post-processing. Model-execution latency is analyzed separately from pipeline throughput to avoid ambiguity between inference time and end-to-end processing rate. On Raspberry Pi 5, CPU-only execution of large models is impractical due to multi-second per-frame latency, whereas NPU acceleration substantially improves energy efficiency for YOLOv8l, albeit with deployment constraints that can reduce accuracy. On Jetson Orin NX, TensorRT provides the strongest deployment path for both architectures; however, the relative ranking of YOLOv8l and RT-DETR-l depends on runtime realization rather than on nominal FLOPs alone. A generalized interpretation based on normalized latency and energy per nominal GFLOP, together with a decomposition of conversion sensitivity and quantization sensitivity, shows that deployment efficiency is jointly determined by nominal compute demand, memory-system behavior, runtime overhead, and accuracy retention after export and quantization. Under the tested conditions, none of the evaluated large-model configurations reaches the strict real-time target of 25 FPS for the full pipeline, indicating that further hardware-specific optimization and/or smaller model variants are still required for real-time edge deployment.

Overview

CoChat Summary

TL;DR

This paper benchmarks large YOLOv8l and RT-DETR-l across edge runtimes on Raspberry Pi 5 and Nvidia Jetson Orin NX, measuring COCO accuracy and end-to-end FPS/W on a realistic video pipeline. It finds deployment efficiency depends jointly on architecture, runtime, hardware, and post-training conversion/quantization, and no evaluated large-model configuration reached the strict ≥25 FPS real-time target.

Key findings

  • On Raspberry Pi 5, CPU-only execution of large detectors is impractical due to multi-second per-frame latency, with efficiency dropping as low as 0.0313 FPS/W. Hailo-8 gives the highest measured efficiency for YOLOv8l at 1.7281 FPS/W, and HailoRT preserves 0.541 mAP50–95; Edge TPU exceeds 1.3 FPS/W but falls to 0.324 mAP50–95 because of mandatory INT8 quantization and reduced 224 × 224 input resolution.
  • On Jetson Orin NX, TensorRT with INT8 quantization achieves the highest efficiency: 0.9960 FPS/W for YOLOv8l and 1.0597 FPS/W for RT-DETR-l, with sub-100 ms inference times for the evaluated large-model configurations and approximately 90% GPU utilization for both models.
  • The ranking is runtime-dependent: YOLOv8l is faster under PyTorch, whereas under TensorRT RT-DETR-l achieves the highest throughput among evaluated Jetson configurations. RT-DETR-l benefits more from compiled execution, but remains less efficient per nominal GFLOP than YOLOv8l; YOLOv8l maintains lower latency and lower energy per nominal workload together with stronger cross-runtime accuracy stability.
  • Accuracy loss decomposition shows RT-DETR-l has a larger relative loss in the PyTorch-to-TensorRT FP32 step, while YOLOv8l loses a larger fraction in the subsequent FP32-to-INT8 step; the larger overall PyTorch-to-TensorRT INT8 reduction for RT-DETR-l reflects combined conversion and quantization effects. None of the evaluated configurations reached ≥25 FPS.

Methods

  • Benchmarked pre-trained YOLOv8l and RT-DETR-l. Accuracy was evaluated on COCO val2017 (5,000 images, 80 categories) using mAP50–95, mAP50, and mAP75. Throughput and energy efficiency were measured on the MOT16-13 benchmark video (750 frames, 1920 × 1080 at 25 FPS) using an end-to-end pipeline including video decoding, preprocessing, inference, and post-processing.
  • Platforms: Raspberry Pi 5 (8GB) running Debian GNU/Linux 12 (kernel 6.6.62) with Coral USB Accelerator and Raspberry Pi AI HAT+; Nvidia Jetson Orin NX (16GB) running L4T 36.4.0 with JetPack 6.1 in MAXN power mode. Software included Ultralytics 8.3, TensorFlow 2.17/2.16, PyTorch 2.5, and specialized runtimes (TensorRT, EdgeTPU, HailoRT, NCNN). Standard factory configurations were used without overclocking or jetson_clocks.
  • Power was measured with an OWON SPE8205 programmable DC power supply and a Fnirsi FNB48P USB power meter, reporting Total System Power. Energy efficiency is FPS/W = FPS / average power. Model-execution latency was analyzed separately from end-to-end throughput; real-time was defined as sustained end-to-end throughput ≥25 FPS, corresponding to ≤40 ms per frame.

Limitations

  • Stated: the generalized interpretation is an analytical model for interpreting measured results, not a direct profiler-based attribution of every kernel-level contribution. No configuration reached ≥25 FPS, so further hardware-specific optimization, reduced pipeline overhead, and/or smaller detector variants remain necessary. RT-DETR could not be deployed on Edge TPU or Hailo-8 due to conversion and operator-support limitations for transformer-based architectures.
  • Evident from the text: the study evaluates only two large models (YOLOv8l and RT-DETR-l) on two edge platforms, uses one benchmark video (MOT16-13) for throughput/energy, and does not fine-tune models, instead exporting pre-trained weights directly. Configurations with mAP50–95 below 0.50 are treated as accuracy-limited for practical use, and standard factory configurations may not reflect fully optimized deployments.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1038/s41598-026-46453-6

Cites

26 of 26 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1 Connection diagram for measurements of Raspberry Pi 5.
Fig. 1 Connection diagram for measurements of Raspberry Pi 5.CC BY-NC-ND · PubMed Central
Fig. 2 Inference time on Raspberry Pi 5.
Fig. 2 Inference time on Raspberry Pi 5.CC BY-NC-ND · PubMed Central
Fig. 3 Inference time on Jetson Orin NX.
Fig. 3 Inference time on Jetson Orin NX.CC BY-NC-ND · PubMed Central
Fig. 4 CPU utilization during inference on Raspberry Pi 5.
Fig. 4 CPU utilization during inference on Raspberry Pi 5.CC BY-NC-ND · PubMed Central
Fig. 5 CPU and GPU utilization during inference on Jetson Orin NX.
Fig. 5 CPU and GPU utilization during inference on Jetson Orin NX.CC BY-NC-ND · PubMed Central
Fig. 6 YOLO Efficiency (FPS/W) on Raspberry Pi 5.
Fig. 6 YOLO Efficiency (FPS/W) on Raspberry Pi 5.CC BY-NC-ND · PubMed Central
Fig. 7 RT-DETR Efficiency (FPS/W) on Raspberry Pi 5.
Fig. 7 RT-DETR Efficiency (FPS/W) on Raspberry Pi 5.CC BY-NC-ND · PubMed Central
Fig. 8 YOLO Efficiency (FPS/W) on Nvidia Jetson Orin NX.
Fig. 8 YOLO Efficiency (FPS/W) on Nvidia Jetson Orin NX.CC BY-NC-ND · PubMed Central
Fig. 9 RT-DETR Efficiency (FPS/W) on Nvidia Jetson Orin NX.
Fig. 9 RT-DETR Efficiency (FPS/W) on Nvidia Jetson Orin NX.CC BY-NC-ND · PubMed Central
Fig. 10 Pareto frontier of detection accuracy versus energy efficiency across embedded platforms.
Fig. 10 Pareto frontier of detection accuracy versus energy efficiency across embedded platforms.CC BY-NC-ND · PubMed Central

Verification

Claims the paper makes

4 Valid · 2 Not valid · 11 Not checked · 57 Own results and interpretations

35 more claims

Create a free account to see all 74 claims and to verify the 1 reference still open.

See all claims — free account
  • Probably not a substantial claim. 14
  • Borderline: one of the tests for a substantial claim is unclear. 3
  • It may state two separate claims as one. 23

Create a free account to see these claims.

Citations

Cites

Show all 26 references

Research works printed but not linked: 1

  1. Vaswani, A. et al. Attention is all you need, in Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, California, USA, pp. 6000–6010. (2017).

Metadata

Authors
Suchý, I., & Turčaník, M.
Affiliations
Armed Forces Academy (SK)
Corresponding author
Turčaník, Michal
Venue
Scientific Reports
Published
2026-03-27
Volume
16
Issue
1
ISSN
2045-2322 (ISSN-L)
OpenAlex
W7141095983
Cited by
14
References
26
Citations by year
2026: 14
FWCI
38.50
Type
journal article
Language
en
Open access
Licence
Locations
  • Scientific Reports (publishedVersion, cc-by-nc-nd)
  • PubMed Central (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
Ministerstvo obrany Slovenskej republiky
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
  • Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
Keywords
YOLOv8, RT-DETR, energy efficiency, edge deployment, real-time object detection, Raspberry Pi 5, GPU acceleration, TensorRT, model quantization, inference latency

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Review of large YOLOv8 and RT-DETR energy efficiency on edge devices for real-time detection. https://app.cochat.ai/papers/10.1038/s41598-026-46453-6
The paper
Suchý, I., & Turčaník, M. (2026). Review of large YOLOv8 and RT-DETR energy efficiency on edge devices for real-time detection. Scientific Reports. https://doi.org/10.1038/s41598-026-46453-6

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.