Cited by 569Parallel Computing and Optimization TechniquesDistributed and Parallel Computing SystemsEmbedded Systems Design Techniques

The DOI matches its Crossref record; 199 of the 199 references it cites were found in a registry or an index; 5 of the 49 sources its claims cite were checked for support.

A Survey of CPU-GPU Heterogeneous Computing Techniques

Open access
Mittal, Sparsh, Vetter, Jeffrey S.·2015·ACM Computing Surveys

Abstract

As both CPUs and GPUs become employed in a wide range of applications, it has been acknowledged that both of these Processing Units (PUs) have their unique features and strengths and hence, CPU-GPU collaboration is inevitable to achieve high-performance computing. This has motivated a significant amount of research on heterogeneous computing techniques, along with the design of CPU-GPU fused chips and petascale heterogeneous supercomputers. In this article, we survey Heterogeneous Computing Techniques (HCTs) such as workload partitioning that enable utilizing both CPUs and GPUs to improve performance and/or energy efficiency. We review heterogeneous computing approaches at runtime, algorithm, programming, compiler, and application levels. Further, we review both discrete and fused CPU-GPU systems and discuss benchmark suites designed for evaluating Heterogeneous Computing Systems (HCSs). We believe that this article will provide insights into the workings and scope of applications of HCTs to researchers and motivate them to further harness the computational powers of CPUs and GPUs to achieve the goal of exascale performance.

Overview

AI snapshot

Written from the paper’s full text.

TL;DR

This survey reviews CPU-GPU heterogeneous computing techniques (HCTs) that use both processing units to improve performance and/or energy efficiency. It classifies approaches across algorithm/runtime, programming/compiler, energy, fused-vs-discrete architecture, application-domain, and benchmark perspectives.

Key findings

  • HCTs are classified by scheduling nature (dynamic vs. static) and workload-partitioning basis, including relative PU performance/load balancing, subtask parallelism or characteristics, floating-point precision, and GPU memory-size limits; pipelining and MapReduce use are also identified.
  • Hardware trends show CPU core counts rising to 8–32 cores with >60 likely, CUDA cores increasing from 240 (GTX 280) to 3072 (GTX TITAN X), CPU LLC growing from 32MB (POWER7) to 96MB (POWER8), and GPU L2 caches from 768KB (Fermi) to 2048KB (Maxwell); NVLink promises 5–12× PCIe Gen3 bandwidth.
  • Fused CPU-GPU systems reduce data-transfer overhead: Daga et al. report an AMD E-series Zacate APU with only 80 GPU cores providing >3× improvement over a discrete Radeon HD 5870 with 1600 more powerful GPU cores; Spafford et al. find fused systems can have better energy efficiency due to low transfer overhead, but higher CPU-GPU contention and lower compute-intensive CPU performance than discrete designs.
  • Energy-saving approaches include intelligent workload partitioning, DVFS on CPU and/or GPU, and resource scaling/low-power modes; benchmark suites include SHOC, Parboil, Rodinia, and Valar. The June 2014 Green500 top 15 systems are all CPU-GPU heterogeneous, and Yang et al. achieve 0.8 PetaFLOP on Tianhe-1A for global atmospheric simulation.

Methods

  • Literature survey of heterogeneous computing techniques using both CPU and GPU, covering discrete and fused CPU-GPU architectures; scope excludes FPGA-based heterogeneous systems, GPU-only performance works, load balancing within a single GPU or across multiple GPUs only, and circuit/device/microarchitectural research.
  • Works are organized by abstraction level and theme: algorithm/runtime workload partitioning, programming languages/frameworks/compiler techniques, energy efficiency, fused CPU-GPU chips, application domains, and benchmark suites, with classification tables.
  • No primary experiments are conducted; the paper synthesizes cited research and reports their evaluation platforms and performance/energy results where available.

Limitations

  • The paper states scope limitations: it includes only CPU-GPU systems/techniques, excludes FPGA-based heterogeneous systems, GPU-only works, single-GPU or multi-GPU-only load balancing, and circuit/device/microarchitectural works; it focuses on key research ideas while only mentioning CPU/GPU platforms used in cited experiments.
  • Evident from the text: as a survey, it provides no new experimental validation, and its classifications and conclusions depend on the selected literature and reported results, which may not generalize across all HCSs.
  • The paper also identifies open needs: fully automatic toolchains, lower-overhead porting of existing CPU/GPU codes, order-of-magnitude energy-efficiency improvement for mobile systems, and more extensive evaluation/development of fused HCSs.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1145/2788396

Cites

199 of 199 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

5 of 49 cited sources checked · 4 supported · 0 flagged · accuracy not yet measured

Show all 24 claims

  • It may state two separate claims as one. 26
  • Probably not a substantial claim. 27
40 of 40
  • The paper relies on this without support.

    Modern multicore CPUs use up to a few tens of cores, which are typically out-of-order, multi-instruction issue cores; CPU cores run at high frequency and use large sized caches to minimize the latency of a single-thread.

    “Modern multicore CPUs use upto a few tens of cores, which are typically out-of-order, multi-instruction issue cores.Also, CPU cores run at high-frequency and use large sized caches to minimize the latency of a single-thread.”
  • The paper relies on this without support.

    After allocating a task to GPU (starting the kernel), CPU stays idle, leading to wastage of energy.

    “Further, after allocating the task to GPU (i.e. starting the kernel), CPU stays idle which leads to wastage of energy.”
  • The authors state this as an assumption.

    The different subtasks are of similar nature and can run on any of the PU.

    “For example, assuming that the different subtasks are of similar nature and can run on any of the PU, the decision about where a subtask should be mapped depends on which PU provides higher performance and which mapping helps in achieving load-balancing.”
  • The paper relies on this without support.

    Different languages offer different tradeoffs between ease of programming, ability to write optimized code, ability to target multiple PUs and products from different vendors etc.

    “Different languages offer different tradeoffs between ease of programming, ability to write optimized code, ability to target multiple PUs and products from different vendors etc.”
  • The paper relies on this without support.

    OpenCL can be used for both CPU and GPU, and hence, applications written in OpenCL can be easily scheduled on any PU.

    “For example, OpenCL can be used for both CPU and GPU, and hence, applications written in OpenCL can be easily scheduled on any PU.”
  • The paper relies on this without support.

    CUDA works only on NVIDIA GPUs.

    “By comparison, CUDA works only on NVIDIA GPUs”
  • The paper relies on this without support.

    ACML and Brook+ can be used to program AMD GPUs only.

    “ACML (AMD Core Math Library) and Brook+ can be used to program AMD GPUs only.”
  • The paper relies on this without support.

    OpenMP, Pthread, TBB and MKL are used to write parallel programs on CPUs.

    “Similarly OpenMP, Pthread, TBB (thread building block) and MKL (math kernel library) are used to write parallel programs on CPUs.”
  • The paper relies on this without support.

    MKL is internally parallelized by OpenMP threading.

    “Note that MKL is internally parallelized by OpenMP threading; for sake of clarity, we mention it separately in Table III.”
  • The paper relies on this without support.

    Small task sizes provide better load-balancing, while large task sizes provide better resource utilization and an optimal value of task sizes achieves a trade-off between these two factors.

    “Small task sizes provide better load-balancing, while large task sizes provide better resource utilization and an optimal value of task sizes achieves a trade-off between these two factors.”
  • The paper relies on this without support.

    HMMER algorithm has large memory requirement and thus, for large sized sequences, shared memory of GPU becomes unsuitable while global memory incurs large performance penalty.

    “HMMER algorithm has large memory requirement and thus, for large sized sequences, shared memory of GPU becomes unsuitable while global memory incurs large performance penalty.”
  • The paper relies on this without support.

    Discrete GPUs have separate memory spaces from the CPU.

    “Discrete GPUs have separate memory spaces from the CPU and hence, data transfer happens over a connection bus (e.g.PCIe bus), which incurs large overhead.”
12 of 12

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Dithering step 3 is assigned to both GPU and CPU

    Minor

    The handover description says step 3 runs on the GPU, but the subsequent description says the CPU performs steps 1 and 3. The account gives conflicting assignments for the final step.

1 of 1

Citations

Cites

Flagged first, then by citations

Research works printed but not linked: 12

  1. Chris Gregg, Jeff Brantley, and Kim Hazelwood. 2010. Contention-aware scheduling of parallel code for heterogeneous systems. In Proceedings of the USENIX Workshop on Hot Topics in Parallelism (HotPar’10).
  2. 10.5555/2015039.2015535 · DOI
  3. 10.5555/1987237.1987259 · DOI
  4. 10.5555/1885276.1885302 · DOI
  5. 10.5555/1964238.1964251 · DOI
  6. 10.5555/1761728.1761755 · DOI
  7. Usman Pirzada. 2015. Nvidia Geforce GTX TITAN X Unveiled - GM200 ‘Big Daddy Maxwell’, 12GB VRam and 8 Billion Transistors. Retrieved from wccftech.com/nvidia-gtx-titan-x-revealed-gdc-2015/.
  8. 10.5555/1855610.1855621 · DOI
  9. 10.5555/1885276.1885305 · DOI
  10. 10.5555/2001252.2001258 · DOI
  11. 10.5555/1863086.1863099 · DOI
  12. Dieter Wendel, Ronald Kalla, Joshua Friedrich, James Kahle, Jens Leenstra, Cedric Lichtenau, Balaram Sinharoy, William Starke, and Victor Zyuban. 2010. The POWER7 processor SoC. In Proceedings of the International Conference on IC Design and Technology (ICICDT’10). 71--73.

Cited by

Sorted by citations

See all 569

Metadata

Authors
Mittal, S., & Vetter, J. S.
Affiliations
Oak Ridge National Laboratory (US)
Venue
ACM Computing Surveys
Published
2015-07-21
Volume
47
Issue
4
Pages
1-35
ISSN
  • 0360-0300 (ISSN-L)
  • 1557-7341
OpenAlex
W1864199185
Cited by
568
References
197
Citations by year
2026: 43 · 2025: 51 · 2024: 57 · 2023: 46 · 2022: 45 · 2021: 58 · 2020: 61 · 2019: 67 · 2018: 68 · 2017: 35 · 2016: 35 · 2015: 2
FWCI
71.01
Type
journal article
Language
en
Open access
Locations
  • ACM Computing Surveys (publishedVersion, public-domain)
  • OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) (submittedVersion)
Funders
  • U.S. Department of Energy
  • Office of Science
  • Advanced Scientific Computing Research
Topics
  • Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
  • Physical Sciences › Computer Science › Computer Networks and Communications › Distributed and Parallel Computing Systems
  • Physical Sciences › Computer Science › Hardware and Architecture › Embedded Systems Design Techniques
Keywords
CPU-GPU heterogeneous computing, workload partitioning, heterogeneous supercomputers, energy efficiency, benchmark suite, exascale computing

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-11-08, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-11-08 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review