The DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index; none of the 39 sources its claims cite is checked for support yet.
Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems
Open accessAbstract
Driven by the increasing demands of machine learning, heterogeneous systems combining CPUs and GPUs have emerged as the dominant architecture for parallel computing in recent years. To optimize memory management and data transfer between CPUs and GPUs, Nvidia GPUs have introduced unified virtual memory ( UVM ) and pinned memory ( PM ) over the last decade. UVM can avoid explicit memory copies and potentially overlap GPU kernel computations with CPU-GPU data transfer. PM ensures that data with high locality remains in the main memory, preventing it from being paged out. In addition to these two techniques, asynchronous memory copy ( Async Memcpy ) was introduced recently in Nvidia GPUs to improve the CPU-GPU pipeline further. By utilizing Async Memcpy , the data transfer from GPU global memory to shared memory can be overlapped with GPU computations, adding an additional stage to the CPU-GPU data transfer pipeline. A thorough performance analysis of how Async Memcpy affects the current UVM and PM CPU-GPU data transfer scheme is desired. In this article, we provide performance implications of the combined effect of UVM , PM , and Async Memcpy , exploring which applications benefit from which combination of these features. We implement all these features on a suite of 25 workloads, including microbenchmarks and realworld applications. We observe an average performance gain of 24% when utilizing UVM and a 34% gain when employing PM on realworld applications, compared to not applying any data transfer optimization techniques. The performance benefits of Async Memcpy vary across different workloads. For workloads featuring extensive shared memory usage and high compute density (e.g., kmeans and lud ), Async Memcpy delivers around a 20% performance improvement over using UVM or PM alone. In other workloads like knn , we note a 20% performance degradation when using Async Memcpy . Furthermore, we conduct an in-depth investigation of the GPU kernel using performance counters to uncover the root causes of performance differences among various data transfer models. We also perform sensitivity analyses to examine how the number of blocks and threads, as well as the L1-cache/shared memory partitioning, impact performance. We explore future research directions aimed at enhancing the data transfer pipeline by overlapping memory allocation with data transfer and computation across GPU kernels.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
This paper characterizes how pinned memory (PM), unified virtual memory (UVM), and asynchronous memory copy (Async Memcpy) affect CPU-GPU data-transfer pipelines across 25 workloads and four machine configurations. It finds that UVM/PM benefits are highly machine-dependent, Async Memcpy is workload-dependent, and combining them helps only when each technique helps individually.
Key findings
- UVM and PM are sensitive to machine system configuration. On Machine-C, UVM improved microbenchmark performance by 48.13% and 50.53% over baseline for Super and Mega inputs, while PM improved it by 57.82% and 56.63%; on Machine-B, UVM caused 93.43% and 84.67% slowdowns, and PM improved performance by only 16.99% and 18.56%. For real-world non-DL applications, UVM improved performance by 23.99% and 4.61% on Machine-C and Machine-D, while PM improved it by 33.90% and 22.80%; on Machine-B, UVM caused a 41.68% slowdown.
- Async Memcpy alone had little average overall effect: microbenchmarks slowed by 2.05% and 1.72% on Machine-C for Super and Mega inputs, and by 6.12% and 4.91% on Machine-D; real-world non-DL applications showed only 0.35% slowdown and 0.43% speedup on Machine-C and Machine-D. Workload-specific effects were large: vector_seq saw 42.42% GPU kernel time reduction but only 7.37% overall improvement, 3DCONV saw 143.37% kernel time increase, lud improved 20.03%, and knn slowed 25.22% on Machine-D.
- Combining Async Memcpy with UVM/PM can help when each helps individually. For Mega input on Machine-C, uvm_async gave 49.21% improvement over baseline versus 50.53% for uvm, and pm_async gave 55.28% versus 56.63% for pm; however, vector_seq and vector_rand with uvm_async achieved 63.50% and 63.37% time savings over baseline, better than uvm’s 59.98% and 61.25%.
- For DL workloads, PM did not improve inference or training, causing 196.43% and 25.30% slowdowns versus baseline, though it reduced CPU-GPU transfer time by 29.75% for inference and 49.14% for training. Async Memcpy had minimal effect on DL workloads, while UVM benefited yolov3 inference because CPU-GPU transfer exceeded 70% of total execution time.
Methods
- The study uses four machines: Machine-A (Intel Skylake, A100), Machine-B (AMD Milan, A100), Machine-C (Intel Sapphire Rapids, A100), and Machine-D (AMD Genoa, H100). Six configurations are compared: baseline, async, uvm, uvm_async, pm, and pm_async.
- The benchmark suite contains 25 workloads: 7 microbenchmarks and 18 real-world applications (10 non-DL and 8 DL, with training and inference counted separately). Each workload was run 30 times; main experiments focus on Machine-C and Machine-D with Super and Mega input sizes.
- Performance counters were collected with Nvidia Nsight Compute, Nsight Systems, and CUPTI. The paper also performs sensitivity studies on CUDA block count, threads per block, and L1-cache/shared-memory partition, and prototypes an inter-job pipeline that overlaps data allocation/deallocation with GPU kernel execution.
Limitations
- The paper states that its prototype for overlapping data allocation with kernel execution can cause out-of-memory errors for Mega input sizes because memory allocated by the running kernel is not released before allocating memory for the next kernel, doubling GPU memory capacity requirements. It also states that for tiny input sizes, the overhead of initializing a separate CUDA stream for data allocation exceeds the CUDA kernel cost.
- The paper states that PM can introduce overhead from frequent allocation/deallocation, especially for DL frameworks that allocate PM per tensor or per layer, and that UVM benefits depend heavily on machine system configuration and are best assessed through runtime profiling.
- Evident from the text: the characterization focuses on single-GPU scenarios and selected benchmark suites, and the paper notes that no automatic compiler tools exist for converting programs to PM, UVM, or Async Memcpy versions, so programmers must hand-tune CUDA programs.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 of 39 cited sources checked · 0 supported · 0 flagged
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“When using Machine-B, there is a 41.68% slow down (considering the geo-mean of the 10 workloads) when using UVM compared with baseline for Mega input size.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > UVM/PM:
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For workloads whose performance cannot benefit from additional pipeline stages due to inefficient shared memory usage, such as 2DCONV, 3DCONV, and gemm, the extra control logic overhead of Async Memcpy can degrade performance”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > Async Memcpy+UVM/PM:
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Overall, When using UVM (with prefetch), there is 48.13% and 50.53% performance improvement over baseline on Machine-C for Super and Mega input sizes, respectively.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 4.1 Microbenchmarks
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesThe paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“A good usage of these features can help programs achieve up to 6× speedups, while an inappropriate usage may lead to up to 3× slow downs on certain workloads.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 1 Introduction
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For CPU-GPU data transfer bounded workloads (like vector _seq), the 42.42% kernel time reduction only results in 7.37% overall performance improvement.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 4.1 Microbenchmarks
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“async achieves 42.42% GPU kernel time reduction over baseline in the vector _seq workload while 143.37% increment over baseline in the 3DCONV workload (both with Mega input sizes on Machine-D).”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 4.1 Microbenchmarks
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesThe paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“A good usage of these features can help programs achieve up to 6× speedups, while an inappropriate usage may lead to up to 3× slow downs on certain workloads.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 1 Introduction
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“async achieves 42.42% GPU kernel time reduction over baseline in the vector _seq workload while 143.37% increment over baseline in the 3DCONV workload (both with Mega input sizes on Machine-D).”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 4.1 Microbenchmarks
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesThe paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Systems featuring newer CPU, DRAM, and PCIe generations with larger (GB-level) input sizes typically yield less noisy results.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 3.3 Benchmark and Machine Configurations
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Machine-C and Machine-D are more stable than Machine-A and Machine-B for all input sizes.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 9 Conclusion
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passes
- It may state two separate claims as one. 70
- The quote only partly states this claim. 1
- Probably not a substantial claim. 36
- Borderline: one of the tests for a substantial claim is unclear. 1
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Machine-D sits in the middle between Machine-B and Machine-C, the performance improvement of using PM (over baseline) is 43.43% and 45.02% for Super and Mega input sizes, respectively.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 4.1 Microbenchmarks
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Such overhead results in the average performance improvement of uvm_async over baseline is 49.21% for the Mega input size on Machine-C, which is slightly lower than uvm over baseline (50.53%).”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > Async Memcpy+UVM/PM:
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, on Machine-B, there is a 93.43% and 84.67% slowdown when comparing UVM with stanard for Super and Mega input sizes.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 4.1 Microbenchmarks
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For GPU kernel and CPU-GPU data transfer time, UVM achieves 79.90% and 82.24% time savings over baseline when using Super and Mega input sizes on Machine-C.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 4.1 Microbenchmarks
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Overall, When using PM, there is 57.82% and 56.63% performance improvement over baseline on Machine-C for Super and Mega input sizes, respectively.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 4.1 Microbenchmarks
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“On Machine-C and Machine-D, performance remains more stable across different input sizes because these newer systems incorporate updated CPUs, DRAM, and PCIe generations, yielding higher CPU-DRAM and CPU-GPU PCIe bandwidth.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 3.3 Benchmark and Machine Configurations
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Similarly, the performance improvement of pm_async is also slightly worse than async only (55.28% versus 56.63% for the Mega input size on Machine-C).”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > Async Memcpy+UVM/PM:
Strength:plain· not yet calibratedSingle caseTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“There is 23.99% and 4.61% performance improvement achieved by using UVM compared with using baseline on Machine-C and Machine-D, respectively.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > UVM/PM:
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For workloads vector_seq and vector_rand, uvm_async can make the time savings over baseline of the two workloads to 63.50% and 63.37%, which is better than the 59.98% and 61.25% time savings over baseline when using uvm (for the Mega input size on Machine-C).”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > Async Memcpy+UVM/PM:
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“On Machine-B, the performance improvement of using PM (over baseline) is only 16.99% and 18.56%.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 4.1 Microbenchmarks
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one. The quote only partly states this claim.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“PM also achieves 33.90% and 22.80% performance improvement over baseline on Machine-C and Machine-D, respectively.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > UVM/PM:
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“On average, there is a 196.43% slowdown for inference and a 25.30% slowdown for training compared to baseline.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 4.3 RealWorld Deep Learning Applications
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“UVM achieves 67.35% and 48.03% memcpy time savings compared with baseline on Machine-C and Machine-D.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > UVM/PM:
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Async Memcpy increases global memory throughput by 73.55% and 26.58% compared to the baseline configuration for the vec-tor_seq and lud workloads, respectively.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > Global Memory Throughput and Pipeline Stall Cycles.
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Although uvm_async generally outperforms uvm when standalone async usage yields performance gains, there are exceptions, such as srad on Machine-C.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > Async Memcpy+UVM/PM:
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“increasing shared memory from 2 KB to 64 KB consistently boosts performance, with improvements of 1.51%, 2.62%, 1.40%, 1.91%, 2.73%, and 3.53% across the configurations, respectively.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 7.2 L1-Cache/Shared Memory Partition
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“When combining these two techniques, lud maintains the same speedup as Async Memcpy only; it is not affected by UVM overhead.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > In addition, Async Memcpy also requires additional control instructions, making the performance even worse (on Machine-B and Machine-C).
Strength:plain· not yet calibratedSingle caseTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Given that yolov3 is the largest of the four networks, with CPU-GPU data transfer accounting for over 70% of the total execution time”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 4.3 RealWorld Deep Learning Applications
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“for workloads like 3DCONV and knn, which suffer performance degradation with Async Memcpy, global memory throughput decreases by 59.44% and 11.07%, respectively, compared with the baseline setup.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > Global Memory Throughput and Pipeline Stall Cycles.
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“For lud, the performance only benefits from Async Memcpy but not UVM (with prefetch).”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > In addition, Async Memcpy also requires additional control instructions, making the performance even worse (on Machine-B and Machine-C).
Tests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Although Async Memcpy is compatible with UVM/PM, the time devoted to GPU kernel execution remains relatively low, leaving GPUs idle for over 60% of the total execution time.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 8.1 Limitations of PM, UVM, and Async Memcpy
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“async performs much better than baseline (4.67% speedups on 1,024 threads, but 28.58% speedups on 32 threads).”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 7.1 CUDA Block and Thread
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Async Memcpy introduces 107.82%, 17.49%, and 43.40% more memory instructions for the 3DCONV, lud, and knn workloads, respectively.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 5.1.2 Other Kernel Performance Counters.
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Overall, Async Memcpy has minimal effect on the performance of these DL workloads, while UVM and PM significantly influence performance.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 4.3 RealWorld Deep Learning Applications
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“PM cannot improve the performance of either inference or training workloads.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 4.3 RealWorld Deep Learning Applications
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“There is a 20.03% performance improvement over baseline when using Async Memcpy on lud, while a 25.22% slowdown on knn (both results on Machine-D).”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > Async Memcpy:
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Overlapping data allocation with GPU kernel execution yields greater performance benefits for gemv, because gemm is more compute-bound and devotes most of its runtime to computation”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 8.3 Prototyping the New Data Transfer Model
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“UVM offers more promising improvements over the baseline configuration on Machine-C and Machine-D, but not on Machine-B.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 5.2 In-Depth Analysis for UVM and PM
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Because data transfers now take less time, a larger proportion of the execution is devoted to GPU kernel processing (from 41.84% to 59.25% and 63.80% for UVM and PM, respectively).”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 8.1 Limitations of PM, UVM, and Async Memcpy
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“On average, async, uvm, uvm_async, pm, and pm_async achieves 8.06%, 0.66%, 8.12%, 42.06%, and 50.19% performance improvement over baseline, respectively.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 7.1 CUDA Block and Thread
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Machine-D has higher global memory throughput and fewer kernel memory dependency stall cycles.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 9 Conclusion
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Similar to knn, the microbenchmark gemm also has similar characteristics”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > Async Memcpy:
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“While data allocation accounts for only 18.99% of total execution time under the baseline configuration, it increases to 37.66% when using Async Memcpy and UVM.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 8.1 Limitations of PM, UVM, and Async Memcpy
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“For instance, the uvm_async saves 8.38% GPU kernel time over baseline for workload kmeans on Machine-D, while uvm spends 9.94% more GPU kernel time over baseline.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > Async Memcpy+UVM/PM:
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The performance of Async Memcpy is more sensitive to the type of workload, e.g., benefits the performance of lud but not knn.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 9 Conclusion
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Async Memcpy reduces stall cycles due to memory dependency by 39.71% for the vector_seq workload and by 6.90% for the knn workload, compared to the baseline configuration.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > Global Memory Throughput and Pipeline Stall Cycles.
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Among the factors contributing to stalls, memory dependency and pipeline busy are the two counters that exhibit significant changes with Async Memcpy.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > Global Memory Throughput and Pipeline Stall Cycles.
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“On average, global memory throughput dropped by 68.28% on Machine-B but only by 38.90% on Machine-D.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 5.2 In-Depth Analysis for UVM and PM
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In addition, using PM helps reduce CPU-GPU data transfer time by 29.75% for inference and 49.14% for training.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 4.3 RealWorld Deep Learning Applications
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“On average, for the ten workloads tested, Async Memcpy results in only a 0.35% slowdown and a 0.43% speedup compared to the baseline configuration for the Mega input size on Machine-C and Machine-D, respectively.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > Async Memcpy:
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“while Async Memcpy shows a 9.13% improvement over the baseline (compared to a 4.84% improvement with only 2 KB of shared memory), the total performance remains lower than that achieved with a smaller shared memory allocation.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 7.2 L1-Cache/Shared Memory Partition
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“for the 3DCONV workload where performance is negatively impacted by Async Memcpy, the stall cycles due to pipeline busy increase by 2.69% relative to the baseline, ultimately slowing overall performance.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > Global Memory Throughput and Pipeline Stall Cycles.
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These workloads show significantly higher FLOPs per shared memory access and higher shared memory access per GPU cycle compared to others”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 5.1.1 FLOPs Per Shared Memory Access.
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“BN has high FLOPs per shared memory access but low shared memory utilization, so cannot benefit from using Async Memcpy”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 5.1.1 FLOPs Per Shared Memory Access.
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In addition, Async Memcpy also requires additional control instructions, making the performance even worse (on Machine-B and Machine-C).”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > In addition, Async Memcpy also requires additional control instructions, making the performance even worse (on Machine-B and Machine-C).
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“To ensure sufficient execution time to amortize system overhead, the program input size must be large enough.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 3.3 Benchmark and Machine Configurations
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The overall system throughput can only be improved with an acceptable number of pipeline bubbles.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 2.3 PM/UVM vs Async Memcpy
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Generally, PM can reduce CPU-GPU data transfer time, whereas UVM may not offer the same advantage.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > Takeaway 2:
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Benchmark performance highly depends on the chosen machine and workload configurations.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 3.3 Benchmark and Machine Configurations
Strength:boosted· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“These architectural features work together to establish a three-stage data transfer pipeline in CPU-GPU heterogeneous systems: (1) from CPU DRAM to GPU global memory (U 1 ), (2) from GPU global memory to shared memory (A 2.1 ), and (3) from shared memory to each thread (A 2.2 ).”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 2.3 PM/UVM vs Async Memcpy
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
In the paper
“This feature enables data to be loaded directly from global memory into the shared memory of streaming multiprocessors (SMs).”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 1 Introduction
What the sources say
- Not matchedcites [36]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
- Not matched
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“As shown in Figure 1, Async Memcpy can be used in conjunction with either PM or UVM.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 2.3 PM/UVM vs Async Memcpy
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
None of the sources cited for this claim could be matched to a paper in Crossref, so there is nothing to check it against.
In the paper
“While AMD's Infinity Fabric technology remains unchanged from Rome to Milan [29], Genoa leverages Infinity Fabric 3.0 with GMI3 interface support for more stable and predictable cross-chiplet data transfers [10].”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 3.3 Benchmark and Machine Configurations
What the sources say
- Not matchedcites [29]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites [10]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
- Not matched
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“As its name implies, Async Memcpy operates in the background, allowing the SM to execute computation tasks concurrently.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 1 Introduction
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“UVM is designed to provide a simplified programming interface for automating the CPU-GPU data transfer.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 4.1 Microbenchmarks
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“PM makes the operating system ensure that the corresponding physical pages remain in the main memory rather than being paged to storage devices.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 4.1 Microbenchmarks
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Fortunately, starting with Ampere architecture (with CUDA 11), Nvidia GPUs support this Async Memcpy of data from global memory to shared memory.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 2.2 Asynchronous Memcpy
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The performance of Async Memcpy is more sensitive to the type of workload, e.g., benefits the performance of lud but not knn.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > Takeaway 2:
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
In the paper
“For example, Svedin et al. [51] compared A100 performance with four previous generations of GPUs, and in their experiments on A100, they observed up to 1.25× performance improvement from Async Memcpy.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 2.2 Asynchronous Memcpy
What the sources say
- Not matchedcites [51]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
- Not matched
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Moreover, no previous work has investigated the intersection between PM, UVM, and Async Memcpy.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 1 Introduction
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
None of the sources cited for this claim could be matched to a paper in Crossref, so there is nothing to check it against.
In the paper
“While there is a wealth of prior studies of PM [14,30,41], UVM [5,6,22,59], and Async Memcpy [51] separately, the performance implications of combined effects of these mechanisms for a given workload remain unclear.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 1 Introduction
What the sources say
- Not matchedcites [14]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites [30]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites [41]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites [5]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites [6]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites [22]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites [59]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites [51]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
- Not matched
None of the sources cited for this claim could be matched to a paper in Crossref, so there is nothing to check it against.
In the paper
“In UVM, page faults can happen on the GPU side when the accessed data is not in the page table, which blocks the data transfer and downgrades the overall system performance [5,6].”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 1 Introduction
What the sources say
- Not matchedcites [5]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites [6]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- Not matched
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We ran each workload 30 times and plotted the distribution of each run.8”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 3.3 Benchmark and Machine Configurations
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Async Memcpy allows the programmer to initiate a transfer of data from global to shared memory, without blocking GPU thread execution (code snippets shown in Figure 2(b)).”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 2.2 Asynchronous Memcpy
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
In the paper
“While large inputs are commonly used in benchmarks to reduce measurement noise, prior work [26] shows that bigger inputs do not always yield stable performance in CPU-GPU heterogeneous systems.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 3.3 Benchmark and Machine Configurations
What the sources say
- Not matchedcites [26]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
- Not matched
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“All pipelines come with overhead.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 2.3 PM/UVM vs Async Memcpy
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In summary, while UVM and PM generally have little effect on the instruction mix, Async Memcpy does alter it, potentially impacting overall program performance.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 5.1.2 Other Kernel Performance Counters.
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This is because lud has a higher computation density than knn, providing more opportunity for an additional data transfer pipeline stage.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > Async Memcpy:
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“PM improves the system performance by avoiding swapping out OS papers with high locality.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 8.1 Limitations of PM, UVM, and Async Memcpy
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“To benefit from Async Memcpy, a program must achieve high FLOPs to shared memory access ratio and sustain substantial shared memory utilization.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > Takeaway 5:
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Enabling the prefetcher (with UVM) can incur additional shared memory traffic, making potential performance degradation when using Async Memcpy.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > Async Memcpy+UVM/PM:
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, it will only improve the overall performance if: (a) the program has intensive shared memory usage; and (b) the shared memory maintains high locality to amortize the overhead of data transfer and additional control logic.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 5.1.1 FLOPs Per Shared Memory Access.
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
In the paper
“In this case, the data allocation for the second job (kernel) can occur while the first job (kernel) is being processed, which could be leveraged in Kernel-as-a-Service (Kaas) systems [42].”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 8.2 Directions for New Data Transfer Models
What the sources say
- Not matchedcites [42]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
- Not matched
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The ratio of FLOPs to shared memory accesses and shared memory utilization are the two metrics that determine whether a program can benefit from Async Memcpy.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 5.1.2 Other Kernel Performance Counters.
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, when using UVM, especially with prefetchers, programmers should be cautious, as prefetching can result in increased runtime data transfers between shared and global memory.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > Async Memcpy:
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Once the GPU kernel for job 1 begins, job 2 can initiate data allocation (cudaMallocManaдed()) using the idle CPU.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 8.2 Directions for New Data Transfer Models
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The performance improvement of Async Memcpy comes from improved global memory throughput and reduced stall cycles due to memory dependency and pipeline busy.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > Takeaway 6:
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes? Based: unclearIt may state two separate claims as one.
None of the sources cited for this claim could be matched to a paper in Crossref, so there is nothing to check it against.
In the paper
“An enhanced UVM version supports prefetch data from global memory to L2 cache[5,22], reducing the global memory to shared memory time.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 9 Conclusion
What the sources say
- Not matchedcites [5]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites [22]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- Not matched
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We verified our findings under both specified and unspecified NUMA node configurations.In each case, results remained consistent because we focus on a single-GPU scenario, and all program memory footprints lie well within the capacity of a single NUMA node.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 9 Conclusion
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, if an application frequently allocates small objects, the overhead of PM allocation may outweigh its benefits.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > Async Memcpy:
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“After the GPU kernel of job 1 completes, job 2 can start its own kernel execution, while job 1 handles data deallocation (cudaFree()) on the CPU.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 8.2 Directions for New Data Transfer Models
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Async Memcpy pipelines the computation and global memory to shared memory data transfer, so uvm_async can save GPU kernel time compared with uvm.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > Async Memcpy+UVM/PM:
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our performance analysis indicates that the benefits of UVM are highly dependent on machine system configurations, which are best assessed through runtime profiling rather than static analysis.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > Async Memcpy:
Strength:hedged· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“By integrating UVM and Async Memcpy, data transfers from CPU memory to GPU global memory, and from global memory to shared memory, can overlap with GPU kernel execution”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 8.2 Directions for New Data Transfer Models
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Frequently allocating/deallocating PM may introduce additional overhead when using PM for DL applications.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > Takeaway 4:
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear? Based: unclearBorderline substantial claim. It is unclear whether the paper’s conclusions depend on it. It is unclear why the paper says it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Ideally, the 37.66% of time spent on data allocation (and deallocation) could overlap with the 37.79% of GPU kernel time, potentially resulting in more than 30% overall performance improvement.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 8.2 Directions for New Data Transfer Models
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Because CPU-GPU data transfers for lud are negligible, the dominant time instead comes from moving data between GPU global memory and shared memory.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > In addition, Async Memcpy also requires additional control instructions, making the performance even worse (on Machine-B and Machine-C).
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The overhead of Async Memcpy comes from an increased instruction footprint.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > Takeaway 6:
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“UVM and PM are affected by both types of machine configurations and workload, while Async Memcpy is mostly affected by workload characteristics.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > Takeaway 3:
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“While increasing shared memory can boost the effectiveness of Async Memcpy, it remains essential to reserve enough memory for the L1 cache.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > Takeaway 9:
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The performance of UVM, PM, and Async Memcpy is not sensitive to the number of CUDA blocks, but very sensitive to the number of threads per block.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > Takeaway 8:
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“because Async Memcpy and UVM do not enhance CPU task performance, overall system time becomes constrained by data allocation operations”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 8.1 Limitations of PM, UVM, and Async Memcpy
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Introducing additional stages to the data transfer pipeline adds extra control logic to the GPU kernels, potentially reducing efficiency.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 4.3 RealWorld Deep Learning Applications
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“We also compared the performance of our own implementation with cutlass[33] to guarantee the efficacy of our kernel implementations.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 9 Conclusion
What the sources say
- Not matchedcites [33]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
- Not matched
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“with the same per-block shared memory capacity, fewer threads in one block can lead to a deeper buffer for each thread and larger FLOPs per shared memory access”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 7.1 CUDA Block and Thread
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
None of the sources cited for this claim could be matched to a paper in Crossref, so there is nothing to check it against.
In the paper
“Consequently, the primary overhead stems from PM allocation [1,7,45], which distinguishes these applications from microbenchmarks and other non-DL real-world applications.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 4.3 RealWorld Deep Learning Applications
What the sources say
- Not matchedcites [1]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites [7]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites [45]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- Not matched
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“data allocation typically needs to be completed before data transfer and kernel execution for each workload, making it impractical to overlap these tasks in most scenarios.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 8.2 Directions for New Data Transfer Models
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“current data transfer pipelines still have limitations.”
§ Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems > 8 A New Data Transfer Model
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
The paper relies on this without support.
DL frameworks typically allocate PM on a per-tensor, per-layer basis, resulting in a large number of allocation function calls.
“The reason is that DL frameworks typically allocate PM on a per-tensor, per-layer basis, resulting in a large number of allocation function calls.”
The paper relies on this without support.
During the training phase, convolutional deep neural networks depend heavily on gemm kernels.
“During the training phase, convolutional deep neural networks depend heavily on gemm (general matrix-to-matrix multiplication) kernels.”
The paper relies on this without support.
PM and UVM cannot be used together.
“However, PM and UVM cannot be used together.”
The paper relies on this without support.
Async Memcpy can be used together with either PM or UVM.
“Async Memcpy can be used together with either PM or UVM.”
The authors state this as an assumption.
Unless otherwise specified, UVM is used with a prefetcher in this article.
“If not specified, we assume UVM is used with prefetcher in this article.”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Baseline execution-time shares exceed the total
MajorThe stated baseline transfer and GPU-kernel shares total 99.80% of execution time. The separately stated baseline allocation share of 18.99% cannot also be part of that same total.
“the CPU-GPU data transfer time decreases from 57.96% to 24.90% and 25.09% of the overall execution time, when compared baseline with UVM and PM.Because data transfers now take less time, a larger proportion of the execution is devoted to GPU kernel processing (from 41.84% to 59.25% and 63.80% for UVM and PM, respectively).”
“While data allocation accounts for only 18.99% of total execution time under the baseline configuration”
Two different Mega-size UVM improvements on Machine-C
MinorFor the same Machine-C comparison of UVM against baseline, the Mega-size improvement is reported as both 50.53% and 51.53%.
“there is 48.13% and 50.53% performance improvement over baseline on Machine-C for Super and Mega input sizes, respectively.”
“There is a 48.13% and 51.53% performance improvement on Machine-C compared with using UVM with baseline for Super and Mega input sizes, respectively.”
PM-only result is attributed to async-only
MinorThe 56.63% Mega-size improvement is previously identified as the PM-only result. The async-only configuration is instead reported as a 1.72% slowdown on Machine-C.
“the performance improvement of pm_async is also slightly worse than async only (55.28% versus 56.63% for the Mega input size on Machine-C).”
“When using PM, there is 57.82% and 56.63% performance improvement over baseline on Machine-C for Super and Mega input sizes, respectively.”
The stated large matrix input size differs from Table 3
MinorThe prototype calls a 4K×4K matrix the large input size used in Figure 3, but Table 3 assigns 8K * 8K to the Large 2D-grid category; 4K * 4K is its Medium category.
“each with a 4K×4K (which is the large input size used in Figure 3) input size”
“| 2D Grid 512 * 512 | 1K * 1K | 4K * 4K | 8K * 8K | 32K * 32K | 64K * 64K | |”
Citations
Cites
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602
Rodinia: A benchmark suite for heterogeneous computing
Che, S., Boyer, M. J., Meng, J., Tarjan, D., Sheaffer, J. W., Lee, S.-H., & Kevin Skadron, Y. (2009) · IEEE International Symposium on Workload Characterization · doi:10.1109/iiswc.2009.5306797 · Cited by 3,100
Llama 2: Open Foundation and Fine-Tuned Chat Models
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.09288 · Cited by 2,604
Efficient Memory Management for Large Language Model Serving with PagedAttention
Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C. H., Gonzalez, J., Zhang, H., & Stoica, I. (2023) · Proceedings of the 29th Symposium on Operating Systems Principles · doi:10.1145/3600006.3613165 · Cited by 1,817
Efficient Memory Management for Large Language Model Serving with PagedAttention
Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C. H., Gonzalez, J. E., Zhang, H., & Stoica, I. (2023) · arXiv · doi:10.48550/arxiv.2309.06180 · Cited by 1,802
Ansel, J., Yang, E., He, H., Gimelshein, N., Jain, A., Voznesensky, M., Bao, B., Bell, P., Berard, D., Burovski, E., et al. (2024) · Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2 · doi:10.1145/3620665.3640366 · Cited by 737
Grattafiori, A., Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Vaughan, A., et al. (2024) · arXiv · doi:10.48550/arxiv.2407.21783 · Cited by 505
NVIDIA A100 Tensor Core GPU: Performance and Innovation
Choquette, J., Gandhi, W., Giroux, O., Stam, N., & Krashinsky, R. (2021) · IEEE Micro · doi:10.1109/mm.2021.3061394 · Cited by 410
Where is the data? Why you cannot debate CPU vs. GPU performance without the answer
Gregg, C., & Hazelwood, K. (2011) · IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS) · doi:10.1109/ispass.2011.5762730 · Cited by 292
(2021) · doi:10.1145/3458817.3476205 · Cited by 257
Superneurons: dynamic GPU memory management for training deep neural networks
Wang, L., Ye, J., Zhao, Y., Wu, W., Li, A., Song, S. L., Xu, Z., & Kraska, T. (2018) · Proceedings of the 23rd ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming · doi:10.1145/3178487.3178491 · Cited by 175
Orion: Interference-aware, Fine-grained GPU Sharing for ML Applications
Strati, F., Ma, X., & Klimovic, A. (2024) · European Conference on Computer Systems · doi:10.1145/3627703.3629578 · Cited by 131
Towards high performance paged memory for GPUs
Zheng, T., Nellans, D. W., Zulfiqar, A., Stephenson, M., & Keckler, S. W. (2016) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca.2016.7446077 · Cited by 129
Lutz, C., Breß, S., Zeuch, S., Rabl, T., & Markl, V. (2020) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/3318464.3389705 · Cited by 118
NVIDIA A100 GPU: Performance & Innovation for GPU Computing
Choquette, J., & Gandhi, W. (2020) · IEEE Hot Chips Symposium · doi:10.1109/hcs49909.2020.9220622 · Cited by 103
Wait of a Decade: Did SPEC CPU 2017 Broaden the Performance Horizon?
Panda, R., Song, S., Dean, J., & John, L. K. (2018) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca.2018.00032 · Cited by 97
Self-citationBatch-Aware Unified Memory Management in GPUs for Irregular Workloads
Kim, H., Sim, J., Gera, P., Hadidi, R., & Kim, H. (2020) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/3373376.3378529 · Cited by 91
Sabet, A. H. N., Zhao, Z., & Gupta, R. (2020) · European Conference on Computer Systems · doi:10.1145/3342195.3387537 · Cited by 85
In-depth analyses of unified virtual memory system for GPU accelerated computing
Allen, T., & Ge, R. (2021) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1145/3458817.3480855 · Cited by 80
Proceedings of the Fifteenth European Conference on Computer Systems
(2020) · doi:10.1145/3342195 · Cited by 64
Hetherington, T., O’Connor, M., & Aamodt, T. M. (2015) · ACM Symposium on Cloud Computing (SoCC) · doi:10.1145/2806777.2806836 · Cited by 56
Demystifying GPU UVM Cost with Deep Runtime and Workload Analysis
Allen, T., & Ge, R. (2021) · IEEE International Parallel and Distributed Processing Symposium (IPDPS) · doi:10.1109/ipdps49936.2021.00023 · Cited by 52
Proceedings of the Sixth ACM Symposium on Cloud Computing
(2015) · doi:10.1145/2806777 · Cited by 41
Wang, L., Ye, J., Zhao, Y., Wu, W., Li, A., Song, S. L., Xu, Z., & Kraska, T. (2018) · ACM SIGPLAN Notices · doi:10.1145/3200691.3178491 · Cited by 41
AMD Next-Generation “Zen 4” Core and 4th Gen AMD EPYC Server CPUs
Bhargava, R., & Troester, K. (2024) · IEEE Micro · doi:10.1109/mm.2024.3375070 · Cited by 37
Evaluating Characteristics of CUDA Communication Primitives on High-Bandwidth Interconnects
Pearson, C., Dakkak, A., Hashash, S., Li, C., Chung, I., Xiong, J., & Hwu, W. (2019) · ACM/SPEC International Conference on Performance Engineering (ICPE) · doi:10.1145/3297663.3310299 · Cited by 36
Proceedings of the Nineteenth European Conference on Computer Systems
(2024) · doi:10.1145/3627703 · Cited by 35
Evaluating the Performance of NVIDIA’s A100 Ampere GPU for Sparse and Batched Computations
Anzt, H., Tsai, Y., Abdelfattah, A., Cojean, T., & Dongarra, J. J. (2020) · IEEE/ACM International Workshop on Performance Modeling, Benchmarking and Simulation of High Performance Computer Systems (PMBS) · doi:10.1109/pmbs51919.2020.00009 · Cited by 32
UVMBench: A Comprehensive Benchmark Suite for Researching Unified Virtual Memory in GPUsPreprint
Gu, Y., Wu, W., Li, Y., & Chen, L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.09822 · Cited by 29
Oversubscribing GPU Unified Virtual Memory
Shao, C., Guo, J., Wang, P., Wang, J., Li, C., & Guo, M. (2022) · ACM/SPEC International Conference on Performance Engineering (ICPE) · doi:10.1145/3489525.3511691 · Cited by 29
Analyzing memory management methods on integrated CPU-GPU systems
Dashti, M., & Fedorova, A. (2017) · ACM SIGPLAN International Symposium on Memory Management (ISMM) · doi:10.1145/3092255.3092256 · Cited by 28
SnakeByte: A TLB Design with Adaptive and Recursive Page Merging in GPUs
Lee, J., Lee, J. M., Oh, Y., Song, W. J., & Ro, W. W. (2023) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca56546.2023.10071063 · Cited by 24
Rome to Milan, AMD Continues Its Tour of Italy
Mattioli, M. (2021) · IEEE Micro · doi:10.1109/mm.2021.3086541 · Cited by 20
DrGPU: A Top-Down Profiler for GPU Applications
Hao, Y., Jain, N., Wijngaart, R. F. V. D., Saxena, N. R., Fan, Y., & Liu, X. (2023) · ACM/SPEC International Conference on Performance Engineering (ICPE) · doi:10.1145/3578244.3583736 · Cited by 20
Benchmarking the Nvidia GPU Lineage
Svedin, M., Chien, S. W. D., Chikafa, G., et al. (2021) · International Symposium on Highly Efficient Accelerators and Reconfigurable Technologies (HEART) · doi:10.1145/3468044.3468053 · Cited by 20
Increasing GPU Translation Reach by Leveraging Under-Utilized On-Chip Resources
Kotra, J., LeBeane, M., Kandemir, M., & Loh, G. H. (2021) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1145/3466752.3480105 · Cited by 18
Performance analysis of memory transfers and GEMM subroutines on NVIDIA Tesla GPU cluster
Allada, V., Benjegerdes, T., & Bode, B. (2009) · IEEE International Conference on Cluster Computing (CLUSTER) · doi:10.1109/clustr.2009.5289124 · Cited by 16
Kernel-as-a-Service: A Serverless Interface to GPUsPreprint
Pemberton, N., Zabreyko, A. A., Ding, Z., Katz, R. H., & Gonzalez, J. E. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2212.08146 · Cited by 15
Analyzing Performance and Power-Efficiency Variations among NVIDIA GPUs
Yoshida, K., Sageyama, R., Miwa, S., Yamaki, H., & Honda, H. (2022) · Proceedings of the International Conference on Parallel Processing · doi:10.1145/3545008.3545084 · Cited by 15
Fine-grain Quantitative Analysis of Demand Paging in Unified Virtual Memory
Allen, T., Cooper, B., & Ge, R. (2023) · ACM Transactions on Architecture and Code Optimization · doi:10.1145/3632953 · Cited by 15
GPGPU Benchmark Suites: How Well Do They Sample the Performance Spectrum?
Ryoo, J. H., Quirem, S., LeBeane, M., Panda, R., Song, S., & John, L. K. (2015) · Proceedings of the International Conference on Parallel Processing · doi:10.1109/icpp.2015.41 · Cited by 9
Self-citationPerformance Implications of Async Memcpy and UVM: A Tale of Two Data Transfer Modes
Li, R., Yadav, S., Wu, Q., Kavi, K. M., Mehta, G., Yadwadkar, N. J., & John, L. K. (2023) · IEEE International Symposium on Workload Characterization · doi:10.1109/iiswc59245.2023.00024 · Cited by 6
Self-citationGermany, H. '. I. S. O. H. E. A. A. R. T. 2. O., & Association for Computing Machinery-Digital Library (2021) · doi:10.1145/3468044 · Cited by 5
UVM Discard: Eliminating Redundant Memory Transfers for Accelerators
Zhu, W., Cox, G., Vesely, J., Hairgrove, M., Cox, A. L., & Rixner, S. (2022) · IEEE International Symposium on Workload Characterization · doi:10.1109/iiswc55918.2022.00013 · Cited by 5
Baca, H. A. H., & Valdivia, F. D. L. P. (2019) · IEEE International Conference on Electronics, Electrical Engineering and Computing (INTERCON) · doi:10.1109/intercon.2019.8853624 · Cited by 3
Analyzing memory management methods on integrated CPU-GPU systems
Dashti, M., & Fedorova, A. (2017) · ACM SIGPLAN Notices · doi:10.1145/3156685.3092256
Flagged first, then by citations
Research works printed but not linked: 9
- Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al. 2016. \(\lbrace\) TensorFlow \(\rbrace\) : A system for \(\lbrace\) Large-Scale \(\rbrace\) machine learning. In Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16). 265–283.
- Amey Agrawal, Nitin Kedia, Ashish Panwar, Jayashree Mohan, Nipun Kwatra, Bhargav Gulavani, Alexey Tumanov, and Ramachandran Ramjee. 2024. Taming \(\lbrace\) throughput-latency \(\rbrace\) tradeoff in \(\lbrace\) LLM \(\rbrace\) inference with \(\lbrace\) sarathi-serve \(\rbrace\) . In Proceedings of the 18th USENIX Symposium on Operating Systems Design and Implementation (OSDI 24). 117–134.
- 10.5555/2015039.2015535 · DOI
- Guyue Huang Yang Bai Liu Liu Yuke Wang Bei Yu Yufei Ding and Yuan Xie. 2023. Alcop: Automatic load-compute pipelining in deep learning compiler for ai-gpus. Proceedings of Machine Learning and Systems 5 (2023) 680–694.
- Zhuohan Li, Lianmin Zheng, Yinmin Zhong, Vincent Liu, Ying Sheng, Xin Jin, Yanping Huang, Zhifeng Chen, Hao Zhang, Joseph E Gonzalez, and Ion Stoica. 2023. \(\lbrace\) AlpaServe \(\rbrace\) : Statistical multiplexing with model parallelism for deep learning serving. In Proceedings of the 17th USENIX Symposium on Operating Systems Design and Implementation (OSDI 23). 663–679.
- Dan Negrut, Radu Serban, Ang Li, and Andrew Seidl. 2014. Unified memory in cuda 6.0. a brief overview of related data access and transfer issues. SBEL, Madison, WI, USA, Technical Reports TR-2014-09 (2014).
- SB Shriram, Anshuj Garg, and Purushottam Kulkarni. 2019. Dynamic memory management for gpu-based training of deep neural networks. In Proceedings of the 2019 IEEE International Parallel and Distributed Processing Symposium (IPDPS). IEEE, 200–209.
- Yuke Wang, Boyuan Feng, Zheng Wang, Tong Geng, Kevin Barker, Ang Li, and Yufei Ding. 2023. MGG: Accelerating graph neural networks with fine-grained intra-kernel communication-computation pipelining on multi-GPU platforms. In Proceedings of the 17th USENIX Symposium on Operating Systems Design and Implementation (OSDI 23). 779–795.
- Gyeong-In Yu, Joo Seong Jeong, Geon-Woo Kim, Soojeong Kim, and Byung-Gon Chun. 2022. Orca: A distributed serving system for \(\lbrace\) transformer-based \(\rbrace\) generative models. In Proceedings of the 16th USENIX Symposium on Operating Systems Design and Implementation (OSDI 22). 521–538.
Cited by
Sorted by citations
Evaluating the benefits of Argument Fusion: Optimizing host-device communication
Sotiropoulos, D.-O., Capotondi, A., & Marongiu, A. (2026) · ACM International Conference on Computing Frontiers: Workshops and Special Sessions (CF Companion) · doi:10.1145/3801488.3806378 · Cited by 2
Popov, A. V., & Pavliuk, A. (2025) · Математичне моделювання · doi:10.31319/2519-8106.2(53)2025.342456
High-Throughput sCMOS Image Real-Time Compression System with FPGA-GPU Heterogeneous Co-processing
Zhang, D., Xiao, Y., Yang, X., & Jiang, T. (2026) · International Conference on Information Science, Electrical and Automation Engineering (ISEAE) · doi:10.1109/iseae69422.2026.11544303
Marongiu, A., Capotondi, A., & Sotiropoulos, D.-O. (2026) · IRIS UNIMORE (University of Modena and Reggio Emilia)
Bose, C. D., Avalos, C., Pan, J., & Rogers, T. (2026) · IEEE International Symposium on Workload Characterization · doi:10.1109/iiswc71020.2026.00039
Metadata
- Authors
- Li, R., Hanindhito, B., Yadav, S., Wu, Q., Kavi, K., Mehta, G., Yadwadkar, N. J., & John, L. K.
- Affiliations
- The University of Texas at Austin (US)
- University of North Texas (US)
- ORCID
- Venue
- ACM Transactions on Architecture and Code Optimization
- Published
- 2025-09-19
- Volume
- 22
- Issue
- 3
- Pages
- 1-26
- ISSN
- 1544-3566 (ISSN-L)
- 1544-3973
- OpenAlex
- W4411670522
- Cited by
- 5
- References
- 30
- Citations by year
- 2026: 4 · 2025: 1
- FWCI
- 2.49
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by
- Locations
- ACM Transactions on Architecture and Code Optimization (publishedVersion, cc-by)
- Funders
- NSF (#2326894 and #2425655)
- Topics
- Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Computer Graphics and Visualization Techniques
- Physical Sciences › Computer Science › Computer Networks and Communications › Distributed and Parallel Computing Systems
- Keywords
- CPU-GPU data transfer, pipelining, heterogeneous computing, memory management, performance counters, memory allocation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-11-08, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-08 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
