The DOI matches its Crossref record; 50 of the 50 references it cites were found in a registry or an index.
SwapAdvisor: Pushing Deep Learning Beyond the GPU Memory Limit via Smart Swapping
Open accessAbstract
It is known that deeper and wider neural networks can achieve better accuracy. But it is difficult to continue the trend to increase model size due to limited GPU memory. One promising solution is to support swapping between GPU and CPU memory. However, existing work on swapping only handle certain models and do not achieve satisfactory performance. Deep learning computation is commonly expressed as a dataflow graph which can be analyzed to improve swapping. We propose SwapAdvisor, which performs joint optimization along 3 dimensions based on a given dataflow graph: operator scheduling, memory allocation, and swap decisions. SwapAdvisor explores the vast search space using a custom-designed genetic algorithm. Evaluations using a variety of large models show that SwapAdvisor can train models up to 12 times the GPU memory limit while achieving 53-99% of the throughput of a hypothetical baseline with infinite GPU memory.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Adaptation in Natural and Artificial Systems
Holland, J. H. (1992) · The MIT Press eBooks · doi:10.7551/mitpress/1090.001.0001 · Cited by 47,340
Bidirectional recurrent neural networks
Schuster, M., & Paliwal, K. K. (1997) · IEEE Transactions on Signal Processing · doi:10.1109/78.650093 · Cited by 10,172
Handbook of Genetic Algorithms
Davis, L. D. (1991) · Cumulative Index of Computer Aided Architectural Design · Cited by 7,330
Proceedings of the 25th international conference on Machine learning
Cohen, W. W., McCallum, A., & Roweis, S. T. (2008) · Cited by 5,691
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
Wu, Y., Schuster, M., Chen, Z., Le, Q. V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., et al. (2016) · arXiv · doi:10.48550/arxiv.1609.08144 · Cited by 5,319 · cited 1×
“The deep learning community has been using larger deep neural network (DNN) models to achieve higher accuracy on more complex tasks over the past few years [16, 47, 55, 58, 60].”
MethodHan, S., Mao, H., & Dally, W. J. (2015) · arXiv · doi:10.48550/arxiv.1510.00149 · Cited by 4,662
Large Scale Distributed Deep Networks
Dean, J. B., Corrado, G. S., Monga, R., Chen, K., Devin, M., Mao, M., Ranzato, M., Senior, A., Tucker, P. A., Yang, K., et al. (2012) · Cited by 3,202
Research works printed but not linked: 16
- Cui Henggang, Proceedings of the 2014 USENIX Conference on USENIX Annual Technical Conference (USENIX ATC'14). USENIX Association, 37--48
- 10.5555/3157382.3157559 · DOI
- Han Song, Proceedings of the 28th International Conference on Neural Information Processing Systems -, 1
- Holland John Henry, 1992, mbox
- 10.5555/3157382.3157557 · DOI
- Jeong Eunji, 2019, 16th USENIX Symposium on Networked Systems Design and Implementation (NSDI 19)
- Man Kim-Fung, 2001, Kit Sang Tang, and Sam Kwong
- Meng Chen, 2017, Proc. of ML Systems Workshop in NIPS .
- Mirhoseini Azalia, 2017, Proceedings of the 34th International Conference on Machine Learning -, 70
- Rhu Minsoo, 2016, Keckler
- Singh Harmel, 1996, 5th IEEE Heterogeneous Computing Workshop (HCW'96)
- 10.5555/523549.822912 · DOI
- Wei Jinliang, Proceedings of the Sixth ACM Symposium on Cloud Computing (SoCC'15)
- Wiggers Maarten H., Proceedings of the 44th Annual Design Automation Conference (DAC'07)
- Yu Yuan, Proceedings of the Thirteenth EuroSys Conference (EuroSys'18)
- Zoph Barret, 2018, Learning Transferable Architectures for Scalable Image Recognition. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 8697--8710
Cited by
Sorted by citations
Items 101–159 of 159
Luo, Y., Lai, J., Shi, S., Chen, C., Qi, S., Zhang, J., & Wang, Q. (2025) · IEEE Transactions on Cloud Computing · doi:10.1109/tcc.2025.3624752 · Cited by 3
“…Accumulation: Given that GPU memory constraints may limit the per-GPU batch size, some schedulers tackle this limitation through memory offloading [43] (which may introduce additional system overhead) or by adjusting batch sizes and other training hyper-parameters [18] (which may compromise model…”
Efficient KV Cache Spillover Management on Memory-Constrained GPU for LLM Inference
Jiang, J., Chen, Y., Zhang, Z., He, B., Luo, P., Lu, M., Chen, Y., Zhang, H., Du, J., Huang, D., et al. (2025) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2025.3626974 · Cited by 2
“SwapAd-visor [18] employs a genetic algorithm to concurrently seek optimal solutions for operator scheduling, memory allocation, and swap decisions.”
H2O: Heterogeneity-Aware Hierarchical Orchestration for Memory-Efficient On-Device LLM Inference
Zeng, F., Lyu, F., Wu, H., Li, Z., Li, S., & Xu, F. (2025) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2025.3628498 · Cited by 2
“Early works [29], [30], [31] relied on manual partitioning and parameter reordering, while [52] applied transparent computing [53], [54], [55], [56] to enable on-demand loading.”
Towards Efficient LLM Inference via Collective and Adaptive Speculative Decoding
Wang, S., Yang, H., Wang, X., Liu, T., Wang, P., Xu, Y., Liang, X., Ma, K., Feng, T., You, X., et al. (2025) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1145/3712285.3759834 · Cited by 2
“On one hand, this leads to frequent triggering of swapping [24, 47] or recomputation [31] to ensure memory safety, resulting in severe performance degradation.”
A Survey on Accelerated Technologies for Mixture-of-Experts Model Training Systems
Zhang, Q., Zhai, J., & Zheng, W. (2025) · Tsinghua Science & Technology · doi:10.26599/tst.2025.9010169 · Cited by 2
Oneiros: KV Cache Optimization through Parameter Remapping for Multi-tenant LLM Serving
Li, R., Pal, S., Pullu, V. N., Sinha, P., Ryoo, J., John, L. K., & Yadwadkar, N. J. (2025) · ACM Symposium on Cloud Computing (SoCC) · doi:10.1145/3772052.3772215 · Cited by 5
“Using CPU memory as an extension of the GPU memory (also called CPU-offloading) is one way to ease the pressure on GPU memory [21, 22, 35, 38, 48, 53, 55, 58, 73].”
ARIADNE: Adaptive UVM Management for Efficient GPU Memory Oversubscription
Shin, H., Bang, S., Park, H., & Kim, D. (2026) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca68181.2026.11408564 · Cited by 2
Parallel intelligent computing: development and challenges
卢, 凯., 赖, 志., 李, 笙., 柳, 炜., 葛, 可., 卢, 锡., & 李, 东. (2023) · Scientia Sinica Informationis · doi:10.1360/ssi-2023-0051 · Cited by 2
DeepTM: Efficient Tensor Management in Heterogeneous Memory for DNN Training
Zhou, H., Rang, W., Chen, H., Zhou, X., & Cheng, D. (2024) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2024.3431910 · Cited by 2
“2) Analysis of Application Characteristics: Existing approaches tend to consider only a limited set of characteristics when making the replacement for hot and cold data in heterogeneous memory systems [18], [29].”
BackgroundInfluentialEfficient Schedule Construction for Distributed Execution of Large DNN Models
Lin, Z., Miao, Y., Xu, G., Li, C., Saarikivi, O., Maleki, S., & Yang, F. (2024) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2024.3466913 · Cited by 2
Embracing Imbalance: Dynamic Load Shifting among Microservice Containers in Shared Clusters
Luo, S., Liao, J., Lin, C., Xu, H., Zhou, Z., & Xu, C. (2025) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/3676641.3716255 · Cited by 2
“To address this challenge, heuristic solutions, such as genetic algorithms [20], iteratively solve the optimization problem (1) based on feed-back generated from trials without relying on pre-defined latency formulas.”
An Application-oblivious Memory Scheduling System for DNN Accelerators
Li, J., Wang, X., Chen, X., Li, G., Dong, X., Zhao, P., Yu, X., Yang, Y., Cao, W., Liu, L., et al. (2022) · ACM Transactions on Architecture and Code Optimization · doi:10.1145/3535355 · Cited by 3
“Another promising approach aiming to reduce the memory footprint of DNN workloads is to virtualize the memory usage of DNNs across the host and device memory [3, 9, 15, 24, 34, 50, 55, 56, 65, 69, 70].”
BackgroundMethodInfluentialSelf-citationPinpointing the Memory Behaviors of DNN Training
Li, J., Dong, X., Li, G., Zhao, P., Wang, X., Chen, X., Yu, X., Yang, Y., Jiang, Z., Cao, W., et al. (2021) · IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS) · doi:10.1109/ispass51385.2021.00038 · Cited by 2
“Suppose we want to reduce the device memory pressure by swapping data back and forth between the host CPUs and the device GPUs [9], [10], the ATI of current device memory block is T .”
Self-citationLow Carbon Footprint Training for 1D-CNNs with Temporal Max-Pooling
Raju, A. D., & Wang, K. (2024) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3627673.3679678 · Cited by 1
“Offloading or out-of-core learning methods [5, 27, 56] orchestrate an idle layer’s full set of activations/gradients between GPU and RAM.”
EDA: Energy-Efficient Inter-Layer Model Compilation for Edge DNN Inference Acceleration
Pao, B.-R., Chen, I.-C., Chang, E., & Yeh, T.-T. (2025) · International Symposium on High-Performance Computer Architecture · doi:10.1109/hpca61900.2025.00050 · Cited by 8
THDP: Temporal History Based Dynamic Prefetching for GPU Memory Oversubscription
Wang, M., Xu, G., Ding, M., & Wang, Z. (2025) · IEEE International Conference on Parallel and Distributed Systems (ICPADS) · doi:10.1109/icpads67057.2025.11323104 · Cited by 1
“Representative works include Capuchin [5] and SwapAdvisor [9].”
Zeng, Y., Jiang, C., Huang, C., Zhu, Y., Lu, X., Yuan, J., & Zhang, J. (2026) · The Computer Journal · doi:10.1093/comjnl/bxag014 · Cited by 1
Self-citationCLM: Removing the GPU Memory Barrier for 3D Gaussian Splatting
Zhao, H.-X., Min, X., Liu, X., Gong, M., Li, Y., Li, A., Xie, S.-N., Li, J., & Panda, A. (2025) · International Conference on Architectural Support for Programming Languages and Operating Systems · doi:10.1145/3779212.3790140 · Cited by 7
“For example, [21, 41, 42] focus on DNN training in general, while [4, 37] specialize in recommendation systems, [2, 46] in GNNs, and [48] in LLMs, all of which optimize for workloads distinct from 3DGS.”
Self-citationMemory-Optimized Offloading: Enabling LLM Fine-Tuning on Memory-Constrained Hardware
Lin, K., Du, H.-C., Wu, D.-W., Lin, Y., Luo, T., Li, Q., & Xue, C. J. (2025) · IEEE Non-Volatile Memory System and Applications Symposium · doi:10.1109/nvmsa66678.2025.00014 · Cited by 1
An Empirical Study of Memory Pool Based Allocation and Reuse in CUDA Graph
Qian, R., Gao, M., Shi, Q., & Xu, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-0808-6_23 · Cited by 1
Deep Reorganization: Retaining Residuals in TinyML
Mendis, H. R., Kang, C.-K., Lin, C., Chen⋆, M.-S., & Hsiu, P.-C. (2024) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1145/3649329.3656219 · Cited by 1
Adaptive Optimisation of PyTorch Memory Pools for DNNs
Li, L., Luo, J., Dong, P., Fang, X., Yu, A., & Jiang, Z. (2024) · Lecture notes in computer science · doi:10.1007/978-981-96-0116-5_37 · Cited by 1
OWL: Worker-assisted server bandwidth optimization for efficient communication federated learning
Han, X., Liu, B., Hu, C., & Cheng, D. (2024) · Journal of Parallel and Distributed Computing · doi:10.1016/j.jpdc.2024.105027 · Cited by 1
Memory-Efficient Batching for Time Series Transformer Training: A Systematic Evaluation
Sinthong, P., Nguyen, N. H., Ekambaram, V., Jati, A., Kalagnanam, J., & Koad, P. (2025) · Algorithms · doi:10.3390/a18060350 · Cited by 5
DNN Training Optimization with Pipelined Parallel Based on Feature Maps Encoding
Shen, L., Mao, Y., Wang, Z., Nie, H., & Huang, J. (2022) · International Conference on Advanced Cloud and Big Data · doi:10.1109/cbd58033.2022.00016 · Cited by 1
“The main memory consumption of each GPU (or stage) is for storing the generated activations in the forward and backward propagation [4].”
BackgroundSrinuan, P., Sigdel, P., Yuan, X., Peng, L., Darby, P. J., Aucoin, C., & Tzeng, N.-F. (2021) · IEEE International Conference on Networking, Architecture and Storage (NAS) · doi:10.1109/nas51552.2021.9605372
“Recent studies on swapping GPU pages to host machine memory smartly [6] and on tensor eviction/prefetching and recomputation [7] during deep ML training aim to let GPUs handle enlarged memory footprints, but they target solely at workloads with regular computation structures or predictable data access patterns known p…”
BackgroundMethodAutoConstruct: Automated Neural Surrogate Model Building and Deployment for HPC Applications
Dong, W., & Ren, J. (2023) · doi:10.1145/3589013.3596677
“Such a performance analysis techniques [28, 65, 68, 88] and model performance prediction techniques [57] are widely used in existing system optimization for ML.”
MethodOLM2: Automatic Optimal Strategy Generating for Large-Scale Model Training with Limited-Memory
Yang, Z., Tang, Y. H., Qiao, L., Yang, X., & Huang, Z. (2023) · IEEE International Conference on Joint Cloud Computing (JCC) · doi:10.1109/jcc59055.2023.00006
Multi-level Storage Optimization for Intermediate Data in AI Model Training
Fu, J., Yang, Y., Hu, G., Luo, X., & Shao, J. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-47843-7_11
Kesavan, S. P., Bhatia, H., Dasu, K., Pearce, O., & Ma, K. (2024) · IEEE Pacific Visualization Conference (PacificVis) · doi:10.1109/pacificvis60374.2024.00013
BOOM: Use your Desktop to Accurately Predict the Performance of Large Deep Neural Networks
Su, Q., Yang, J., & Pekhimenko, G. G. (2024) · International Conference on Parallel Architectures and Compilation Techniques (PACT) · doi:10.1145/3656019.3676950
Libra: A Python-Level Tensor Re-Materialization Strategy for Reducing Deep Learning GPU Memory Usage
Wang, L.-S., Lin, S.-H., & Chou, J. (2025) · International Conference on High Performance Computing in Asia-Pacific Region (HPC Asia) · doi:10.1145/3712031.3712325
DOME: Dynamic Optimization of GPU Memory for Training DNN Models with Limited Resources
Ma, Z., Jian, S., & Tan, Y. (2024) · International Conference on Mobility, Sensing and Networking (MSN) · doi:10.1109/msn63567.2024.00102
SPipe: Hybrid GPU and CPU Pipeline for Training LLMs under Memory Pressure
Ryu, J., Jeong, Y., Park, D., Kim, J.-P., Kim, H., & Lee, J. (2025) · International Conference on Parallel Architectures and Compilation Techniques (PACT) · doi:10.1109/pact65351.2025.00014
“Another widely-used approach to alleviating the memory requirement is offloading [2], [11], [12], [15], [20], [23], [35], [46], [47].”
Huang, Y., & Chen, W. (2026) · Communications in computer and information science · doi:10.1007/978-981-95-3483-8_2
vTensor-Based GPU Memory Management for Edge Deep Learning Training
Gao, H.-R., Luo, D.-H., Wu, Y., Wu, H., & Zhang, W.-B. (2025) · Journal of Computer Science and Technology · doi:10.1007/s11390-025-4788-2
10C ache : Heterogeneous Resource-Aware Tensor Caching and Migration for LLM Training
Afroz, S., Khan, R. I. S., Albahar, H., Han, J., & Butt, A. E. (2025) · ACM Symposium on Cloud Computing (SoCC) · doi:10.1145/3772052.3772236 · Cited by 3
BEEMS: Boosting Machine Vision Efficiency via Computation Graph-Based Memory Smoothing
Shen, H., Liu, F., Liu, J., Jiang, L., & Guan, H. (2026) · ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming · doi:10.1145/3774934.3786430
PASTA: A Modular Program Analysis Tool Framework for Accelerators
Lin, M., Jeon, H., & Zhou, K. (2026) · Proceedings of the CGO/Proceedings of the ... CGO · doi:10.1109/cgo68049.2026.11395237
“In this case, users maintain a map to record the number of times each kernel is invoked—an intuitive yet insightful statistic.”
Memory Saving Strategies for Deep Neural Network Training
Shilova, A. (2021) · theses.fr (ABES) · doi:10.70675/6e1199d8z5998z46e3z9082z49b78a47d92c
Optimizing Memory Usage when Training Deep Neural Networks
Zhao, X. (2024) · theses.fr (ABES) · doi:10.70675/14b989cezef3bz4263z8c05z34a16bd91adc
Enabling Low-Latency, GPU-Efficient Serverless Inference with Model Swapping
Yu, M., Wang, A., Wu, B., Liu, Y., Chen, D., Yu, H., Wang, W., Chen, R., Nie, D. X., Yang, H., et al. (2026) · ACM Transactions on Architecture and Code Optimization · doi:10.1145/3800690
“For example, vDNN [48], Salus [67] and SwapAdvisor [35] leverage host memory for deep learning jobs with large GPU memory footprints; Batch-aware [39] and HUVM [23] optimize GPU memory access for general workloads; POS [57] supports efficient GPU checkpointing and restoring.”
MIDAS: A Dynamic Cross-GPU KV Cache Offloading Framework for LLM on GPU Cluster Systems
Zhang, Z., Xu, C., Liu, Z. Y., Zhi, S., Ma, D., Yin, L., & Li, C. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11461518
Crimson: Collaborative Parameter Updates for Efficient Pipeline Training of Large Language Models
Jiang, Y., Chen, W., 黄干洪, Huang, Y., Hong, Z., Guo, S., & Yu, Y. (2026) · European Conference on Computer Systems · doi:10.1145/3767295.3803606
“As a result, many exsiting research [9, 16, 18, 28] has focused on reducing GPU memory footprints during training.”
朱家安, Wang, Q., Zhao, L., Fang, C., Jin, Z., Bai, Y., Lin, Z., & Li, C. (2026) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2026.3691363
“Forinstance,anoperatorcanbeaugmentedbyanadditional recomputingoperatororamemory-swappingoperatortosave memory[19],[23],[24].”
Offload-DMSS: Heterogeneous Training with Limited CPU Memory Bandwidth by Direct Model State Swap
Zhu, M., Ji, Y., & Zhu, J. (2026) · Communications in computer and information science · doi:10.1007/978-981-95-8318-8_4
Zhou, Y., & Wei, L. (2026) · Journal of Computer Science and Frontier Technologies · doi:10.63313/jcsft.9085
A Survey on Memory-Efficient Fine-Tuning for Large Language Models
Kim, Y., Lee, M., & Lee, S. (2026) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl.a.692
Elastic Memory Remapping for Multi-tenant LLM Serving
Li, R., Pal, S., Pullu, V. N., Sinha, P., Ryoo, J., John, L. K., & Yadwadkar, N. J. (2026) · ACM SIGOPS Operating Systems Review · doi:10.1145/3830422.3830427
VMOC: MLIR-based memory optimization for CNN deployment on vector accelerators
Chen, X., & Liu, Z. (2026) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2026.103933
Maghyastha, M., Underwood, R., Burns, R., & Nicolae, B. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-35251-4_31
“SwapAdvi-sor [7] uses genetic algorithms to control memory allocations and swap decisions. vDNN [18] employs offloading and prefetching.”
SynergyScale: Optimizing Offloading and Task Partitioning for Efficient Model Training
Sun, X., Xu, J., & Wang, Z. (2026) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2026.3719351
“Memory of-floading methods [16], [17], [18], [19] transfer model states between GPU and CPU memory (or secondary storage), while distributed approaches partition parameters and computation across devices [13], [14], [15], [11].”
Observability-Aided Gpu Memory Oversubscription
Pratheek, B., Shah, K., & Basu, A. (2026) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1109/isca66397.2026.00108
“SwapAdvisor [25], Capuchin [51], Sentinel [54] and DeepUM [29], optimize DNN training under UVM with profiling.”
ATX: Accelerator Task Extensions
Gerogiannis, G., Eyerman, S., Torrellas, J., & Heirman, W. (2026) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1109/isca66397.2026.00027
“Unfortunately, inter-leaving CPU and accelerator phases often carries substantial overheads (e.g., data movement) that decrease the end-to-end efficiency [25], [29], [37].”
Global Scheduling of Transient Parameter Materialization for Sharded Training
Zhang, Z., Wang, J., Tang, Z., & Zhao, J. (2026) · ACM Transactions on Architecture and Code Optimization · doi:10.1145/3838601
Ding, Y., & Shao, J. (2026) · ACM Transactions on Architecture and Code Optimization · doi:10.1145/3839239
“Tensor swapping methods reduce GPU memory requirements by swapping parameters or intermediate results into CPU memory [3, 4, 10, 21, 22, 27, 33].”
Selective Tensor Freezing for Efficient Fine-Tuning in Resource-Constrained Federated Learning
Wei, Z., Zhang, Z., Zhou, Y., Wang, Z., Zhao, J., Li, K., & Fu, X. (2026) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2026.3727569
“[20].”
Hermes \({}^{+}\) : Adaptive Memory-Efficient Pipeline Inference for Large Models on Edge Devices
Cai, Z., Han, X., Jiang, N., Zhang, B., Qiang, Y., Wu, T., Liu, Y., & Ma, R. (2026) · ACM Transactions on Autonomous and Adaptive Systems · doi:10.1145/3845607
ZenFlow: Enabling Stall-Free Offloading for LLM Training
Lan, T., Wu, Y., Ma, B., Su, Z., Yang, R., Biçer, T., Tanaka, M., Ruwase, O., Li, D., & Cheng, Y. (2026) · Proceedings of the ACM on Management of Data · doi:10.1145/3837130 · Cited by 8
Metadata
- Authors
- Huang, C.-C., Jin, G., & Li, J.
- Affiliations
- New York University (US)
- New York University, New York, NY, USA
- ORCID
- Li, Jinyang
- Venue
- Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems
- Publisher
- ACM
- Published
- 2020-03-09
- Pages
- 1341-1355
- OpenAlex
- W3012479151
- Cited by
- 158
- References
- 49
- Citations by year
- 2026: 24 · 2025: 30 · 2024: 36 · 2023: 23 · 2022: 21 · 2021: 22 · 2020: 2
- FWCI
- 12.72
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) (publishedVersion)
- Funders
- National Science Foundation (CNS-1816717)
- Topics
- Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Information Systems › Cloud Computing and Resource Management
- Keywords
- GPU memory constraints, operator scheduling, memory allocation, training throughput
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
