The DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
AI and Memory Wall
Open accessAbstract
The availability of unprecedented unsupervised training data, along with neural scaling laws, has resulted in an unprecedented surge in model size and compute requirements for serving/training LLMs. However, the main performance bottleneck is increasingly shifting to memory bandwidth. Over the past 20 years, peak server hardware FLOPS has been scaling at 3.0×/2yrs, outpacing the growth of DRAM and interconnect bandwidth, which have only scaled at 1.6 and 1.4 times every 2 years, respectively. This disparity has made memory, rather than compute, the primary bottleneck in AI applications, particularly in serving. Here, we analyze encoder and decoder Transformer models and show how memory bandwidth can become the dominant bottleneck for decoder models. We argue for a redesign in model architecture, training, and deployment strategies to overcome this memory limitation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“These observations are consistent with Gholami et al.'s (2024) findings on AI's restricted memory capacity and its tendency to produce narrative incoherence in longer texts.”
From How Creative Is AI Writing? Generative and Collaborative AI in Japanese Fiction (2026) · cites this paper as Gholami et al.'s (2024)
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“the relative weakness of zero-shot generation is consistent with research on AI's limitations in sustaining narrative coherence (Gholami et al., 2024)”
From How Creative Is AI Writing? Generative and Collaborative AI in Japanese Fiction (2026) · cites this paper as Gholami et al., 2024
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Gholami et al. (2024) identify restricted memory capacity as one of the primary challenges of AI-assisted fiction writing: as a story progresses, the model tends to lose track of earlier plot points and character details, which result in narrative inconsistencies.”
From How Creative Is AI Writing? Generative and Collaborative AI in Japanese Fiction (2026) · cites this paper as Gholami et al. (2024)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173 · cited 3×
“We examine two different variations of the Trans-former architecture: the encoder architecture (e.g., BERT [7]), which concurrently processes all tokens, and the decoder architecture (e.g., GPT [3, 33]), which runs auto-regressively to process and generate one token at each iteration.”
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 3×
“We examine two different variations of the Trans-former architecture: the encoder architecture (e.g., BERT [7]), which concurrently processes all tokens, and the decoder architecture (e.g., GPT [3, 33]), which runs auto-regressively to process and generate one token at each iteration.”
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 4×
“Deploying recent SOTA models [4, 5, 16, 37] such as GPT-3 [3] or large recommendation systems [29] is quite challenging, as they require distributed-memory deployment for inference.”
InfluentialWilliams, S., Waterman, A., & Patterson, D. A. (2009) · Communications of the ACM · doi:10.1145/1498765.1498785 · Cited by 2,798 · cited 1×
“This is due to the higher memory operations and lower arithmetic intensity of matrix-vector operations inherent in the auto-regressive inference of GPT (see Figure 3c).”
Wulf, W. A., & McKee, S. A. (1995) · ACM SIGARCH Computer Architecture News · doi:10.1145/216585.216588 · Cited by 1,625 · cited 1×
“They predicted that the diverging speed of improvement of how fast computations can be performed versus how fast data can be fetched is going to create a “memory wall” issue [25, 42].”
A Survey of Quantization Methods for Efficient Neural Network Inference
Gholami, A., Kim, S., Dong, Z., Yao, Z., Mahoney, M. W., & Keutzer, K. (2022) · Low-Power Computer Vision · doi:10.1201/9781003162810-13 · Cited by 1,219
Self-citationTraining Compute-Optimal Large Language Models
Hoffmann, J., Borgeaud, S., Mensch, A., Buchatskaya, E., Cai, T., Rutherford, E., Casas, D. D. L., Hendricks, L. A., Welbl, J., Clark, A., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-2176 · Cited by 932 · cited 1×
“In this work, we re-examined this trend by studying more recent data, with a particular focus on hardware used to train AI models, as well as the characteristics of the computations used for training/serving them.”
Method(2020) · doi:10.1109/sc41405.2020.00024 · Cited by 928 · cited 1×
“A promising line of work for that is the Zero framework from Microsoft, which showed how one can train 8 × bigger models with the same memory capacity by removing/sharding redundant optimization state variables [2, 34].”
Hoefler, T., Alistarh, D., Ben-Nun, T., Dryden, N., & Peste, A. (2021) · ETH Zurich · doi:10.3929/ethz-b-000519202 · Cited by 423
ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning
Yao, Z., Gholami, A., Shen, S., Mustafa, M., Keutzer, K., & Mahoney, M. W. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i12.17275 · Cited by 257
Self-citation
Cited by
Sorted by citations
Efficient Memory Management for Large Language Model Serving with PagedAttention
Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C. H., Gonzalez, J. E., Zhang, H., & Stoica, I. (2023) · arXiv · doi:10.48550/arxiv.2309.06180 · Cited by 1,802
Efficient Memory Management for Large Language Model Serving with PagedAttention
Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C. H., Gonzalez, J., Zhang, H., & Stoica, I. (2023) · Proceedings of the 29th Symposium on Operating Systems Principles · doi:10.1145/3600006.3613165 · Cited by 1,817
(2021) · doi:10.1145/3458817.3476205 · Cited by 257
Towards sustainable AI: a comprehensive framework for Green AI
Tabbakh, A., Amin, A., Islam, M., Mahmud, G., Chowdhury, I. K., & Mukta, S. H. (2024) · Discover Sustainability · doi:10.1007/s43621-024-00641-4 · Cited by 168
ANT: Exploiting Adaptive Numerical Data Type for Low-bit Deep Neural Network Quantization
Guo, C., Zhang, C., Leng, J., Liu, Z., Yang, F., Liu, Y., Guo, M., & Zhu, Y. (2022) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1109/micro56248.2022.00095 · Cited by 122
“1 × every two years) [29].”
Two-Dimensional Materials for Brain-Inspired Computing Hardware
Hadke, S. S., Kang, M., Sangwan, V. K., & Hersam, M. C. (2025) · Chemical Reviews · doi:10.1021/acs.chemrev.4c00631 · Cited by 107
Present and Future, Challenges of High Bandwith Memory (HBM)
Kim, K., & Park, M.-J. (2024) · IEEE International Memory Workshop (IMW) · doi:10.1109/imw59701.2024.10536972 · Cited by 70
“Consequently, there has been a notable emergence of the need for memory solutions optimized for high-capacity data processing [1].”
BackgroundPipeEdge: Pipeline Parallelism for Large-Scale Model Inference on Heterogeneous Edge Devices
Hu, Y., Imes, C., Zhao, X., Kundu, S., Beerel, P. A., Crago, S. P., & Walters, J. P. (2022) · Euromicro Conference on Digital System Design (DSD) · doi:10.1109/dsd57027.2022.00048 · Cited by 48
“More importantly, the super-linear growth of the models of ⇠ 240 ⇥ /2-years [11], coupled with the slowdown of Moore’s law has created a significant memory bottleneck for their resource-efficient deployment.”
BackgroundJunior, G. F. D. O., Olgun, A., Yağlıkçı, A. G., Bostanci, F. N., Gómez-Luna, J., Ghose, S., & Mutlu, O. (2024) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca57654.2024.00024 · Cited by 45
Insights into DeepSeek-V3: Scaling Challenges and Reflections on Hardware for AI Architectures
Zhao, C., Deng, C., Ruan, C., Dai, D., Gao, H., Li, J., Zhang, L., Huang, P., Zhou, S., Ma, S., et al. (2025) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1145/3695053.3731412 · Cited by 92
Physics and artificial intelligence: illuminating the future of optics and photonics
Rahman, M. S. S., & Özcan, A. (2024) · Advanced Photonics · doi:10.1117/1.ap.6.5.050500 · Cited by 39
SambaNova SN40L: Scaling the AI Memory Wall with Dataflow and Composition of Experts
Prabhakar, R., Sivaramakrishnan, R., Gandhi, D., Du, Y., Wang, M., Song, X., Zhang, K., Gao, T., Wang, A., Li, X., et al. (2024) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1109/micro61859.2024.00100 · Cited by 36
“Finally, systems that cater to monolothic models have scaled compute TFLOPs much faster than memory bandwidth and capacity, creating the memory wall [14] where the memory system can no longer feed the compute efficiently.”
Large-scale photonic processors and their applications
Pérez, D., & Torrijos‐Morán, L. (2025) · npj Nanophotonics · doi:10.1038/s44310-025-00075-4 · Cited by 44
“Furthermore, the size of cutting-edge AI models has drastically surpassed GPU memory capacity, making data centers the new standard for compute units 92 .”
A Survey: Collaborative Hardware and Software Design in the Era of Large Language Models
Guo, C., Cheng, F., Du, Z., Kiessling, J., Ku, J., Li, S., Li, Z., Ma, M., Molom-Ochir, T., Morris, B., et al. (2025) · IEEE Circuits and Systems Magazine · doi:10.1109/mcas.2024.3476008 · Cited by 32
“Index Terms —Large language model, hardware-software co-design.”
Flex-PE: Flexible and SIMD Multiprecision Processing Element for AI Workloads
Lokhande, M., Raut, G., & Vishvakarma, S. K. (2025) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2025.3553069 · Cited by 29
“In the past 20 years, the TOPS performance has been enhanced by 60000 × , while the DRAM bandwidth and interconnect bandwidth have been improved by only 30 × and 100 × , respectively, creating a disparity known as the memory wall problem [1].”
InfluentialChallenges and Opportunities to Enable Large-Scale Computing via Heterogeneous Chiplets
Yang, Z., Ji, S., Chen, X., Zhuang, J., Zhang, W., Jani, D., & Zhou, P. (2024) · Asia and South Pacific Design Automation Conference (ASP-DAC) · doi:10.1109/asp-dac58780.2024.10473961 · Cited by 32
BitMoD: Bit-serial Mixture-of-Datatype LLM Acceleration
Chen, Y., AbouElhamayed, A. F., Dai, X., Wang, Y., Andronic, M., Constantinides, G. A., & Abdelfattah, M. S. (2025) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca61900.2025.00084 · Cited by 37
“However, the growth of LLM size and complexity continues to outpace the scaling of compute performance and memory capacity in existing hardware platforms [22].”
TSPLIT: Fine-grained GPU Memory Management for Efficient DNN Training via Tensor Splitting
Nie, X., Miao, X., Yang, Z. C., & Cui, B. (2022) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde53745.2022.00241 · Cited by 27
“Also, empirical evidence shows that the size of state-of-the-art NLP models has been increasing at a rate of 240 × every 2 years [13].”
BackgroundBBS: Bi-Directional Bit-Level Sparsity for Deep Learning Acceleration
Chen, Y., Meng, J., Seo, J.-S., & Abdelfattah, M. S. (2024) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1109/micro61859.2024.00048 · Cited by 30
“However, the growth of DNN model size and complexity continues to outpace the scaling of compute performance in existing hardware platforms [12].”
BackgroundExploring Post-training Quantization in LLMs from Comprehensive Study to Low Rank Compensation
Yao, Z., Wu, X., Li, C., Youn, S., & He, Y. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29908 · Cited by 142
“As the progression of hardware bandwidth lags behind that of computational demand [13], the resource demands of extra-large models such as MT-NLG-530B [25]—which necessitates the deployment of multiple nodes for operation—escalate, adding to the complexities of cross-node communication.”
BackgroundSelf-citationArtificial intelligence: Supply chain constraints and energy implications
de Vries, A. (2025) · Joule · doi:10.1016/j.joule.2025.101961 · Cited by 21
Fire-Flyer AI-HPC: A Cost-Effective Software-Hardware Co-Design for Deep Learning
An, W., Bi, X., Chen, G.-T., Chen, S., Deng, C., Ding, H., Dong, K., Du, Q., Gao, W., Guan, K., et al. (2024) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1109/sc41406.2024.00089 · Cited by 35
“And as summarized in Figure 2 [32], while AI’s demand for computational power is growing at 10x per year, Moore’s Law lags behind with hardware FLOPs increase at only 3.0x every two years, DRAM bandwidth at 1.6x, and interconnect bandwidth at 1.4x.”
Chiang, K. H., Ho, Y.-C., Chuang, M.-Y., Chang, C., Kao, Y.-F., Yang, H.-Y., Huang, C. ‐., Chien, Y.-J., Wu, Y.-H., Liu, Y.-C., et al. (2025) · IEEE Symposium on VLSI Technology and Circuits (VLSI Technology and Circuits) · doi:10.23919/vlsitechnologyandcir65189.2025.11074854 · Cited by 20
Fischer, T., Rogenmoser, M., Benz, T., Gürkaynak, F. K., & Luca Benini, C. (2025) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2025.3527225 · Cited by 19
“Moreover, the memory-intensive nature of these workloads results in massive bandwidth at the boundary of the mesh, as the tiles frequently require expensive off-chip memory accesses [4], [5].”
Co-Designed Silicon Photonics Chip I/O for Energy-Efficient Petascale Connectivity
Wang, Y., Wang, S., Parsons, R., Sanyal, S., Gopal, V., Novick, A., Rizzo, A. J., Lipson, M. F., Gaeta, A. L., & Bergman, K. (2024) · IEEE Transactions on Components Packaging and Manufacturing Technology · doi:10.1109/tcpmt.2024.3492189 · Cited by 19
“However, such progress has not been matched by the link technologies that connect these accelerators and HBMs across clusters [10], [11].”
Mu, C., Huang, Z., Jiang, H., Liao, J., Zhou, Y. L., Chen, L., Zhang, Y., Zhu, H., Yang, J., Liu, Q., et al. (2025) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2025.3577335 · Cited by 17
“In addition, the limited interconnect bandwidth between memory and processing units in traditional Von Neumann architectures creates major memory and power bottlenecks [1].”
Mosaic: Breaking the Optics versus Copper Trade-off with a Wide-and-Slow Architecture and MicroLEDs
Benyahya, K., Diaz, A. G., Liu, J., Lyutsarev, V., Pantouvaki, M., Shi, K., Siew, S. Y., Ballani, H., Burridge, T., Cletheroe, D., et al. (2025) · ACM SIGCOMM Conference (SIGCOMM) · doi:10.1145/3718958.3750510 · Cited by 17
Green AI techniques for reducing energy consumption in AI systems
Khan, S., Naz, N. S., Mazhar, T., Tariq, M. U., Shahzad, T., Guizani, S., & Hamam, H. (2025) · Array · doi:10.1016/j.array.2025.100652 · Cited by 29
DASS: Differentiable Architecture Search for Sparse Neural Networks
Mousavi, H., Loni, M., Alibeigi, M., & Daneshtalab, M. (2023) · ACM Transactions on Embedded Computing Systems · doi:10.1145/3609385 · Cited by 18
“However, DNNs are not intrinsically tailored for the limited computing and memory capacities of tiny edge devices, prohibiting their deployment in such applications [18, 57, 51, 50, 56].”
Ferroelectric Transistors: from Materials Innovation to Intelligent Electronic Systems
Li, E., Wang, W., Liu, Y., Wang, R., Luo, C., Zhou, H., Chen, S., Chen, S., Xie, Z., Zhu, K., et al. (2025) · Advanced Materials · doi:10.1002/adma.202515480 · Cited by 14
Efficient Inter-Datacenter AllReduce With Multiple Trees
Luo, S., Wang, R., & Xing, H. (2024) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2024.3419030 · Cited by 12
“Regarding the model size, it might range from the magnitude of 10 7 (e.g., ResNet50) to 10 8 (e.g., Bert), to 10 12 (e.g., GPT3), and continues to increase explosively in the era of large language models [19].”
BackgroundIbrahim, M., Wan, Z., Li, H., Panda, P., Krishna, T., Kanerva, P., Chen, Y., & Raychowdhury, A. (2024) · International Conference on Hardware/Software Codesign and System Synthesis · doi:10.1109/codes-isss60120.2024.00012 · Cited by 14
Choi, S., Sim, J., Shin, J., Ryu, S., Hwang, T., Lim, S. Y., Oh, H., Kwag, J., Lee, J., Song, K., et al. (2024) · Small Structures · doi:10.1002/sstr.202400495 · Cited by 12
Emerging Nonvolatile Memory Technologies in the Future of Microelectronics
Katehi, L. P. B., Yi, S., Lin, Y. C., Banerjee, S., Xia, Q., & Yang, J. J. (2025) · ACS Omega · doi:10.1021/acsomega.5c01414 · Cited by 12
OPIMA: Optical Processing-in-Memory for Convolutional Neural Network Acceleration
Sunny, F., Shafiee, A., Balasubramaniam, A., Nikdast, M., & Pasricha, S. (2024) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2024.3446870 · Cited by 11
“F OR EMERGING machine learning (ML) models being used across application domains [1], [2], [3], the exponential growth in their computational demands has significantly outpaced the rate of advances in traditional computing architectures [4], [5].”
PAL: A Variability-Aware Policy for Scheduling ML Workloads in GPU Clusters
Jain, R., Tran, B., Chen, K., Sinclair, M. D., & Venkataraman, S. (2024) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1109/sc41406.2024.00032 · Cited by 13
“Moreover, new applications, particularly ML models, are emerging frequently and often change the footprint of workloads run on the cluster [10], [31]–[33].”
HCiM: ADC-Less Hybrid Analog-Digital Compute in Memory Accelerator for Deep Learning Workloads
Negi, S., Saxena, U., Sharma, D., & Roy, K. (2025) · Asia and South Pacific Design Automation Conference (ASP-DAC) · doi:10.1145/3658617.3697572 · Cited by 11
Inverse-designed nanophotonic neural network accelerators for ultra-compact optical computing
Sved, J., Song, S., Li, L., Li, G., Meng, D., & Yi, X. (2026) · Nature Communications · doi:10.1038/s41467-026-68648-1 · Cited by 15
“Moreover, physically embedded weights enable in-memory optical computing 4 , signi fi cantly alleviating memory-bandwidth constraints present at run-time in conventional computing architectures at scale 15 .”
Hwang, I., Lee, J., Kang, H., Lee, G., & Kim, H. (2024) · Simulation Modelling Practice and Theory · doi:10.1016/j.simpat.2024.103032 · Cited by 10
TB-STC: Transposable Block-wise N:M Structured Sparse Tensor Core
Liu, J., Zeng, S., Zhao, J., Ding, L., Wang, Z., Li, J., Zhu, Z., Ning, X., Zhang, C., Wang, Y., et al. (2025) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca61900.2025.00075 · Cited by 10
“This success is accompanied by the exponential growth in computation and model size by several hundred times per year, which has far outpaced the advancements in hardware’s capability (about 2-3 × per two years) [13], [66].”
Chrono: Meticulous Hotness Measurement and Flexible Page Migration for Memory Tiering
Qi, Z., Zheng, S., Huang, Y., Hui, Y., Zhang, B., Huang, L., & Mei, H. (2025) · European Conference on Computer Systems · doi:10.1145/3689031.3717462 · Cited by 9
“However, the scalability of DRAM has proven challenging in meeting the escalating requirements [20, 43, 47, 56].”
UniNDP: A Unified Compilation and Simulation Tool for Near DRAM Processing Architectures
Xie, T., Zhu, Z., Li, B., He, Y., Li, C., Sun, G., Yang, H., Xie, Y., & Wang, Y. (2025) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca61900.2025.00054 · Cited by 14
“What is worse, the improvement of hardware computational performance significantly outpaces the growth of DRAM bandwidth [19].”
Forest: Access-aware GPU UVM Management
Lin, M., Feng, Y., Cox, G., & Jeon, H. (2025) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1145/3695053.3731047 · Cited by 14
Emerging memory devices for neuromorphic computing in the Internet of Medical Things
Cao, T., Shen, C., Wang, D., Zhao, H.-N., Tan, R., Zhou, Z., Li, H., Li, B., Zhao, M., & Huang, H. (2025) · Cell Reports Physical Science · doi:10.1016/j.xcrp.2025.102735 · Cited by 12
Vali, A., Comai, S., & Matteucci, M. (2024) · Remote Sensing · doi:10.3390/rs16142561 · Cited by 8
MCAIMem: A Mixed SRAM and eDRAM Cell for Area and Energy-Efficient On-Chip AI Memory
Nguyen, D.-T., Bhattacharjee, A., Moitra, A., & Panda, P. (2024) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2024.3439231 · Cited by 8
“State-of-the-art DNNs [11] require billions of operations and a huge memory to store activations and weights, as evidenced by the 240 × increase in transformer size over two years [12].”
BackgroundBridging the Gap Between LLMs and LNS with Dynamic Data Format and Architecture Codesign
Haghi, P., Wu, C.-S., Azad, Z., Li, Y., Gui, A., Hao, Y., Li, A., & Geng, T. (2024) · Micro · doi:10.1109/micro61859.2024.00118 · Cited by 10
“However, the computational and memory requirements of these models (750 × and 400 × per two years) significantly outpace the hardware progress (1.6-3 × per two years) [16].”
InfluentialA Framework for Near Memory Processing With Computation Offloading and Load Balancing
Maity, S., Ghose, M., & Pasricha, S. (2025) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2025.3542361 · Cited by 8
“While these data-intensive applications bring benefits, they also present challenges, such as the need for swift processing and time-sensitive information extraction, extensive memory usage, and bandwidth dependence [1].”
Yao, Y.-J., Lin, T.-J., Wei, C.-Y., Huang, K., Chen, B.-X., Fang, Y.-T., Chang, H.-J., Fu, Y., Luo, G.-L., Hou, F.-J., et al. (2025) · IEEE Transactions on Electron Devices · doi:10.1109/ted.2025.3540768 · Cited by 8
“I N THE rapidly evolving landscape of semiconductor technology, the demand for high-performance memory solutions is escalating, driven by the proliferation of data-intensive applications such as artificial intelligence (AI) and high-performance computing (HPC) [1], [2], [3].”
Memory technology enabling future computing systems
Fantini, P. (2025) · APL Machine Learning · doi:10.1063/5.0253063 · Cited by 8
Ahmadpanah, S. H., Mirabi, M., Sobhanloo, S., Afsharfarnia, P., Fallah, D., & Bayati, M. (2025) · Ad Hoc Networks · doi:10.1016/j.adhoc.2025.104080 · Cited by 8
3D integrated system for advanced intelligent computing
Munde, R., Vaillancourt, N., Chuang, H.-R., Gu, C., Wang, Y., & Islam, R. (2025) · Advances in Physics X · doi:10.1080/23746149.2025.2599301 · Cited by 8
“The ability to combine technologies on single chips through 3D integration provides better coordination of storage, communication, and computation to accelerate the development of intelligent computing and large scale AI models (Figure 3) [23,29].”
Kwag, J., Choi, S., Kim, D., Lee, J.-Y., Hwang, T., Oh, H.-J., Park, C., & Park, J. (2025) · International Journal of Extreme Manufacturing · doi:10.1088/2631-7990/add7a3 · Cited by 7
Schwarzenberger, A., Fang, D., Kotz, A., Kholeif, H., Wilhelm, C., Eschenbaum, C., Kelany, M., Grünewald, L., Singer, S., Feng, C., et al. (2025) · Optical Fiber Communication Conference · doi:10.1364/ofc.2025.m4k.1 · Cited by 7
OptiPIM: Optimizing Processing-in-Memory Acceleration Using Integer Linear Programming
Liu, J., Zhou, M., Pan, Y., Yang, C.-Y., Josipović, L., & Rosing, T. (2025) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1145/3695053.3731041 · Cited by 11
A Diagonal Structured State Space Model on Loihi 2 for Efficient Streaming Sequence Processing
Meyer, S. M., Weidel, P., Plank, P., Campos‐Macías, L., Shreshta, S. B., Stratmann, P., Timcheck, J., & Richter, M. (2025) · Neuro Inspired Computational Elements Conference (NICE) · doi:10.1109/nice65350.2025.11065663 · Cited by 7
“These models have achieved state-of-the-art accuracy in sequence modeling on tasks including speech commands [10], the Path-X version of the Pathfinder challenge [9], autoregressive language modeling, audio wave-forms, and DNA sequences [12].”
Zhang, H., Ran, S., Lu, L., Liu, Y., Wang, S., Cai, Y., Guo, Y., Li, Y., Chen, J., & Zhou, L. (2025) · Laser & Photonics Review · doi:10.1002/lpor.202500944 · Cited by 7
Samanta, A., Lee, S.-H., Cheng, C.-Y., Palermo, S. M., & Yoo, S. J. B. (2026) · IEEE Journal of Selected Topics in Quantum Electronics · doi:10.1109/jstqe.2026.3674145 · Cited by 7
“Physical limitations such as die/package size, available shoreline for the interconnect I/O, and pocket power budget have become bottlenecks for scaling modern microprocessor architectures, leading to issues such as the “memory wall” problem [7].”
Peykani, P., Emrouznejad, A., Ghanidel, S., Javadi-Sisi, I., & Mirjalili, S. (2026) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-026-10546-2 · Cited by 10
Strategies of high-accuracy memristor-based analogue computing in memory for artificial intelligence
Jiang, Z., Zhao, H., Tang, J., Lu, Y., Qin, Q., Wang, Z., Hu, R., Yu, R., He, Y., Zhang, J., et al. (2026) · Nature Materials · doi:10.1038/s41563-026-02600-y · Cited by 8
“The overhead of these array-level precision compensation methods is qualitatively estimated in Supplementary Table 3.”
InfluentialAn efficient optical-based binary neural network hardware accelerator for harsh environments
Jahannia, B., Ye, J.-C., Altaleb, S., Patil, C., Heidari, E., & Dalir, H. (2024) · OPTO · doi:10.1117/12.3003287 · Cited by 12
A Lightweight PUF-Based Weights Obfuscation Technique for Secure In-Memory AI Inference
Parrini, L., Kar, A., Hettwer, B., Soliman, T., Chauhan, Y. S., Amrouch, H., & Wehn, N. (2025) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2025.3552902 · Cited by 6
“This has led to the on-chip memories becoming a crucial constraint in terms of power consumption and latency in modern digital-embedded AI accelerators [1].”
Kim, D. E., Sharma, T., & Roy, K. (2025) · IEEE transactions on circuits and systems for artificial intelligence. · doi:10.1109/tcasai.2025.3601975 · Cited by 6
“On the other hand, the limitations in traditional computing technology such as the slower growth rate of memory bandwidth and capacity, known as the “memory wall”, demands for new computing paradigms [27], [28].”
Metareasoning for Edge-Cloud Collaborative LLM Planning for Efficient Autonomous Navigation
Navardi, M., Gao, S., Walczak, M., Camacho, F., & Mohsenin, T. (2025) · ACM Transactions on Embedded Computing Systems · doi:10.1145/3765750 · Cited by 7
A Review of Memory Wall for Neuromorphic Computing
Le, D., Arig, B., Isik, M., Dikmen, I. C., & Karadağ, T. (2025) · IEEE International Conference on Computing and Machine Intelligence (ICMI) · doi:10.1109/icmi65310.2025.11141074 · Cited by 6
“Despite this mitigation, present bottlenecks are potentially within the chip memory [7].”
Update Disturbance‐Resilient Analog ReRAM Crossbar Arrays for In‐Memory Deep Learning Accelerators
Choi, W., Stecconi, T., Falcone, D. F., Galetta, M., Clerico, V., Zaccaria, E., Ram, M. S., Porta, A. L., Horst, F., Jubin, D., et al. (2025) · Advanced Science · doi:10.1002/advs.202504578 · Cited by 7
Li, J., Zheng, L., Hong, P., Ye, C., Liu, H., Si, M., Miao, X., & Li, X. (2025) · Advanced Functional Materials · doi:10.1002/adfm.202514194 · Cited by 6
Sub-picojoule-per-bit volitional neuromorphic devices for precise targeting and tracking
Huang, Y., Sun, Q., Dai, F., Wang, F., Zhou, X., Huang, C., Wu, Y., Deng, Y., Luo, L., Li, X. F., et al. (2026) · Nature Communications · doi:10.1038/s41467-025-66295-6 · Cited by 6
Zhu, Y., Liu, Y., Yang, X., Liu, K., Hua, X., Luo, M., Liu, J., Chang, S., Yan, J., Zhang, S., et al. (2026) · Light Science & Applications · doi:10.1038/s41377-026-02209-5 · Cited by 9
“(A diagram of the structure modi fi cation and expansion is presented in the supplementary information note 10).”
Noise-Resilient Photonic Analog Neural Networks
Varri, A., Brückerhoff‐Plückelmann, F., Dijkstra, J., Wendland, D., Bankwitz, R., Agnihotri, A., & Pernice, W. H. P. (2024) · Journal of Lightwave Technology · doi:10.1109/jlt.2024.3433454 · Cited by 6
“Further, their integration with programmable, non-volatile materials such as phase-change materials (PCMs) [12], [13], [14], provide avenues to tackle the memory bottleneck plaguing conventional hardware solutions [15].”
Bandwidth-Latency-Thermal Co-Optimization of Interconnect-Dominated Many-Core 3D-IC
Das, S., Riedel, S., Naeim, M., Brunion, M., Bertuletti, M., Luca Benini, C., Ryckaert, J., Myers, J., Biswas, D., & Milojevic, D. (2024) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2024.3467148 · Cited by 6
A qua : Network-Accelerated Memory Offloading for LLMs in Scale-Up GPU Domains
Kumar, A. V., Antichi, G., & Singh, R. (2025) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/3676641.3715983 · Cited by 12
Neuromorphic Device Based on Material and Device Innovation toward Multimode and Multifunction
Guo, F., Ren, H., Zhang, Y., & Hao, J. (2025) · Advanced Intelligent Systems · doi:10.1002/aisy.202500477 · Cited by 5
GPU-accelerated simulations of turbulence: Review of current applications and future perspectives
Roccon, A., Amati, G., Brandt, L., Calhoun, D. A., Costa, P., Lu, W., Pirozzoli, S., Richter, D. H., Umair, M., You, D., et al. (2026) · Physical Review Fluids · doi:10.1103/vz9c-bbzm · Cited by 5
Tensor Movement Orchestration in Multi-GPU Training Systems
Lin, S., Chen, Y.-J., Cheng, H.-Y., & Yang, C.-L. (2023) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca56546.2023.10071043 · Cited by 6
“For example, some language models require hundreds of gigabytes of training memory [11], [40].”
BackgroundAlhussein, O., Wei, M., & Akhavain, A. (2023) · Globecom · doi:10.1109/globecom54140.2023.10437933 · Cited by 5
“Hardware technology, as predicted by Moore’s law, has been unable to keep up with the exponential growth of computational and storage requirements for modern ML models, resulting in a concerning widening gap [1].”
MethodFixed-point Encoding and Architecture Exploration for Residue Number Systems
Deng, B., Nadendla, B. S., Suo, K., Xie, Y., & Lo, D. C.-T. (2024) · ACM Transactions on Architecture and Code Optimization · doi:10.1145/3664923 · Cited by 5
“For example, the size growth of NLP models reaches 240x every two years [30].”
Design Considerations for Sub-1-V 1T1C FeRAM Memory Circuits
Adnaan, M., Chang, S.-C., Li, H., Liao, Y.-C., Young, I. A., & Naeemi, A. (2024) · IEEE Journal on Exploratory Solid-State Computational Devices and Circuits · doi:10.1109/jxcdc.2024.3488578 · Cited by 5
Ye, W., Gao, H., Zhou, Z., Wang, L., Li, W., Li, Z., Yue, J., Xu, X., Yang, J., Hu, H., et al. (2025) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2025.3545866 · Cited by 4
“With the skyrocketing development of intelligent computing, the memory wall issue has been brought to the front line and become the major performance bottleneck [1].”
Corsair: An In-Memory Computing Chiplet Architecture for Inference-Time Compute Acceleration
Srivastava, S., Arunkumar, A., Kurella, N., Panda, A., Jain, G., Kamath, P., Wutzke, M., Tiruvur, A., Gupta, M. M., Soloveychik, I., et al. (2025) · IEEE Micro · doi:10.1109/mm.2025.3593444 · Cited by 4
“5 While some alternative accelerators 6,8 are used for LLM inference, they are usually not specifically designed for this purpose.”
CXL-INTERPLAY: Unraveling and Characterizing CXL Interference in Modern Computer Systems
Mao, S., Luo, J., Li, Y., Zhou, J., Zhang, W., Liu, Z., Ma, T., & Deng, S. (2025) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1109/dac63849.2025.11132607 · Cited by 4
“Compute Express Link (CXL) [8] provides a comprehensive solu-tion to existing memory wall problems [13] and has garnered significant attention from both industry and academia.”
Sharma, D. D., Choudhary, S., Onufryk, P., & Pelt, R. (2025) · IEEE Symposium on High-Performance Interconnects (HOTI) · doi:10.1109/hoti66940.2025.00027 · Cited by 4
“Memory has been the bottleneck from a bandwidth and capacity point of view, resulting in inefficient use of computing resources, negatively impacting both power consumption and infrastructure costs 1,2,3 .”
Wichrowski, M., Rezaee‐Hajidehi, M., Korelc, J., Kronbichler, M., & Stupkiewicz, S. (2025) · International Journal for Numerical Methods in Engineering · doi:10.1002/nme.70166 · Cited by 4
“While these methods significantly boost computational performance, including applications in solid mechanics [9, 10, 11], their adoption is hindered by the formidable challenge of deriving and implementing complex tangent operators.”
Alinsaif, S. (2026) · Electronics · doi:10.3390/electronics15061179 · Cited by 4
PIMnast: Balanced Data Placement for GEMV Acceleration with Processing-In-Memory
Ibrahim, M. A., Islam, M., & Aga, S. (2024) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1109/scw63240.2024.00137 · Cited by 4
“As memory capacity fails to keep up with this scaling [7], innovations in data-formats (e.g., BF16, INT8, INT4) have been relied on.”
Mario: Near Zero-cost Activation Checkpointing in Pipeline Parallelism
Liu, W., Li, M., Tan, G., & Jia, W. (2025) · ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming · doi:10.1145/3710848.3710878 · Cited by 4
Li, T., Zhong, H., Xu, Y., Narayanan, V., Ni, K., Yang, H., Kämpfe, T., & Li, X. (2024) · doi:10.1145/3676536.3676715 · Cited by 4
“Conventional von Neumann architectures face bottlenecks due to the substantial data movements between processing elements and the memory hierarchy, known as the memory wall [4].”
A Compact One-Transistor-Multiple-RRAM Characterization Platform
Uhlmann, M., Krysik, M., Wen, J., Frohberg, M., Baroni, A., Reddy, K. D. S., Pérez, E., Ostrovskyy, P., Piotrowski, K., Carta, C., et al. (2025) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2025.3555234 · Cited by 4
“I N RECENT years, due to the prospected end of Moore’s law [1], [2] and the arising of data transfer limitations [3], [4], [5], also referred to as the von Neumann bottleneck [6], new computing paradigms have gained significant attention.”
Overcoming Memory Capacity Wall of GPUs With Heterogeneous Memory Stack
Hong, J., Cho, S., & Kim, G. (2022) · IEEE Computer Architecture Letters · doi:10.1109/lca.2022.3196932 · Cited by 4
“Rapidly growing data sizes in various application domains [1], [2] have posed tremendous challenges in architecting GPUs.”
Unlocking High Performance with Low-Bit NPUs and CPUs for Highly Optimized HPL-MxP on Cloud Brain II
Xue, W., Yang, K., Liu, Y., Fan, D., Xu, P., & Tian, Y. (2024) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1109/sc41406.2024.00088 · Cited by 3
“This disparity presents a significant obstacle to fully harnessing the computing potential of AI chips, since memory transfer emerges as the main limiting factor in AI applications, especially noticeable in large language models [16].”
SparSynergy: Unlocking Flexible and Efficient DNN Acceleration Through Multi-Level Sparsity
Yang, J., Wen, M., Shen, J., Yang, J., Cao, Y., He, J., Tang, M., Chen, Z., & Shi, Y. (2025) · Design, Automation, and Test in Europe · doi:10.23919/date64628.2025.10992756 · Cited by 4
“To address the storage and computational challenges posed by DNN workloads [1], a common approach is to leverage the inherent sparsity within DNNs.”
Cha, J., Lee, M., Kwon, J., Lee, J., & Kwon, Y. (2025) · ACM SIGPLAN/SIGBED Conference on Languages, Compilers, and Tools for Embedded Systems (LCTES) · doi:10.1145/3735452.3735538 · Cited by 3
Li, T., Kämpfe, T., Wang, J., Ni, K., Narayanan, V., Yang, H., & Li, X. (2025) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1109/dac63849.2025.11132800 · Cited by 3
“However, in conventional von Neumann architectures, efficient DNN inference faces the bottleneck of substantial data movements between processing units and memories [4].”
Xie, X., Wang, L., Xiao, L., Han, M., Liu, L., Xu, X., Wang, J., Song, Z. Q., & Liao, X. (2025) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1145/3725843.3756113 · Cited by 4
Silicon Photonic Accelerated Memory Pooling for Efficient Compute Resource Allocation
Wu, Z., & Bergman, K. (2025) · IEEE Symposium on High-Performance Interconnects (HOTI) · doi:10.1109/hoti66940.2025.00022 · Cited by 3
Zheng, T., Xu, Y., Mei, C., Chen, Y., Liu, C., Yang, G., Hu, Q., Shi, T., Yuan, Y., Yu, Z., et al. (2025) · Micromachines · doi:10.3390/mi16121304 · Cited by 3
Comparative Study on Device Type Configurations of 2T0C DRAM for Compute-in-Memory Applications
Kong, S., & Shim, W. (2025) · Electronics · doi:10.3390/electronics14234742 · Cited by 3
Panneerselvam, S., Khan, S., Francis, B., Rao, M. S. R., & Manivannan, A. (2025) · ACS Applied Electronic Materials · doi:10.1021/acsaelm.5c02081 · Cited by 3
A survey on memory-efficient transformer-based model training in AI for science
Tian, K., Qiao, L., Liu, B., Jiang, G., Li, S., & Li, D. (2026) · Frontiers of Computer Science · doi:10.1007/s11704-025-50302-6 · Cited by 13
Shen, J., Zhou, Z., Zha, W., Li, Z., Li, W., Wang, B., Zhu, J., Gao, H., Han, Z., Wang, Y. F., et al. (2026) · Digest of technical papers/Digest of technical papers - IEEE International Solid-State Circuits Conference · doi:10.1109/isscc49663.2026.11409272 · Cited by 3
Metadata
- Authors
- Gholami, A., Yao, Z., Kim, S., Hooper, C., Mahoney, M. W., & Keutzer, K.
- Affiliations
- International Computer Science Institute (US)
- Snowflake Inc. (United States) (US)
- University of California, Berkeley (US)
- Venue
- IEEE Micro
- Published
- 2024-05
- Volume
- 44
- Issue
- 3
- Pages
- 33-39
- ISSN
- 0272-1732 (ISSN-L)
- 1937-4143
- OpenAlex
- W4393145114
- Cited by
- 369
- References
- 13
- Citations by year
- 2026: 143 · 2025: 166 · 2024: 47
- FWCI
- 110.55
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Micro (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- KAIST Institutional Repository (KAIST) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- National Science Foundation
- U.S. Department of Energy
- Intel Corporation
- Samsung Advanced Institute of Technology
- Korea Foundation for Advanced Studies
- Samsung
- Office of Naval Research
- Topics
- Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- memory wall, memory bandwidth, large language models, transformer models, DRAM scaling, interconnect bandwidth, model serving
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
