The DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
AI and Memory Wall
Open accessAbstract
The availability of unprecedented unsupervised training data, along with neural scaling laws, has resulted in an unprecedented surge in model size and compute requirements for serving/training LLMs. However, the main performance bottleneck is increasingly shifting to memory bandwidth. Over the past 20 years, peak server hardware FLOPS has been scaling at 3.0×/2yrs, outpacing the growth of DRAM and interconnect bandwidth, which have only scaled at 1.6 and 1.4 times every 2 years, respectively. This disparity has made memory, rather than compute, the primary bottleneck in AI applications, particularly in serving. Here, we analyze encoder and decoder Transformer models and show how memory bandwidth can become the dominant bottleneck for decoder models. We argue for a redesign in model architecture, training, and deployment strategies to overcome this memory limitation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“These observations are consistent with Gholami et al.'s (2024) findings on AI's restricted memory capacity and its tendency to produce narrative incoherence in longer texts.”
From How Creative Is AI Writing? Generative and Collaborative AI in Japanese Fiction (2026) · cites this paper as Gholami et al.'s (2024)
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“the relative weakness of zero-shot generation is consistent with research on AI's limitations in sustaining narrative coherence (Gholami et al., 2024)”
From How Creative Is AI Writing? Generative and Collaborative AI in Japanese Fiction (2026) · cites this paper as Gholami et al., 2024
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Gholami et al. (2024) identify restricted memory capacity as one of the primary challenges of AI-assisted fiction writing: as a story progresses, the model tends to lose track of earlier plot points and character details, which result in narrative inconsistencies.”
From How Creative Is AI Writing? Generative and Collaborative AI in Japanese Fiction (2026) · cites this paper as Gholami et al. (2024)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173 · cited 3×
“We examine two different variations of the Trans-former architecture: the encoder architecture (e.g., BERT [7]), which concurrently processes all tokens, and the decoder architecture (e.g., GPT [3, 33]), which runs auto-regressively to process and generate one token at each iteration.”
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 3×
“We examine two different variations of the Trans-former architecture: the encoder architecture (e.g., BERT [7]), which concurrently processes all tokens, and the decoder architecture (e.g., GPT [3, 33]), which runs auto-regressively to process and generate one token at each iteration.”
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 4×
“Deploying recent SOTA models [4, 5, 16, 37] such as GPT-3 [3] or large recommendation systems [29] is quite challenging, as they require distributed-memory deployment for inference.”
InfluentialWilliams, S., Waterman, A., & Patterson, D. A. (2009) · Communications of the ACM · doi:10.1145/1498765.1498785 · Cited by 2,798 · cited 1×
“This is due to the higher memory operations and lower arithmetic intensity of matrix-vector operations inherent in the auto-regressive inference of GPT (see Figure 3c).”
Wulf, W. A., & McKee, S. A. (1995) · ACM SIGARCH Computer Architecture News · doi:10.1145/216585.216588 · Cited by 1,625 · cited 1×
“They predicted that the diverging speed of improvement of how fast computations can be performed versus how fast data can be fetched is going to create a “memory wall” issue [25, 42].”
A Survey of Quantization Methods for Efficient Neural Network Inference
Gholami, A., Kim, S., Dong, Z., Yao, Z., Mahoney, M. W., & Keutzer, K. (2022) · Low-Power Computer Vision · doi:10.1201/9781003162810-13 · Cited by 1,219
Self-citationTraining Compute-Optimal Large Language Models
Hoffmann, J., Borgeaud, S., Mensch, A., Buchatskaya, E., Cai, T., Rutherford, E., Casas, D. D. L., Hendricks, L. A., Welbl, J., Clark, A., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-2176 · Cited by 932 · cited 1×
“In this work, we re-examined this trend by studying more recent data, with a particular focus on hardware used to train AI models, as well as the characteristics of the computations used for training/serving them.”
Method(2020) · doi:10.1109/sc41405.2020.00024 · Cited by 928 · cited 1×
“A promising line of work for that is the Zero framework from Microsoft, which showed how one can train 8 × bigger models with the same memory capacity by removing/sharding redundant optimization state variables [2, 34].”
Hoefler, T., Alistarh, D., Ben-Nun, T., Dryden, N., & Peste, A. (2021) · ETH Zurich · doi:10.3929/ethz-b-000519202 · Cited by 423
ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning
Yao, Z., Gholami, A., Shen, S., Mustafa, M., Keutzer, K., & Mahoney, M. W. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i12.17275 · Cited by 257
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Items 101–200 of 371
Hydrogen‐State‐Engineered Oxide Semiconductor Channels Enabling Reliable 2T0C DRAM Operation
Lee, J.-Y., Hwang, T., Choi, S., Oh, H., Park, C. K., & Park, J. (2026) · Advanced Electronic Materials · doi:10.1002/aelm.202500825 · Cited by 3
“However, improvements in the density and performance of DRAM have lagged behind those of processors, resulting in a memory wall problem that restricts overall system performance [ 4–6 ].”
Shen, A., Lai, Z., & Zhang, L. (2024) · Applied Sciences · doi:10.3390/app142110025 · Cited by 3
Training large-scale language models with limited GPU memory: a survey
Tang, Y. H., Qiao, L., Yin, L., Liang, P., Shen, A., Yang, Z., Zhang, L., & Li, D. (2025) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.2300710 · Cited by 4
Reliability of III-V/Si heterogeneously integrated lasers
Vallini, F., Narayanan, K., Helms, C., Wang, W., Ghiurcan, G. A., Doussiere, P., Jones, R., Hoshino, I., Tran, Q., & Kamath, K. K. (2025) · SPIE Silicon Photonics · doi:10.1117/12.3043449 · Cited by 3
“These new tasks demand a significant increase in interconnect bandwidth while reducing power consumption due to the number of FLOPs required to train AI models [3,4].”
FlowCheck: Decoupling Checkpointing and Training of Large-Scale Models
Huang, Z., Nie, H., Jia, H., Jiang, B., Guo, J., Lu, J., Wen, R., Lyu, B., Zhu, S., & Wang, X. (2025) · European Conference on Computer Systems · doi:10.1145/3689031.3696088 · Cited by 3
Towards Neural Network Equalizer Implementations for IM/DD Transceivers
Bluemm, C., Liu, B., Rahman, T., Li, B., Schlichtmann, U., & Calabrò, S. (2023) · Opto-Electronics and Communications Conference (OECC) · doi:10.1109/oecc56963.2023.10209762 · Cited by 3
“Within the past 20 years, the peak compute rates (in hardware flops) scaled up about 3000 times more than interconnect bandwidths, as shown in [1].”
BackgroundEfficient Model Switching in RRAM-Based DNN Accelerators
Gu, F.-Y., Lin, I.-C., Li, B., Schlichtmann, U., & Zhang, G. L. (2025) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2025.3550403 · Cited by 3
Reducing Errors and Powers in LPDDR for DNN Inference: A Compression and IECC-Based Approach
Hong, J.-Y., Jang, J.-W., Cho, S.-H., Kong, Y., Kim, S., Kang, Y., Ko, J., Chung, J., & Yang, J.-S. (2025) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2025.103409 · Cited by 3
Self-citationImplications of memory embedding and hierarchy on the performance of MAVeC AI accelerators
Hossain, M. R., & Rahman, M. (2025) · Memories - Materials Devices Circuits and Systems · doi:10.1016/j.memori.2025.100131 · Cited by 5
TierTrain: Proactive Memory Tiering for CPU-Based DNN Training
Swaminathan, S., Kumar, S., Prasad, A., & Subramoney, S. (2025) · ACM SIGPLAN International Symposium on Memory Management (ISMM) · doi:10.1145/3735950.3735956 · Cited by 2
Zhang, H., Cao, L., & Peng, C. Y. (2025) · ACM Transactions on Graphics · doi:10.1145/3731186 · Cited by 2
Hard way or hardware? Taking the heat out of AI
Jayakrishnan, A. R., Hellenbrand, M., Dixon, S. C., Mehonić, A., Silva, J., & MacManus‐Driscoll, J. L. (2025) · APL Machine Learning · doi:10.1063/5.0270385 · Cited by 2
He, Z., Wang, Y., Wu, Z., Han, H., Wei, S., Hu, Y., Li, C., Tu, F., & Yin, S. (2025) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2025.3593946 · Cited by 2
“With the pursuit of higher accuracy and better adaptability on increasingly diverse scenarios, NNs’ structures are becoming more complex [14], [25], and their sizes are rapidly growing at 240 times every two years [23].”
McPAL: Scaling Unstructured Sparse Inference with Multi-Chiplet HBM-PIM Architecture for LLMs
Liu, S., Huang, Z., Yu, J., Liu, Q., & Chen, C. (2025) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1109/dac63849.2025.11132914 · Cited by 2
“Unfortunately, CMOS compute logic scales much faster than inter-chip links and memory capacity [4].”
Ma, X., Zhao, Y., Liu, H., Wang, P., Huang, S., Fan, J., Zhang, L., & Jiang, B. (2025) · Applied Surface Science · doi:10.1016/j.apsusc.2025.164957 · Cited by 2
Advanced Packaging – A must for the Next-Gen AI and HPC Hardware and not Only!
Hachemi, M.-B. (2025) · European Microelectronics and Packaging Conference · doi:10.23919/empc63132.2025.11222514 · Cited by 2
AGILE: Lightweight and Efficient Asynchronous GPU-SSD Integration
Yang, Z., Zhuang, J., Chen, X., Jones, A. K., & Zhou, P. (2025) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1145/3712285.3759778 · Cited by 2
“These new trends necessitate innovative memory extension techniques and optimizations.”
When Light Bends to the Collective Will: A Theory and Vision for Adaptive Photonic Scale-up Domains
Addanki, V. (2025) · ACM Workshop on Hot Topics in Networks (HotNets) · doi:10.1145/3772356.3772395 · Cited by 2
Optical Interconnects for Large Scale Parallel Compute System
TAKAI, A. (2025) · The Journal of the Institute of Electrical Engineers of Japan · doi:10.1541/ieejjournal.145.702 · Cited by 2
HBM-HBF-Centric Memory Pooling Architecture With Custom Base Die for Terabyte-Scale LLM Inference
Park, J., An, H., Suh, H., Yoon, Y., LEE, H., & Kim, J. (2026) · IEEE Computer Architecture Letters · doi:10.1109/lca.2026.3703982 · Cited by 2
“The combined memory footprint therefore far exceeds High Bandwidth Memory (HBM) capacity, creating a critical memory-bound bottleneck during inference [2].”
Brashear, W. A., Reddy, V., Baum, S., Chakravorty, D. K., Dang, F., Pérez, L. M., & Liu, H. (2024) · Practice and Experience in Advanced Research Computing · doi:10.1145/3626203.3670620 · Cited by 2
Fast and Energy Efficient (0.01-2.78 aJ) Logic In Memory Module using SRAM Cells
Tiwari, P., & Misra, P. K. (2024) · IEEE International Conference on Electronics, Computing and Communication Technologies (CONECCT) · doi:10.1109/conecct62155.2024.10677041 · Cited by 2
“These applications also suffer from the performance gap caused by the microprocessor and off chip memory which is also called memory wall problem [3] in Von-Neumann architecture-based processors.”
BackgroundEfficient Training of Large-Scale Neural Networks Using Linear Pipeline Broadcast
Yu, C., & Park, K. (2024) · IEEE Access · doi:10.1109/access.2024.3492314 · Cited by 2
“Accordingly, there is an increase in the amount of memory required for model training [1].”
BackgroundPIMCoSim: Hardware/Software Co-Simulator for Exploring Processing-in-Memory Architectures
Shin, J., An, S., Lee, S., & Lee, S. E. (2024) · Electronics · doi:10.3390/electronics13234795 · Cited by 2
Enhancing DNN Computational Efficiency via Decomposition and Approximation
Schweitzer, O., Weiser, U. C., & Gabbay, F. (2024) · IEEE Access · doi:10.1109/access.2024.3521980 · Cited by 2
Deep Neural Network Accelerator: Exploring the Potential of 3D Chiplet Integration
Wang, S., Wang, T., Bi, S., Jiao, F., Long, X., & Ma, Z. (2024) · International Conference on Pattern Recognition and Artificial Intelligence (PRAI) · doi:10.1109/prai62207.2024.10826865 · Cited by 2
“High levels of parallelism and data throughput significantly amplify the memory and data transfer burden and the rate of data transfer becomes critical factor limiting the actual NPU performance, which is also known as “Memory wall” [20, 21].”
Tomorrow's Modular & Scalable Compute Systems
Aleksov, A., Dogiamis, G. C., Elsherbini, A. A., & Swan, J. M. (2024) · Technical digest - International Electron Devices Meeting/Technical digest · doi:10.1109/iedm50854.2024.10873585 · Cited by 2
WITCH: WeIghTed Coding Scheme for Crosstalk Reduction in High Bandwidth Memory
Jang, S., Jeon, S., Shin, K., Lee, D.-K., Chi, H., Shin, W., Pyo, C., Kim, J., & Jeon, D. (2025) · Asia and South Pacific Design Automation Conference (ASP-DAC) · doi:10.1145/3658617.3697752 · Cited by 2
GOLDYLOC: Global Optimizations & Lightweight Dynamic Logic for Concurrency
Pati, S., Aga, S., Jayasena, N., & Sinclair, M. D. (2025) · ACM Transactions on Architecture and Code Optimization · doi:10.1145/3730584 · Cited by 3
MOST: Memory Oversubscription-Aware Scheduling for Tensor Migration on GPU Unified Storage
Kim, J. H., Jeon, J., Park, J., Choi, S., Gil, M., Hong, S., Koo, G., Yoon, M. K., & Oh, Y. (2025) · IEEE Computer Architecture Letters · doi:10.1109/lca.2025.3580264 · Cited by 2
“D EEP Neural Network (DNN) training continues to push the limits of GPU off-chip memory capacity due to its exponential scaling [1], [2].”
Revisiting Trim for CXL Memory
Lee, H. T., Kim, J., Lee, W., Park, J., Jung, S., Han, J., Kim, B. S. J., Lee, S., & Lee, E. (2025) · ACM Workshop on Hot Topics in Storage and File Systems (HotStorage) · doi:10.1145/3736548.3737825 · Cited by 2
MaGrIP: Magnitude and Gradient-Informed Pruning for Task-Agnostic Large Language Models
Kallakuri, U., Humes, E., Rashid, H.-A., & Mohsenin, T. (2025) · ACM Transactions on Embedded Computing Systems · doi:10.1145/3766068 · Cited by 3
CIMFlow: An Integrated Framework for Systematic Design and Evaluation of Digital CIM Architectures
Qi, Y., Yang, J., Wang, Y., Wang, Y., Wang, D., Tang, L., Duan, C., He, X., & Zhao, W. S. (2025) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1109/dac63849.2025.11133270 · Cited by 2
“However, traditional von Neumann architectures [1], [2] are hitting the “memory wall” due to frequent data transmission between separate computing and memory units [3].”
Energy-Efficient Supervised Learning With a Binary Stochastic Forward-Forward Algorithm
Jaiswal, R., Datta, S., & Makin, J. G. (2025) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2025.3645750 · Cited by 2
“Whether a given algorithm is compute-or memory-dominated depends on several architectural choices, such as dense layers versus convolutional layers, skip connections, and batch normalization versus instance normalization [50], [51].”
Regression with an Ensemble of Noisy Base Functions
Ben-Hur, Y., Cassuto, Y., & Cohen, I. (2022) · Machine learning for signal processing ... · doi:10.1109/mlsp55214.2022.9943368 · Cited by 2
“However, hardware scaling rates cannot meet the exponentially growing requirements of state-of-the-art learning algorithms [1].”
BackgroundDegen, B., Vogel, S., & Rzejak, D. (2024) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/ju8dk · Cited by 2
Toward Fine-Grained Partitioning of Low-Level SRAM Caches for Emerging 3D-IC Designs
Das, S., Kumari, B., Sahoo, S. S., Chen, Y., Myers, J., Milojevic, D., Biswas, D., & Ryckaert, J. (2024) · IEEE Journal on Exploratory Solid-State Computational Devices and Circuits · doi:10.1109/jxcdc.2024.3468386 · Cited by 1
“The effect of the mismatch in the scaling of compute performance and main memory speed is further exacerbated for AI workloads that rely on intensive data movement [2].”
BackgroundA Cycle-Accurate Simulation Platform for NPU-DRAM Integrated Systems
Park, H., Kim, B., Lee, S., Lee, H.-J., & Lee, H. (2025) · International Conference on Consumer Electronics · doi:10.1109/icce63647.2025.10929895 · Cited by 1
“For example, the increasing number of depths and layers in convolutional neural networks (CNNs) [16] and the growing number of parameters in transformer-based natural language processing (NLP) models [3] contribute to higher off-chip memory bandwidth utilization and turnaround latency.”
Unblocking Placement and Routing in Rearrangeable Multi‐Stage Networks
Morais, C. V. R., Penha, J. C., Nacif, J. A. M., & Ferreira, R. (2025) · Concurrency and Computation Practice and Experience · doi:10.1002/cpe.70055 · Cited by 1
Reliable Analog In-Memory Computing with Crossbars: Memristors for Analog Neural Computing
James, A. P. (2025) · Foundations and Trends® in Integrated Circuits and Systems · doi:10.1561/3500000018 · Cited by 1
A 3D Unified Analysis Method (3D-UAM) for Wafer-on-Wafer Stacked Near-Memory Structure
Wang, S., Guo, Y., Tao, W., Jia, X., Bai, F., Tan, J., Wang, Y., Bai, L., Guo, F., Liu, Q., et al. (2025) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2025.3566468 · Cited by 1
“In contrast to the exponential growth in the scale of AI models, the memory capacity of high-performance computing devices has been increasing, roughly doubling every two to three years [5], [6], forming a significant gap with the rapidly growing application demands [7], [8].”
A System Level Performance Evaluation for Superconducting Digital Systems
Kundu, J., Bhattacharjee, D., Josephsen, N., Pokhrel, A., Silva, U. D., Guo, W., Winckel, S. V., Brebels, S., Herr, Q., Herr, A., et al. (2025) · Design, Automation, and Test in Europe · doi:10.23919/date64628.2025.10992879 · Cited by 1
“This is obviously not sustainable given the trend of increasing LLM size [7].”
The Sparsity-Aware LazyGPU Architecture
Liu, C., Yu, M., Sun, Y., & Carlson, T. E. (2025) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1145/3695053.3731009 · Cited by 1
3D IGZO Charge-Coupled Memory DTCO & STCO Analysis for Compute-near-Memory Applications
Wang, B., Akhunov, K., Oh, H., Garćıa-Redondo, F., Chen, Y., Sharma, A., Sun, J., Gamage, S. S. H. U., Rosmeulen, M., Mahato, S. B., et al. (2025) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas56072.2025.11043541 · Cited by 1
DPE-CIM: Compute-In-Memory Accelerator using Dynamic Posit Encoding and Speculative Alignment
He, J., Cheng, K.-T. T., & Tsui, C.-Y. (2025) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas56072.2025.11043319 · Cited by 1
Efficient Scheduling of Transformer Neural Network Computation for Edge-AI Deployment
Sedlak, D., Klhufek, J., Mrázek, V., & Vašíček, Z. (2025) · Genetic and Evolutionary Computation Conference Companion (GECCO Companion) · doi:10.1145/3712255.3734345 · Cited by 1
Silicon Photonics in AI Interconnects
de Valicourt, G., Pupalaikis, P., Giles, R., Lamponi, M., Elsinger, L., Liu, S., Sawyer, B. M. D., Proesel, J., Le, S. T., Ho, E., et al. (2025) · OptoElectronics and Communications Conference and International Conference on Photonics in Switching and Computing (OECC/PSC) · doi:10.23919/oecc/psc62146.2025.11111327 · Cited by 1
“The rapid scaling of artificial intelligence (AI) models is driving the development of increasingly large compute clusters, where performance is often limited by data movement rather than compute itself [1].”
Jeon, M.-U., Kim, P.-S., Han, M.-H., Lee, S.-H., Lee, H.-M., Kim, S.-B., Park, J.-H., Cho, K.-C., Park, J., & Park, J. (2025) · Nanomaterials · doi:10.3390/nano15161248 · Cited by 1
Self-citationMedina, R., Yu, P., Levisse, A., Biswas, D., Zapater, M., Ansaloni, G., Catthoor, F., & Atienza, D. (2025) · ACM Transactions on Embedded Computing Systems · doi:10.1145/3762641 · Cited by 2
SPICE Modeling of GaN MicroLEDs for Optical Communication
Badaoui, S. E., Maître, P. L., Cibié, A., Simon, J., Lardeau‐Falcy, A., Bilde, J., Cherruault, L., Arch, M., Pelissier, M., & Guennec, Y. L. (2025) · Journal of the Society for Information Display · doi:10.1002/jsid.2105 · Cited by 1
Analysis of UCIe 48/64-GT/s Electrical Links
Wu, Z., & Guo, J.-R. (2025) · IEEE Open Journal of the Solid-State Circuits Society · doi:10.1109/ojsscs.2025.3605558 · Cited by 1
Wang, G., Cai, S., Li, Z., Li, W., Lyu, D., Sun, Y., Jiang, J., & He, G. (2025) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1109/dac63849.2025.11133000 · Cited by 2
“However, the superior performance of DNNs comes with rapid scaling of model size and computation complexity [1].”
Ultrafast Generative AI by Ultradense 3D Integration: A Case Study on LLM-based Edge Inference
Akarvardar, K., Sun, X., Crafton, B., Peng, X., Mori, H., Bhattacharjee, A., Fujiwara, H., & Wong, H. ‐. P. (2025) · ACM Transactions on Design Automation of Electronic Systems · doi:10.1145/3768168 · Cited by 1
Accelerating LLM Inference via Dynamic KV Cache Placement in Heterogeneous Memory System
Fang, Y., Xie, R., Haq, A. U., Ma, L., Maghraoui, K. E., Wang, N., Wang, M., Liu, L., & Zhang, T. (2025) · IEEE Computer Architecture Letters · doi:10.1109/lca.2025.3622724 · Cited by 2
“However,thebandwidthofmemorysubsystemshasnotkeptpacewiththeacceleratingcomputecapabilitiesofmodernproces-sors[2],resultinginamismatchthatlimitstheinferencethroughput.”
Schwarzenberger, A., Kotz, A., Fang, D., Kholeif, H., Eschenbaum, C., Grünewald, L., Wilhelm, C., Singer, S., Kelany, M., Feng, C., et al. (2025) · Journal of Lightwave Technology · doi:10.1109/jlt.2025.3625813 · Cited by 1
Athena, F. F., Kang, J., Passlack, M., Safron, N. S., Dede, D., Jana, K., Saini, B., Wang, X., Liu, S., Hartanto, J., et al. (2025) · IEEE Transactions on Electron Devices · doi:10.1109/ted.2025.3616087 · Cited by 1
“T HE growing disparity between processor speed and o ff - chip memory bandwidth—known as the “memory wall” has renewed interest in compact on-chip memories [1], [2].”
Bit-Serial Acceleration of LLM Inference With Mixture-of-Datatype Quantization
Chen, Y., Chang, C.-C., Dai, X., AbouElhamayed, A. F., Andronic, M., Constantinides, G. A., & Abdelfattah, M. S. (2025) · IEEE Transactions on Computers · doi:10.1109/tc.2025.3628193 · Cited by 1
“Furthermore, the growth of LLM size and complexity continues to outpace the scaling of compute performance and memory capacity in existing hardware platforms [3].”
MLP-Offload: Multi-Level, Multi-Path Offloading for LLM Pre-training to Break the GPU Memory Wall
Maurya, A., Rafique, M. M., Cappello, F., & Nicolae, B. (2025) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1145/3712285.3759864 · Cited by 1
Implementation of Boolean Logic Operations and Refresh Circuit for 2T DRAM-Based PIM Architecture
Kim, Y., & Kwon, M.-W. (2025) · Electronics · doi:10.3390/electronics14224483 · Cited by 1
Murali, G., Bhargava, M., Salahuddin, S. M., Chen, Z., Pandey, A., Srinivasa, S., Budhkar, P., Kuttappa, R., Honkote, V., Sakthi, P., et al. (2025) · doi:10.1109/iccad66269.2025.11240957 · Cited by 1
“Several recent analyses characterize a growing “AI memory wall” [5]–[7] where bandwidth and capacity dominate system bottlenecks for large transformer inference.”
Guo, W., Kundu, J., Tos, U., Kong, W., Sisto, G., Evenblij, T., & Perumkunnil, M. (2025) · IEEE International Symposium on Workload Characterization · doi:10.1109/iiswc66894.2025.00019 · Cited by 1
“Using the optimal technique for the model, and using the optimal communication algorithm for the optimal network topology are crucial to minimizing communication overhead [12], especially since communication bandwidth scales more slowly than the DRAM bandwidth ( 1.4x every 2 years) [13].”
Memory-Efficient Training for Deep Speaker Embedding Learning in Speaker Verification
Liu, B., & Qian, Y. (2025) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2025.3639717 · Cited by 1
“Given the limited GPU memory capacity [57], this restricts the training of very deep neural networks, particularly in resource-constrained scenarios, making it difficult to train large-scale speaker verification systems.”
Multi-Modal Explainable Deep Learning-Assisted Signal Integrity Prediction Methodology
Yoon, J., Kim, T., Chun, K. C., Kim, J., & Jung, S. (2025) · IEEE International Conference on Electronics, Circuits and Systems (ICECS) · doi:10.1109/icecs66544.2025.11270621 · Cited by 1
“The explosive proliferation of AI-centric workloads such as large-language-model training, neural machine translation and graph analytics continues to drive dynamic random access memory (DRAM) interfaces toward multi-gigabit per-pin bandwidth [1].”
Recent Progress in Capacitor-Less IGZO DRAM Technology
Zhang, X., Yang, G., Geng, D., Lu, N., Li, L., & Liu, M. (2025) · IEEE electron devices reviews. · doi:10.1109/edr.2025.3642785 · Cited by 1
“This growing gap causes significant system-level limitations as traditional memory fails to keep pace with exponentially increasing AI workload complexity and data throughput demands [7].”
A Scalable 1L2D Multi-Core Near-DRAM Computing Accelerator Based on 3D Hybrid Bonding for AI Models
Han, Z., Cao, Y., Xuanzhi, L. I. U., CHENG, J., & Jianguo, Y. A. N. G. (2025) · IEICE Transactions on Electronics · doi:10.1587/transele.2025ecp5020 · Cited by 1
Improving Error Resilience of Nanoscale Memories [Feature]
Hu, J., Liu, S., Reviriego, P., Wang, S., Qi, X., Gao, Z., & Lombardi, F. (2026) · IEEE Nanotechnology Magazine · doi:10.1109/mnano.2025.3638504 · Cited by 1
“Additionally, the training and inference processes of modern ML models often face the “memory wall” problem [2], i.e., the growth rate of memory bandwidth lags the computational power, so resulting in time spent accessing memory to even exceed computation.”
Panneerselvam, S., Khan, S., & Manivannan, A. (2026) · physica status solidi (RRL) - Rapid Research Letters · doi:10.1002/pssr.202500474 · Cited by 1
RoboPIM: A ReRAM-Based Accelerator for LLM-Based Robotics Applications via Dynamic Task Slicing
Xiao, W., Wang, J., Chen, D., Li, H., Guizani, M., Chen, M., & Wu, T. (2026) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2026.3660201 · Cited by 1
“FPGAs and ASICs su ff er from high data movement [28] during memory-intensive LLM inference, and static / dynamic random access memory (SRAM) PIMs [29], [30] are limited by low density and significant leakage / refresh power, which are prohibitive for energy-constrained robotics.”
Liu, S., Yu, J., Li, P., Feng, L., & Chen, C. (2026) · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · doi:10.1109/jetcas.2026.3660689 · Cited by 1
“[9], as large-scale weights and KV caches must be frequently transferred.”
Oh, H., Lee, J.-Y., Beom, J. Y., Sun, J. E., Lee, C.-H., Park, C.-K., Kim, D., Kim, M., Lee, J., Cho, S., et al. (2026) · Applied Surface Science · doi:10.1016/j.apsusc.2026.166278 · Cited by 1
Zou, S., Yang, Y., Yang, W., Huang, T., Du, X., Wang, Y., & Li, Y. (2026) · Microelectronics Reliability · doi:10.1016/j.microrel.2026.116041 · Cited by 1
Area-Efficient Heterogeneous MRAM for High-Performing AI Acceleration
Mugdho, S. S., Gupte, K. K., Hasan, M. S., & Wang, C. (2025) · doi:10.1109/ccmcc67628.2025.11380744 · Cited by 1
“Enabling larger on-chip memory is a promising solu-tion to mitigating the memory bottleneck [5]–[7].”
Exploiting LPDDR6 Metadata to Cache Byte-Addressable Non-Volatile Main Memories
Wilbert, N., Christ, D., & Jung, M. (2025) · doi:10.1109/ccmcc67628.2025.11380739 · Cited by 1
“To be precise, with the growth in peak floating-point performance vastly outpacing the progress in bandwidth and capacities achievable by traditional DRAM architectures, these workloads are often memory-rather than compute-bound [1].”
Tawa: Automatic Warp Specialization for Modern GPUs with Asynchronous References
Chen, H., Fan, B., Collins, A., Hagedorn, B., Gaburov, E., Masuda, M., Brookhart, M., Sullivan, C., Knight, J., Zhang, Z., et al. (2026) · Proceedings of the CGO/Proceedings of the ... CGO · doi:10.1109/cgo68049.2026.11394849 · Cited by 1
PIM-ORAM: Towards Oblivious RAM Primitives in Commodity Processing-In-Memory
Woo, B. O., Duy, K. D., Han, Y., Kang, B. B., & Lee, H. (2025) · Annual Computer Security Applications Conference · doi:10.1109/acsac67867.2025.00083 · Cited by 1
“The cloud today is suffering from the so-called memory wall problem [20], [21], [22], where the memory band-width becomes a bottleneck in multiple memory-intensive computations carried out by powerful processors and accelerators.”
Enhancing Parallelism and Energy-Efficiency in SOT-MRAM based CIM Architecture for On-Chip Learning
Sehgal, A., Shukla, A. K., Diware, S., Soni, S., Dhull, S., Shreya, S., Roy, S., & Bishnoi, R. (2025) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1109/dac63849.2025.11424425 · Cited by 1
“Traditional computing systems rely heavily on data movement between processing and memory units, facing memory-wall challenges like high energy consumption and latency [1], [2].”
Heterogeneous Computing: The Key to Powering the Future of AI Agent Inference
Zhao, Y., & Liu, J. (2026) · Computer · doi:10.1109/mc.2026.3659288 · Cited by 1
RaidenSwap: A Multi-Swap Remote System for Multi-core Applications
Liu, K., Liu, K., Zhang, X., Yuan, H., Zheng, X., Liu, N., Wang, S., Zhang, G., Bao, Y., Chen, M., et al. (2026) · European Conference on Computer Systems · doi:10.1145/3767295.3803575 · Cited by 1
Ferro, E., Ottaviano, A., Lammie, C., İslamoğlu, G., Garofalo, A., Conti, F., Boesch, T., Benini, L., Sebastian, A., & Boybat, I. (2026) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2026.3692104 · Cited by 1
“Asthecomplexity of Deep Learning (DL) models increases, traditional von Neumann architectures become increasingly power-and energy-ine ffi cient due to the well-known memory wall problem [5], [6], [7].”
A Unified Flash Memory Platform for Mode‐Adaptive and Robust AI Computation
Yu, D., Hwang, H., Oh, B., Lee, J., Yu, S., Kim, T., & Kim, H. (2026) · Advanced Science · doi:10.1002/advs.76619 · Cited by 1
Thakur, V., More, V. N., & Bhatt, A. (2026) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2026.103940 · Cited by 1
Shi, Y., Li, H., Hua, Q., Zhai, J., & Hu, W. (2026) · Chemical Engineering Journal · doi:10.1016/j.cej.2026.180619 · Cited by 1
Co-packaged optics for high-performance computing and artificial intelligence
Kim, B., Choi, S. H., Zograf, G., Yoo, Y. J., Baek, Y., Kim, S., Kim, J., Kim, J., Chae, S. H., Kim, H., et al. (2026) · Nature Electronics · doi:10.1038/s41928-026-01681-6 · Cited by 2
“Even with advanced packaging approaches such as chip-on-wafer-on-substrate, embedded multi-die interconnect bridge and high-density silicon interposers, the intrinsic energy–distance trade-off of inter-package electrical interconnects remains a fundamental barrier 7,8 .”
Self-citationSigns Unveiled: The Power and Promise of AI-Based Sign Recognition Systems
Dubey, P., Bhagat, P. N., Khan, G. H. A., & Akhter, P. (2024) · BENTHAM SCIENCE PUBLISHERS eBooks · doi:10.2174/9789815256864124010007 · Cited by 1
NeutronCache: An Efficient Cache-Enhanced Distributed Graph Neural Network Training System
Zhao, C., Dong, S., Zhao, Y., & Li, Y. (2024) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3627673.3679815 · Cited by 1
Sahoo, S. S., Abdi, D. B., Ryckaert, J., Myers, J., & Biswas, D. (2024) · IEEE International System-on-Chip Conference (SOCC) · doi:10.1109/socc62300.2024.10737780 · Cited by 1
Efficient Deep Neural Network Training With a Novel 5.3-Bit Block Floating Point Data Format
Sadi, M. H., Sudarshan, C., Nassif, S. R., & Wehn, N. (2025) · IEEE Transactions on Circuits and Systems for Artificial Intelligence · doi:10.1109/tcasai.2025.3532254 · Cited by 2
“One of the major contributors to the total training energy consumption is accessing data from external memory devices like DRAMs [2].”
SSC: An SRAM-Based Silence Computing Design for On-chip Memory
Chen, Z., Deng, Q., Hu, Y. H., He, X., Huang, L., & Wang, Y. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-1545-2_4 · Cited by 1
LightCL: Compact Continual Learning with Low Memory Footprint For Edge Device
Wang, Z., Cheng, F., Ji, K., & Huang, B. (2025) · Asia and South Pacific Design Automation Conference (ASP-DAC) · doi:10.1145/3658617.3697784 · Cited by 1
Photonic Analog-to-Digital Architecture for Accelerating Multiply-Accumulate Operations
Nauman, N., Robinson, J. C., Wang, Y., Jang, K., Meng, X. Z., & Bergman, K. (2025) · Optical Fiber Communication Conference · doi:10.1364/ofc.2025.w3d.5 · Cited by 1
ARCANE: Adaptive RISC-V Cache Architecture for Near-memory Extensions
Petrolo, V., Guella, F., Caon, M., Schiavone, P. D., Masera, G., & Martina, M. (2025) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1109/dac63849.2025.11132598 · Cited by 1
“Traditional von Neumann architectures struggle with scaling due to the so-called “memory wall” [1] bottleneck, impacting performance and energy efficiency.”
Lu, L., Zhang, H., Ran, S., Liu, Y., Wang, S., Cai, Y., Guo, Y., Li, Y., Chen, J., & Zhou, L. (2024) · Research Square · doi:10.21203/rs.3.rs-5365298/v1 · Cited by 1
Scientific Machine Learning: A Symbiosis
Keith, B., O’Leary-Roseberry, T., Sanderse, B., Scheichl, R., & Waanders, B. V. B. (2024) · Foundations of Data Science · doi:10.3934/fods.2024051 · Cited by 5
How Creative Is AI Writing? Generative and Collaborative AI in Japanese Fiction
Asano, Y., & Bastos, M. (2026) · Journal of Creative Communications · doi:10.1177/09732586261463934
Convolución dinámica sobre red pre entrenada en CTPA
Javier, D. A. C. (2022) · doi:10.11144/javeriana.10554.58710
Yu, C., & Park, K. (2023) · Research Square · doi:10.21203/rs.3.rs-3596530/v1
Yang, Y., Fan, Q., Yan, T., Pei, J., & Li, G. (2024) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2024.3379165
“Research shows that the growth rate of model parameter quantity has reached 240 times every two years, and the growth rate of computing quantity can reach 750 times every two years [14], while the corresponding growth rate of hardware storage and transistorsbecomestwiceeverytwoyears.”
Li, F., Xia, Y., Xiao, S., Yang, P., Zhu, X., Li, B., Zhou, J., Han, G., & Mao, W. (2024) · IEEE International System-on-Chip Conference (SOCC) · doi:10.1109/socc62300.2024.10737831
“The advent of artificial intelligence generated content (AIGC) through large-scale neural network models has increased al-gorithm complexity [1], which has led to challenges in data handling within traditional von-Neumann structure [2].”
A Framework for SOT-MRAM Scaling Road-Map with Density and Application Evaluation
Kallinatha, H. D., & Talawar, B. (2024) · International Conference on Computer Systems (ICCS) · doi:10.1109/iccs62594.2024.10795852
“Chip Multiprocessor systems depend heavily on extensive cache hierarchies to minimize memory access delays and reduce the need to access slower off-chip memory [1].”
High-density high-speed linear I/O for AI clusters
Winzer, P. J., & Harding, D. J. (2025) · Next-Generation Optical Communication: Components, Sub-Systems, and Systems · doi:10.1117/12.3045631
“Figure 1 [1][2] compares the scaling of AI model parameters with technology scaling in electronic data processing and communications.”
Metadata
- Authors
- Gholami, A., Yao, Z., Kim, S., Hooper, C., Mahoney, M. W., & Keutzer, K.
- Affiliations
- International Computer Science Institute (US)
- Snowflake Inc. (United States) (US)
- University of California, Berkeley (US)
- Venue
- IEEE Micro
- Published
- 2024-05
- Volume
- 44
- Issue
- 3
- Pages
- 33-39
- ISSN
- 0272-1732 (ISSN-L)
- 1937-4143
- OpenAlex
- W4393145114
- Cited by
- 369
- References
- 13
- Citations by year
- 2026: 143 · 2025: 166 · 2024: 47
- FWCI
- 110.55
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Micro (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- KAIST Institutional Repository (KAIST) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- National Science Foundation
- U.S. Department of Energy
- Intel Corporation
- Samsung Advanced Institute of Technology
- Korea Foundation for Advanced Studies
- Samsung
- Office of Naval Research
- Topics
- Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- memory wall, memory bandwidth, large language models, transformer models, DRAM scaling, interconnect bandwidth, model serving
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
