16.3 A 28nm 384kb 6T-SRAM Computation-in-Memory Macro with 8b Precision for AI Edge Chips

“Although several compute-in-memory or near-memory approaches based on SRAMs and digital compute 35–38 have been presented in the literature, most of these do not address the energy and time costs of reloading weights, thus making direct side-by-side comparisons against NVM-based weight-stationary…”

“Recent work examined tradeoffs between MAC throughput, energy efficiency and accuracy for analog based CIM [1-3].”

“This paper presents a true FP-IMC macro featuring (1) a hybrid-domain macro structure that enables computation of both the exponent and mantissa in an FP-MAC within the same IMC macro.”

“2) For Challenge1b , we design the CIM macro with a bitline-transpose structure to align the directions of input feeding and weight writing, instead of the previous wordline-feeding solutions [6], [15], [23].”

“Unlike previous SRAMCIMs [1-6] that only support INT-MAC operations, this work supports both FP-MAC and INT-MAC operations and achieve >1.”

“…(CIM) eliminates the bottleneck by integrating multiply–accumulation (MAC) into memory, such as static random access memory (SRAM) [13], [14], [15], [24], [25] and resistive RAM (ReRAM) [6], [21], [22], which has proved to be a promising architecture for energy-efficient NN acceleration.”

“precharge/discharge similar to [6], or 2× transistor-count reduction at the same power”

“Other works using a voltage divider to generate the analog signal for MAC operations [24], [26], [28], [29], [32], [34], [36], [37], [38], [39], [40] are prone to considerable dc current consumption on the bitline (BL).”

“Existing SRAM-based CIM macros implemented arithmetic operations using different analog circuit operations, such as the bitline discharging operations [2], [7], [8], [10], [11], [14], [17] and charge-domain operations using capacitors [3], [4], [12], [13], [15].”

“The previous works [1], [2], [3], [4], [5], [6] use an ADC with 4–5-bit precision, and [2] uses an ADC with adjustable precision, whereas the proposed processor can operate without high-cost ADC.”

“Next, we apply the data analysis strategy to SRAM-based IMCs first and then compare SRAM-, eNVM-, eDRAM-based IMCs, and digital accelerators.”

“As compared to the conventional ADC-first approach [26], [31], [35], our scheme brings a 60.7% reduction in the A/D conversion area as well as a 66.7% reduction in the A/D conversion energy.”

“The mixed approach has been demonstrated either with emerging NVM, such as RRAM [137], [138], [139], [140], [141] or FeFET [142], [143], or with various SRAM cells, generally from 6T to 12T [23], [144], [145], [146], [147].”

“As with previous in-array local-computing-cell structures [29], [31], [32], [33], [34], [35], [37], [41], the two transmission gates (T1 and T2) in the EDC-MUX connect global BL (GBL-GBLB) to local BL (LBL-LBLB) for read and write operations in memory mode.”

“However, in recent years, attention to static random access memory (SRAM)-based implementation has increased rapidly [5], [6], [7], [8], [9], [10], [11], [12], [13], [14].”

“…compared to charge-domain operations [12], [35], while avoiding multiple sequential BL(B) switching as in high-precision current-domain designs [36]; 2) relaxed ADC precision because of the multi-bit ABN implementation, yielding to a better amor-tization of the ADC conversion energy; and 3) an…”

“Recent works show large capacity SRAM-CIM macros, for example, 64 Kb [28], 384 Kb [29], and 4.5 Mb [33], which allows the application to speech recognition and image classification.”

“Advanced artificial intelligence (AI) chips [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37], [38], [39], [40], [41], [42], [43] require high-precision inputs (IN), weights ( W ) , and outputs (OUT) for high inference accuracy as well as high energy efficiency and rapid…”

“…including SRAM-based [15], [16], [17], [18], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37], [38], [39], [40], [41], [42], eDRAM-based [43], [44], [45], and emerging memory-based ones [46], [47], [48], [49], [50], [51], [52].”

“Therefore, most previous designs [15], [16], [17], [21], [22], [23], [25], [27], [29], [30] adopt the combination with programmable digital logic for multibit and fan-in extension, as shown in Fig.”

“These partial MAC results are then processed by MAC aggregation or near-memory computing (NMC) circuitry to complete full-precision MACs [13], [14], [15], [16], [17], [18], [19], [20], [21], [22], [23], [24], [25].”

“The mixed-signal CIM accelerates CNN algorithms [7], [17], [18], whereas digital-domain CIM accelerates AES algorithms [3], [19].”

“The E local term corresponds to the local read energy in near memory computing [49] or CIM architecture with shared computing circuits [41], [50] Apart from the energy overhead, the latency of weight reloading L reload can be reduced and even hidden by ping-pong storage at the penalty of doubled…”

“Fabricated results in [11], [29], and [30] have demonstrated the high energy efficiency of current-domain SRAM-CiM.”

“4% transistor reduction with 2569 F2/b area efficiency[21].”

“Since a smaller memory can not store and process the huge data at the same time, and hence requires frequent energy-consuming weight reloading for computation [17], as shown in Fig.”

“Capacitor-based CIM SRAM [8], [9], [10], [11], [12], [13], [14], [15] has recently become popular because of its high energy efficiency for DCNNs and robustness against PVT variations.”

“grating multiply accumulation (MAC) into memory, such as resistive RAM (ReRAM) [2], [20], [24], [30] and SRAM [22], [25], [26], [32].”

“In a custom accelerator chip design, to reduce the memory-wall overheads of energy and latency in frequent DRAM data accesses [1][2], computing-in-memory (CiM) techniques have been proposed based on CMOS SRAM/eDRAM[3][4][5] and other beyond-CMOS technologies [6][7].”

“Unfortunately, the existing on-chip CIM storage (16 kb–4.5 Mb) [7], [8], [24], [25], [26], [27], [28], [29] cannot hold all weight data for a large NN model of complex datasets.”

“Moving forward, we expect designers to continue to focus on CIM systems with ADC designs that are energy efficient across a lower precision range ( e.g. 2-7 bits), like the medium-precision successive-approximation register (SAR) designs recently shown in some CIM SRAM works [17]–[19], [21].”

“Notably, these initiatives aimed at enlarging the CiM capacity, as presented by the capacity (384 kb in [10] and 1 Mb in [11]) or the CiM macro density (1.041 Mb/mm 2 in 28-nm CMOS in [12]).”

“Most of all, static random-access memory (SRAM)-based CIM techniques [3], [4], [5], [6], [7], [8], [9] have attracted interest due to their fast write speed, low write energy, and good compatibility with the latest CMOS technology [2].”

“The in-memory multiply accumulation (MAC) computations are discussion in [1], [2], [3], [4], [5], [6], [7], [8], [9], [10].”

“…buffers or external DRAM, thereby decreasing the footprint, and reducing high-cost weight reloads from external memories in terms of dataflow [8]. eDRAM has a higher density than SRAM and more mature fabrication than emerging NVMs, making it a promising solution for CiM with high memory…”

“[27], 28, 2b/4b [24], 16, 4b/4b [28], 28, 5b/1b”

“…this so-called memory wall issue, including compute-in-memory (CIM) devices based on SRAM (SRAM-CIM) [1], [2], [3], [4], [5], [6], [7], [8], [9], [10] or nonvolatile CIM (nvCIM) [11], [12], [13], [14], [15], [16], [17], [18], [19], [20], [21], [22], [23], [24], [25], [27], [28], [29], [30],…”

“Thus, many recent works [4], [5], [6], [7], [8], [9], [10], [11], [12] approached compute in memory (CIM) to alleviate the Von Neumann bottleneck and perform the fundamental operation of CNN: MAC (multiply and accumulate) in the memory itself, which eliminates the huge power consumption for data…”

“The SRAM-based CIM design [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11], [12], [13], [14], [15], [16], [17], [18], [19], [20], [21], [22], [23], [24] has surpassed the thousands of TOPS/W computation energy efficiency.”

“It performs computation with current summing [9], charge sharing [10], [11], or a combination of analog and digital circuits [12], [13].”

“Then low-power 1HT1R-based CIM component conducts cognitive processing with CNN for the image segmentation [105,106].”

“MAC operations incorporating a source-injection local multiplication cell and segmented bit-line sharing is discussed in [6].”

“1, computing-in-memory addresses this limitation by incorporating the processing unit into memory cells to perform basic operations (OPs) locally [6], [7], [8], [9], [10], [11], [12], [13], [14], [15], [16], [17], [18], [19], [20], [21], [22], [23], [24], [25], [26].”

“This subsection outlines some of the silicon-verified SRAM-CIM studies that adopt charge domain computing paradigm (see Table 3) [69,75,76].”

“In [17] and [18], the ADC features dynamic resolution by detecting the partial sum voltage but requires an additional circuit and detection phase, lowering the throughput of the CIM.”

“Given that SRAM features full maturity and compatibility in advanced technologies compared with the emerging memory devices like RRAM, MRAM and PCM, etc., the SRAM-based CIMs have been under intensive study for CNN acceleration in energy-efficient edge applications [2], [3], [4], [5].”

“MAC operations are implemented across each column, typically by current or charge summation along the bitline (BL) in analog CIM (ACIM) [23], through an adder tree in DCIM [8], or through a combined analog–digital–hybrid scheme [24].”

“Charged mode IMC [6] represents the multiplication result as capacitive charge in the memory unit, realizes the summation of multiple capacitive charges in the same column through the charge redistribution of the bit line, and transmits the result to the ADC output multiplication and accumulation.”

“Many CIM accelerators [16], [17], [19], [23], [24] have been designed in the past few years, proving CIM is a promising technique for efficient NN acceleration.”

“Many DNN accelerators based on SRAM-based CIM macros [14], [15], [16], [17], [18], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37], [38], [39], [40], [41], [42], [43], [44], [45], [46] have recently been proposed to demonstrate…”

“weight storage density due to larger bitcells, while eDRAM is too leaky, requiring frequent refresh; 2) MRAM, RRAM, and eFLASH require additional process steps increasing the integration cost; 3) Power-hungry DACs for providing Inputs (IN); 4) Accurate reference voltage (VREF) generation circuits for ADCs; 5) Limited…”

“In-SRAM computing is an attractive solution for thememorywall problem experienced by neural networks [2], [3], [4], [5], [6], [7], [8], [9], [10], [11], [12], [13], [14], [15], [16], [17], [18].”

“…CIM (SRAM-CIM) enables multiply-and-accumulate (MAC) operations [20], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37], [38], [39], [40] with short compute latency, high energy efficiency, and fast system-level response times in the…”

“[8], [9], [10], [11], [12], [13], [14].”

“However, the limited on-chip capacity of SRAM and even eDRAM results in inevitable on-off-chip data movement when a large network is deployed [11].”

“Figure 1 (b) shows the utilization for processing the first 18 layers of MobileNet-v2 with state-of-the-art CIM [7], where all of the depth-wise layers achieve near-zero utilization.”

“Prior efforts to reduce ADC precision in CiM hardware have explored techniques such as exploiting input sparsity [9], [12] and quantizing partial sums using non-linear quantization [13], [14] or linear quantization with a smaller dynamic range [15].”

“1) [2].”

“1, BWN and TWN, generally require both positive (Q) and inverse (QB) code in the weight memory [3], [6], [8], while AND-based multi-bit MAC needs only the positive code [2], [4], [7].”

“The intensive MAC operations in CONV and FC can greatly benefit from the use of IMC, as demonstrated by earlier works [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11], [12].”

“CIM designs [16], [17], [18], [19], [20], [21], [22], [23],”

“However, Analog CIM architectures [7]–[11] encounter problems such as non-linearity, signal degradation, and low signal margin.”

“Some works adopt compute-in-memory (CIM) techniques [11], [12], [13], [14] to improve the energy efficiency, yet they consume microwatt-level power consumption that exceeds nanowatt-level power budget.”

“7(a), the 32-bit RISC-V extension instruction includes opcode in [6:0] for custom definition, segments [11:7], [19:15], [24:20] for register indices, [31:25] for funct7 intervals, and bits 12, 13, 14 to control registers rs2, rs1, rd usage.”

“This Work ISSCC’21 [1] VLSI’19 [3] ISSCC’21 [4] ISSCC’21 [5] JSSC’22 [6] Technology 65nm 65nm 150nm 65nm 28nm 65nm Computing Cell Structure 2X6T+6T (SRAM) 8X6T+LCC (SRAM) 1T1R (RRAM) 1T1C (eDRAM) 32X6T +SILMC (SRAM) 8T (SRAM) Computing Topology Charge-IAF Current-mode Current-IAF Charge-sharingCharge-sharing Current-m…”

“Manytypes of memory devices have been proposed as memory for PIM, such as static random access memory (SRAM) [1], [2], [3], [4], [5], resistive random access memory (RRAM) [6], [7], [8], and phase change memory (PCM) [9], [10], [11].”

“Most reported works have focused on conducting forward data propagation for inference [1-2].”

“Novel architectures based on charge [3] or digital [5] computing help to reduce this loss, yet at the expense of area and/or energy efficiency.”

“Therefore, successive approximation (SAR) and single-slope (SS) type ADCs are commonly used to meet the requirements [5], [6].”

“Compared to digital CIM, analog CIM can perform accumulation directly on the bitline or CL [26], [37], [38], [39], significantly increasing storage density.”

“Both issues severely degrade the energy e ffi ciency of the CMOS-based CiMs [11].”

“SRAM-based CIM (SRAM-CIM) enables low-latency multiply-and-accumulate (MAC) operations with high energy e ffi ciency (EEF) [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11], [12], [13], [14].”

“Among CIM accelerators based on various memory technologies (DRAM [16], SRAM [17], and resistive random access memory (ReRAM) [18]), ReRAM-based and SRAM-based CIM have obtained widespread research interest due to their high density and reliability [19].”

“Enhanced SRAM-based IMC architectures [14], [15], [16], [17], [18], [19] achieved Boolean logic functions like AND , NOR , and XOR .”

“…the memory and computing units, processing-in-memory (PIM) architecture is adopted by integrating computing within memory arrays themselves, as described in recent works across diverse types of memories, including SRAM [17], [18], [19], [20], [21], [22], DRAM [4], [23], and NVM [5], [24], [25].”

“Mainstream CIM designs can be broadly categorized into analog [7]–[9] and digital [10]–[12] approaches.”

“Existing works have explored weight sparsity and zero-skipping (ZS) techniques to improve energy efficiency [5], [6].”

“Analog CIM [10, 14, 22, 23, 29] performs MAC operations in analog domains, such as current, charge, and time.”

“…proposes efficient accelerator architectures, including logic-side NPM (e.g., Eyeriss [9], GenASM [7], and CryoCore [6]), logic-side PUM (e.g., SCM [57], Analog Compute Engine [19], and IMNPU [13]), memory-side PUM accelerators (e.g., Ambit [52] and Samsung’s PIM-HBM [41]), and memory-side NDP…”

“ADCs have also been applied to output operations [10], [11], [16], [17]; however, that approach is not energy efficient, due to the need to transfer the ADC output to the readout circuit.”

“As one of the promising solutions to overcome von Neumann bottleneck [16], [17], [18], [19], [20], this technology has been developed to perform energy-efficient computations in many AI applications [21], [22], [23], [24], [25], [26], [27], [28].”

“8/1 bit ISSCC19 Sandwich [3] ISSCC21 [7] 4/4 bit 8/8 bit 8/8 bit 4/4 bit ISSCC21 [8] Input /Weight bit ISSCC20 TWT [6] 8/8 bit 4/4 bit Fig.”

“A 384kb SRAM-based 8b precision CIM in 28nm [66] is demonstrated to have 28% area overhead compared to a pure SRAM array while achieving up to 22.”

“This approach not only increases the capacity of the in-memory architecture, but also improves the overall system performance and efficiency [16], [22].”

“The FoM of the proposed time-digital-hybrid SRAM-CIM was 1.2-1.61x higher than that of previous works using single-domain INT SRAM-CIM [6], [14], [21], due to the high accuracy of digital-domain computation (used for the MSB input) and the high energy ef fi ciency of time-domain computation (used…”

“CIM accelerators based on non-volatile memories (NVM) [4], [5], [6] offer in-situ storage and computation advantages, resulting in smaller area and lower power consumption than SRAM-based accelerators [7], [8], [9].”

“Several CIM circuits have been developed via analog-and digital-domain computation schemes [6], [7], [11].”