CN118132507B - A new in-memory computing architecture that supports a variety of workloads - Google Patents

Abstract

The present invention discloses a novel in-memory computing architecture supporting multiple workloads, relates to in-memory computing technology, and proposes this solution to solve the problem of insufficient diversity of computing tasks in the prior art. It includes an in-memory computing array, a precharge module, a read word line and write word line driver module, an input value and control signal driver module, a bit line driver module, a sensitive amplifier group, a peripheral computing logic module, a reconfigurable address generation unit module, and a top-level control module. Each module performs data calculation and circulation under the control of the top-level control module. The advantage is that it can support any existing algorithm that can be run on the CPU to achieve universal computing capability.

Description

A new in-memory computing architecture that supports a variety of workloads

Technical Field

The present invention relates to in-memory computing technology, and in particular to a novel in-memory computing architecture that supports multiple workloads.

Background technique

In the era of artificial intelligence, graphics processing units (GPUs), as the cornerstone of computing power, have made great contributions to accelerating AI computing. However, GPUs also have many limitations, including high overall power consumption, significant instruction execution delays, and lack of hardware programmability. Under standard operating conditions, GPUs typically consume more energy than highly customized chips. Even the latest Nvidia GPU chips that deploy the Volta architecture still require higher power consumption to achieve the same computing performance than fully customized ASIC chips. In addition, when GPUs perform multi-threaded tasks, the frequent DRAM read and write operations result in higher latency, which increases the latency of instruction execution. Furthermore, the hardware structure of the GPU is fixed at the beginning of its design and does not have the flexibility of temporary programming.

In response to these challenges, in-memory computing technology provides an innovative solution. By integrating SRAM storage units and computing units, this technology significantly shortens the distance that data needs to be transmitted between the two, greatly reducing the energy consumption and latency caused by data transmission. However, in the existing technology, the in-memory computing architecture can only optimize one or a limited number of operators. During AI training and reasoning, it is often necessary to switch between different layers of the same neural network or between multiple neural networks that share the same hardware resources (for example, performing image classification and speech recognition at the same time), which requires the in-memory computing architecture to support a variety of computing tasks, and the actual available in-memory computing processing architecture lacks relevant versatility.

Summary of the invention

The purpose of the present invention is to provide a new in-memory computing architecture that supports multiple workloads to solve the problems existing in the above-mentioned prior art.

The present invention provides a novel in-memory computing architecture that supports multiple workloads, including:

An in-memory computing array for storing input data and performing data computing according to operator selection control signals;

A precharge module, used for charging and pulling up the write bit line signal and the write bit line inversion signal to VDD when the in-memory computing array enters the read phase;

A read word line and write word line driver module, used for providing a read word line signal and a write word line signal for the in-memory computing array;

An input value and control signal driving module, used to provide a part of the data signal and operator selection control signal of the in-memory computing array in the next round, and to provide an operator selection control signal of the peripheral computing logic;

A bit line driving module, used for driving signals of write bit lines and their inverse signals;

A sensitive amplifier group, used for reading out the data stored in the in-memory computing array;

A peripheral calculation logic module, used for performing left shift, right shift, or high-bit filling operation on the read data of the sense amplifier group;

The reconfigurable address generation unit module is used to read out the logical address of each operation module in the next round and send out the data signal when the algorithm instruction mapping address and the handshake signal are synchronized;

The top-level control module is used to parse external command signals and control the flow of data signals in the new in-memory computing architecture, as well as adjust the read and write access time of data signals.

The novel in-memory computing architecture supporting multiple workloads described in the present invention has the advantage of supporting any existing algorithm that can be run on the CPU to achieve universal computing capabilities. At the same time, under the conditions of 1.8V power supply voltage and 800MHz operation, the architecture can achieve an operating capacity of 47.65TOPS/W, an average throughput of 3.7GOPS, and a maximum of 5.2GOPS. Due to the significant reduction in the data transfer path, the power consumption of in-memory computing is reduced by about 10 times on average compared to other 64-bit processors.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a schematic diagram of the novel in-memory computing architecture described in the present invention.

FIG. 2 is a schematic diagram of the structure of the SRAM storage unit in the present invention.

FIG. 3 is a schematic diagram of the interaction between the SRAM storage unit and the internal computing logic unit in the present invention.

FIG. 4 is a schematic diagram of addition and subtraction operations of the internal calculation logic unit in the present invention.

FIG. 5 is a functional schematic diagram of the peripheral computing logic unit described in the present invention.

FIG. 6 is a functional schematic diagram of the reconfigurable address generation unit module in the present invention.

FIG. 7 is a schematic diagram of the process of compiling a mapping address table according to the novel in-memory computing architecture described in the present invention.

FIG8 is a timing diagram of the phase in which the reconfigurable address generation unit module of the present invention writes data into the mapping address table.

FIG. 9 is a timing diagram of the stages in which the reconfigurable address generation unit module of the present invention participates in the in-memory computing array computing process.

FIG10 is a schematic diagram of the workflow of the novel in-memory computing architecture described in the present invention.

FIG11 is a schematic diagram of data flow and control flow of the novel in-memory computing architecture described in the present invention.

Detailed ways

As shown in FIG. 1 , the novel in-memory computing architecture supporting multiple workloads described in the present invention includes the following core component circuit modules as a circuit entity:

(1) An in-memory computing array CIMA, comprising a plurality of SRAM memory cells arranged in rows and columns and internal computing logic cells arranged between the rows. The plurality of SRAM memory cells are configured to store write data signals and partial data signals for the next round of computing. The plurality of internal computing logic cells are configured to store partial data signals for the next round of computing and read SRAM memory cell data signals, and participate in addition, subtraction, multiplication, and Boolean logic operations together.

(2) A precharge module is configured to charge and pull up the write bit line signal WBL and the write bit line inversion signal WBLb to VDD when the in-memory computing array enters the read phase.

(3) A read word line and write word line driver module configured to provide a plurality of read word line signals RWL and write word line signals WWL.

(4) An input value and control signal driving module, configured to provide partial data signals for the next round and multiple operator selection control signals required by the internal computing logic unit and the peripheral computing logic.

(5) A bit line driving module configured to provide driving signals of a write bit line signal WBL and a write bit line inverted signal WBLb.

(6) A sense amplifier group, configured to provide a plurality of sense amplifiers SAs to read out data signals stored in the in-memory computing array.

(7) A peripheral computing logic unit, configured to receive the SAs read signal and perform left shift, right shift, and high bit fill operations.

(8) A reconfigurable address generation unit module RAGU, comprising an algorithm instruction mapping address RAM and a FIFO memory. The algorithm instruction mapping address RAM is configured to be able to write into the algorithm instruction mapping address table, and to be able to read out the next round of in-memory computing arrays, internal computing logic units and peripheral computing logic addresses, and to send a handshake signal to the FIFO memory. The FIFO memory is configured to receive data signals output by the internal computing logic unit and the peripheral computing logic unit, and to send out the data signals when synchronized with the handshake signal of the algorithm instruction mapping address RAM.

(9) A top-level control module, including a top-level control unit and a timing generation unit. The top-level control unit is configured to parse external command signals and control data signals to flow in the novel in-memory computing architecture. The timing generation unit is configured to adjust the data signal with a clock frequency not higher than a threshold of 150 MHz to meet a read and write access time of t1 = 7ns for writing data to the in-memory computing array and t2 = 10ns for reading data from the in-memory computing array.

As shown in FIG2 , the read-write decoupled 8T SRAM memory cell includes a first inverter formed by an NMOS/PMOS transistor pair M1 and M2, a second inverter formed by an NMOS/PMOS transistor pair M3 and M4, and storage transistors/transmission gates M5, M6, M7, and M8. When the write word line signal WWL 100 is pulled high, the write bit line signal WBL 102 and the write bit line inversion signal WBLb 103 write the data signal. When the read word line signal RWL 101 is pulled high, the read bit line signal RBL 111 and the read bit line inversion signal RBLb112 read the data signal. The read and write operations can be decoupled to prevent read-write interference.

The input value and control signal driving module drives the next round of partial data signal 104 to the internal calculation logic unit, and drives the control signal 105 to the internal calculation logic unit and the peripheral calculation logic circuit. Data with data precision of 8 bits, 16 bits, 32 bits, and 64 bits can be sent to the internal calculation logic unit for calculation. The control signal 105 can control the calculation type of the internal calculation logic unit and the peripheral calculation logic circuit.

As shown in FIG3 , the read word line signal RWL 101 is selected and pulled high to VDD, and the 8T SRAM outputs the data signal to the read bit line signal RBL 111 and the read bit line inversion signal RBLb 112. The data is arranged from the least significant bit LSB to the most significant bit MSB from left to right. The read bit line inversion signal RBLb 112 can be regarded as the inverse code of the read bit line signal RBL 111, and the read bit line signal RBL 111 is added by a chain full adder to obtain the complement code 121 of the read bit line signal RBL 111. The complement code 121 and the read bit line signal RBL 111 are output to the operator selector OP_MUX through the data selector MUX, and participate in the calculation together with the input data 104, and the calculation result 106 is output. MUX and OP_MUX can select the positive code/complement code 121 and the calculation type by the control signal 105.

As shown in FIG. 4 , the input value 104 and the positive code/complement code 121 are passed through a chained full adder to output a 64-bit addition/subtraction result 106 .

As shown in Fig. 5, the peripheral calculation logic circuit inputs the sense amplifier group SAs to read out the data signal 107, and outputs the shift and fill result 108 via the operator selector OP_MUX. The OP_MUX can select the calculation type by the control signal 105.

As shown in Figure 6, the reconfigurable address generation unit module includes an algorithm instruction mapping address storage RAM and a FIFO memory. The algorithm instruction mapping address storage RAM can be configured to write the data signal of the mapping address table, and a counter is set to count the number of calculation steps. When the calculation starts, the algorithm instruction mapping address storage RAM is configured to the read state to read the mapping address table data. Each time it is read out, the counter is reduced by one and a handshake signal is sent to the FIFO memory. When the counter is zero, it proves that the algorithm calculation is completed and an interrupt signal is output to the top-level control unit. The FIFO memory is configured to receive the in-memory calculation array output data signal 106 and the peripheral calculation logic circuit output data signal 108. When the handshake signal sent by the algorithm instruction mapping address storage RAM is received, the next round of data signals are sent synchronously.

The algorithm instruction mapping address table compilation process is shown in Figure 7. Taking a CSR-scalar SpMV algorithm as an example, the pseudo code is converted into ANSI C language program code, and then the high-level language program code is converted into an assembly language. For example, the assembly language of the RISCV64 instruction set architecture is compiled and deleted according to the assembly language, and redundant instructions are removed, such as the lw load word instruction, and converted into a mapping address table with a row of 32 bits wide. The content in the table includes the next round of calculation addresses in the in-memory calculation array, the internal calculation logic unit and the peripheral calculation logic circuit, and the selection of internal calculation logic unit or peripheral calculation logic circuit signals.

The operation timing diagrams of the reconfigurable address generation unit module in writing the mapping address table data and participating in the in-memory computing array calculation process are shown in Figures 8 and 9.

Figure 8 shows a 7-cycle operation, where the input includes the clock signal CLK, the write enable signal WR_EN, the read enable signal RD_EN, the mapping address table input address signal ADDR[31:0], and the mapping address table input data signal RAM DATA Input[31:0]. When the write enable is pulled high, the write operation is valid, the mapping address table input address signal selects the RAM address, and the mapping address table input data signal is written sequentially.

FIG9 shows a 14-cycle operation, where the input includes the clock signal CLK, the write enable signal WR_EN, the read enable signal RD_EN, the mapping address table output data signal RAM DATA Output[31:0], the handshake signal Valid, and the FIFO memory output signal FIFO DATA Output[63:0]. When the read enable is pulled high, the read operation is valid, and the mapping address table data signal is output sequentially with a delay of one clock cycle, and the handshake pulse signal is triggered at the same time. After detecting the rising edge of the handshake signal, the FIFO memory data output signal is output synchronously.

The workflow of the novel in-memory computing architecture described in the present invention is shown in Figure 10. In the first step, the reconfigurable address generation unit module writes the mapping address table Address LUT, and the read-write counter records the number of LUT rows. In the second step, the in-memory computing array stores the initial data into the SRAM storage unit and outputs part of the data to the RAGU. In the third step, the reconfigurable address generation unit module outputs the first calculation type and the data involved in the calculation to the internal calculation logic unit or the peripheral calculation logic circuit for calculation. The in-memory calculation array completes the first calculation and sends the intermediate calculation results to the reconfigurable address generation unit module. In the fourth step, the counter in the reconfigurable address generation unit module is reduced by one, and at the same time, it is determined whether the counter is 0. If it is, it means that the calculation is completed and the result is output. Otherwise, it means that the next round of calculation will continue until the calculation is completed.

The data flow and control flow of the new in-memory computing architecture described in the present invention are shown in Figure 11, which shows in detail the process of data input, calculation, and result output of any algorithm. The architecture first receives instructions issued by the central processing unit and external data transferred from other RAMs, decodes the received instructions through the control circuit, and passes the control signal to the subsequent modules. External initial data, such as mapping address table data and algorithm-related initial data, are passed to the RAGU module and CIMA respectively through the data path. When CIMA starts calculating according to the instructions of the mapping address table, the intermediate value calculated by CIMA is passed to RAGU, and the new address of the next round given by the mapping address table is allocated to CIMA for calculation until the calculation is completed, RAGU sends an interrupt signal, CIMA outputs the calculation result, and the calculation process of the entire algorithm ends, waiting for the next calculation to start.

For those skilled in the art, various other corresponding changes and deformations can be made according to the technical solutions and concepts described above, and all of these changes and deformations should fall within the protection scope of the claims of the present invention.

Claims (6)

1. A novel in-memory computing architecture supporting multiple workloads, comprising:

the in-memory computing array is used for storing input data and performing data computation according to operator selection control signals;

the precharge module is used for charging and pulling up the write bit line signal and the write bit line inversion signal to VDD when the in-memory computing array enters a read phase;

The read word line and write word line driving module is used for providing read word line signals and write word line signals of the in-memory computing array;

The input value and control signal driving module is used for providing partial data signals of the memory computing array, operator selection control signals and operator selection control signals of peripheral computing logic in the next round;

A bit line driving module for writing a driving signal of the bit line and an inversion signal thereof;

the sense amplifier group is used for reading out the data stored in the memory computing array;

The peripheral calculation logic module is used for performing left shift, right shift or high-order filling operation on the read data of the sense amplifier group;

The reconfigurable address generation unit module is used for reading out the logic address of each operation module of the next round and sending out the data signal when the algorithm instruction mapping address is synchronous with the handshake signal;

The top layer control module is used for analyzing external instruction signals, controlling data signals to circulate in the novel memory computing architecture and adjusting read-write access time for the data signals;

the in-memory computing array comprises a plurality of SRAM memory cells arranged in a determinant manner and internal computing logic units arranged among each row of SRAM memory cells;

the SRAM storage unit is used for storing the write-in data signal and the partial data signal calculated in the next round;

The internal calculation logic unit is used for storing partial data signals of the next round of calculation and reading the data signals of the SRAM storage unit to participate in the addition, the subtraction, the multiplication or the Boolean logic operation;

the reconfigurable address generation unit module comprises an algorithm instruction mapping address RAM and a FIFO memory;

The algorithm instruction mapping address RAM can be written into an algorithm instruction mapping address table, can read out a calculation array, an internal calculation logic unit and a peripheral calculation logic address in the next round of memory, and simultaneously sends out handshake signals to the FIFO memory;

the FIFO memory is used for receiving data signals output by the internal computing logic unit and the peripheral computing logic unit, and sending out the data signals when synchronizing with the handshake signals of the algorithm instruction mapping address RAM.

2. The novel in-memory computing architecture supporting multiple workloads as claimed in claim 1, wherein said top-level control module comprises a top-level control unit and a timing generation unit;

The top control unit is used for analyzing external instruction signals and controlling data signals to circulate in the novel in-memory computing architecture;

the timing generation unit is used for adjusting the data signal which is not higher than the clock frequency threshold value to write data into the memory to calculate an array at time t1, and calculating the read-write access time of the data read out from the memory by the array at time t 2.

3. The novel in-memory computing architecture supporting multiple workloads of claim 2, wherein the clock frequency threshold is 150 MHz.

4. The novel in-memory computing architecture supporting multiple workloads of claim 2, wherein the time t1 is less than the time t2.

5. The novel in-memory computing architecture supporting multiple workloads of claim 4, wherein the time t1 is 7ns.

6. The novel in-memory computing architecture supporting multiple workloads of claim 5, wherein the time t2 is 10ns.