CN115936064A - Neural network acceleration array based on weight cycle data stream - Google Patents

Abstract

The present invention specifically relates to a neural network acceleration array based on weight cyclic data flow, which fully reuses the weight value read from memory and input feature map data, greatly reduces the access to external memory, and belongs to the hardware acceleration of neural network technology field. In the field of artificial intelligence chips, the convolution operation occupies more than 90% of the calculation amount of the entire convolutional neural network model. With , a weighted recurrent data flow is proposed. By designing a PE array based on the weighted cyclic data flow, the convolution operation is optimized to effectively reduce the power consumption and delay of the hardware acceleration structure, thereby improving the overall performance of the system.

Description

A Neural Network Acceleration Array Based on Weighted Circular Data Flow

technical field

The invention relates to the technical field of hardware acceleration of a neural network, in particular to a design method of a neural network acceleration array based on a weight cycle data flow.

Background technique

With the rapid development of the Internet of Things technology, wearable smart products use various parts of the human body as the interface of the Internet, truly realize the human-machine integrated product experience with the characteristics of miniaturization, portability, and intelligent wearable products, and provide consumers with portable real-time Information collection and data services have greater technical content and market appeal.

Unfortunately, designing high-performance wearable computing devices is not an easy task, and its implementation faces many challenges. The field is the intersection of different fields of study such as computer science and engineering, using various technologies such as microelectronics and wireless communication. Advances in microelectronics have led to small, low-power artificial intelligence chips suitable for wearable computing devices.

In the field of artificial intelligence chips, deep learning, which belongs to the category of machine learning, is widely used in image classification, speech recognition, object detection, etc., and has achieved remarkable results. Convolutional Neural Networks, Recurrent Neural Networks, and Deep Belief Networks are the main focus of deep learning research, with Convolutional Neural Networks being the most advanced.

Typical convolutional neural network structures include: convolutional layers, activation layers, pooling layers, fully connected layers, and input and output feature maps. Among them, the role of the convolutional layer is feature extraction, the role of the pooling layer is pixel compression, and the role of the fully connected layer is classification. Convolutional layers are computation-intensive operations, while fully-connected layers are data-intensive operations.

Convolutional neural networks have three bottlenecks in data processing: one is data-intensive, and the amount of data to be processed is extremely large. The second is computing-intensive, which consumes a lot of computing resources and a lot of time for data processing and storage. The third is the speed mismatch problem, that is, the data processing speed is slower than the data generation speed. Therefore, the development of dedicated chips suitable for artificial intelligence architecture is urgently needed, and the realization of high-speed data transmission and high-speed computing neural network accelerators will be of great significance in many aspects.

At present, due to the improvement of hardware performance, in the process of accelerating neural network training and inference, it is mainly in the form of CPU, GPU, FPGA and ASIC. About twenty years ago, the CPU was the mainstream for implementing neural network algorithms, and its optimization areas were mainly focused on the software part. The ever-increasing computational cost of CNNs necessitates hardware acceleration of their inference process. On the GPU side, GPU clusters can accelerate very large networks with more than 1 billion parameters in parallel. The mainstream GPU clustering neural network usually uses the distributed SGD algorithm. Many studies have further exploited this parallelism in an effort to enable communication between different clusters. FPGAs are a good platform for hardware acceleration of CNNs due to their many attractive features. In general, FPGAs offer higher energy efficiency than CPUs and GPUs, and higher performance than CPUs. Compared with GPU, the throughput of FPGA is tens of gigaflops, and the memory access is limited. In addition, it does not natively support floating-point calculations, but has lower power consumption. ASIC is a special-purpose processor designed for specific applications. Although ASIC has low flexibility, long development cycle and high cost, it has the advantages of small size, low power consumption, fast calculation speed and high reliability.

There are two typical architectures of neural network hardware accelerators: time domain computing architecture (tree structure) and space domain computing architecture (PE array structure). The tree structure centrally controls the arithmetic calculation unit and storage resources based on the instruction flow, and each arithmetic logic unit obtains the operation data from the centralized storage system and writes back the result to it. It consists of a multiply-add tree, a buffer for allocating input values, and a prefetch buffer. PE array structure, each arithmetic operation unit has a local memory, the entire architecture adopts data flow control, that is, all PE units form a processing chain relationship, and data is directly transmitted between PEs. It consists of global buffer, FIFO and PE array. Each PE consists of one or more multipliers and adders, enabling highly parallel computing.

The neural network hardware accelerator has four typical data flow modes: no local multiplexing data flow, input fixed flow, output fixed flow and weight fixed flow. For data streams without local multiplexing, in order to maximize storage capacity and minimize off-chip memory bandwidth, instead of allocating local storage to the PE, all regions are allocated to the global buffer to increase its capacity, which must multiplex the input Feature map, unicast the convolution kernel weights, and then accumulate the partial sums through the PE array. For the input fixed stream, the calculation core reads the input feature map into the local input register; the calculation core fully reuses these input data, and updates all relevant output partial sums in the output buffer; the updated output partial sum will be written back to the output buffer . For the output fixed flow, the calculation core reads each channel of the input feature map into the local input register; the output part stored in the output register of the calculation core is fully multiplexed to complete the full accumulation in the direction of the three-dimensional convolution channel; the final The output feature maps are written to the output cache after pooling. For the weight fixed stream, the calculation core reads the input feature map block to the local input register; the calculation core uses these input data to update the output part of the block; the weight of the block convolution kernel stored in the weight cache is fully reused , to update the output partial sum stored in the output buffer.

Figure 1 is a schematic diagram of a simple convolutional layer, where 10 is a 7×7 input feature map, 11 is a 3×3 convolution kernel, and 12 is a 5×5 output feature map. The convolution kernel window slides on 10 line by line in a "Z" shape to perform convolution operation, and the result 12 is obtained. Convolution operations occupy more than 90% of the calculation of the entire convolutional neural network model. Therefore, by designing a structured PE array and optimizing the convolution operation, the area and power of the hardware acceleration structure can be effectively reduced. consumption, thereby improving the overall performance of the system.

Contents of the invention

In view of the above background content, the present invention proposes an accelerated array design method for neural network convolution layer operations. It can be applied to the design of FPGA or ASIC neural network hardware accelerator as the computing part of AI acceleration processor.

In the airspace computing structure, each computing unit is controlled through data flow, so the key is to solve the problem of data flow. In order to reduce repeated calls and movements of input data and maximize data reuse, it is best to input feature maps at one time, so weight ring dataflow (WR) is proposed.

The present invention builds a new neural network PE array architecture based on the proposed WR data flow. Assuming that the size of the convolution kernel of a certain convolutional layer is K ² , the size of the corresponding PE array is also K ² . PE units are horizontally interconnected for data movement and vertically cyclically interconnected for weight cycling.

The advantages of the present invention mainly include: the weight value read from the memory and the input feature map data are fully reused, the access to the external memory is greatly reduced, and the overall power consumption and delay are reduced.

Description of drawings

Figure 1 is a schematic diagram of a convolutional layer;

Fig. 2 is a schematic diagram of weight cycle data flow;

Fig. 3 is a schematic diagram of a neural network acceleration array based on a weighted cyclic data flow;

Figure 4 is a workflow diagram of the PE array of the weight cycle data flow

Detailed ways

The weight cycle data flow and corresponding array hardware implementation of the present invention will be described in detail below in conjunction with the accompanying drawings:

The specific embodiment of the weight cycle data flow is shown in Figure 2. 20 is the input feature map with a size of N ^2. Here, it is assumed that N is 7, and the convolution kernel 201 of the first cycle has a size of K ² . Here, it is assumed that K is 3 , the number of sliding rows ranges from 1 to K rows. In the convolution kernel 202 of the next cycle, the number of sliding rows ranges from 2 to K+1 rows, and so on for the convolution kernel of the cycle 203 and the convolution kernel of 204;

From the operation law of the convolutional layer in Figure 1, it can be seen that the data in the 11 window only updates one line of data after each new line. That is, only one row of the original K rows of data is discarded. Therefore, if all the data is updated after each line change, the amount of data access will be greatly increased. Therefore, keep the position of the inherent data row unchanged, and just cover the discarded data row with the update row.

The neural network acceleration array based on the weighted cyclic data flow is shown in FIG. 3 , the array containing ^K2 PE units 301 is a simulation of the convolution kernel window sliding row by row on the image. First, the weight value on the convolution kernel is preset into the PE array 30. When K lines of image lines are simultaneously moved into the array for convolution, it is originally called the weight fixed flow. However, the WR flow proposed in this study is slightly different. It connects the weights of each column and provides a path for the flow of weights, that is, the vertical loop line in 30.

For the PE array workflow shown in Figure 4, the PE array works as follows:

1) First input the 1 to K lines of the image line at the same time, until the input feature map data is filled with 30 for the first time, move the image line to the right as a whole according to the step size, and perform convolution operations in turn;

2) After the traversal of the input feature map of rows 1 to K is completed, an image row is updated. The rest of the K-1 row data that has not been updated continue to be re-entered;

3) When the image row of each cycle of 31 is updated, the weight of 30 will be moved down by the overall cycle of the row, stored in a single weight register of 301 for each row, and the image row of K rows of 30 will be shifted to the right as a whole for convolution ;

4) When the image rows are updated, 30 completes the convolution of the last K rows of the input feature map, and the operation ends.

A single 301 is mainly composed of multipliers, adders and weight registers. The preset weights are first input into the weight registers for storage, and are continuously reused until the image row is changed, and then are moved down to the weight registers of 301 in the next row respectively.

Before the input feature map data fills 30 for the first time, the input excitation does not pass through the multiplier, but is directly output to the forward transmission excitation signal, and the data is transmitted to 301 on the right.

When the input feature map data is full of 30, after the convolution operation is started, the input excitation and weight are input into the multiplier to generate a partial sum. This partial sum is accumulated with the input partial sum signal from the left to generate the forward partial sum, which is input to 301 on the right. When the partial sum of each row reaches the far right of 30, it is summarized by K-1 edge adders to obtain the final convolution result.

In addition to the above structure, the adder part can also be replaced with an addition tree structure.

The above descriptions are only preferred embodiments of the present invention, and do not limit the present invention in any form. Any person familiar with the art, without departing from the scope of the technical solution of the present invention, can use the methods and technical content proposed above to make some possible changes and modifications to the technical solution of the present invention, or modify it to be equivalent to equivalent changes Example. Therefore, any simple modifications, equivalent changes and modifications made to the above embodiments according to the technical essence of the present invention, which do not deviate from the technical solution of the present invention, still fall within the protection scope of the technical solution of the present invention.

Claims (9)

1. A neural network acceleration array based on weight cyclic data flow, characterized in that the PE array size is the size of the convolution window, PE units are horizontally interconnected for data movement, and vertically cyclically interconnected for weight circulation. 2. The method according to claim 1, characterized in that the data flow in the array obeys the flow rules of the weighted cycle data flow. 3. The working condition of the neural network acceleration array according to claim 2, wherein, at first, input 1 to K lines of the image line at the same time, until the input feature map data fills the PE array for the first time, and move the image line to the right as a whole according to the step size , perform convolution operations in turn; when the traversal of the input feature map of rows 1 to K is completed, update a row of image rows; the rest of the unupdated K-1 rows of data continue to be re-circulated and input; when the image rows are updated in each cycle, PE The weight of the array is shifted down by row as a whole, stored in the weight register of a single PE unit in each row, and the K rows of image rows in the PE array are shifted to the right as a whole for convolution; when the image row is updated, the PE array completes the input After the convolution of the last K rows of the feature map, the operation ends. 4. The method according to claim 3, characterized in that, during the process of shifting the input feature map to the right, the position of the weight in the array remains unchanged, and the array performs a convolution once for each bit shifted to the right as a whole. 5. The method according to claim 4, characterized in that, except that K rows of image rows enter at the same time at the beginning, the update interval of the image row is a cycle after the PE array traverses the convolution K rows, instead of every Just shoot once. 6 . The method according to claim 5 , wherein the update of the image line in each period moves along with the last line of the convolution kernel, and is updated sequentially according to the gradient. 7. The method according to claim 6, characterized in that the convolution operation is not performed as the input data moves to the right, but waits until it reaches the corresponding weight before performing the convolution operation. 8. The PE unit according to claim 1, characterized in that it is composed of at least one data register, a multiplier and multiple adders, and may also have structures such as FIFO. 9. The PE unit according to claim 8, characterized in that, in addition to the data transmission signals, there are control signals controlling the flow of data.

Images