CN109635944B - A sparse convolutional neural network accelerator and implementation method - Google Patents

Abstract

A sparse convolutional neural network accelerator and implementation method, the connection weight of a sparse network in an off-chip DRAM is read into a weight input buffer, decoded by a weight decoding unit, and stored in the weight on-chip global buffer; The input buffer is read into the input buffer of the neuron, and then the read neuron is decoded by the neuron decoding unit and stored in the global buffer on the neuron chip; the calculation mode of the PE computing unit array is determined according to the configuration parameters of the current layer of the neural network, Send the decoded neurons and connection weights to the PE computing unit; calculate the product of neurons and connection weights; in the accelerator of the present invention, the multipliers in the PE unit are all replaced by shifters, and all basic modules are It can be configured according to network computing and hardware resources, so it has the advantages of high speed, low power consumption, small resource occupation and high data utilization.

Description

A sparse convolutional neural network accelerator and implementation method

technical field

The invention belongs to the technical field of deep neural network accelerated computing, and relates to a sparse convolutional neural network accelerator and an implementation method.

Background technique

The remarkable performance of Deep Neural Network (DNN) comes from its ability to use statistical learning to obtain an efficient representation of the input space from a large amount of data, and to extract high-level features from the raw data. However, these algorithms are computationally intensive. As the number of network layers continues to increase, DNNs require higher and higher storage and computing resources, making it difficult to deploy them on embedded devices with limited hardware resources and tight power budgets. DNN model compression technology is conducive to reducing the memory usage of network models, reducing computational complexity and system power consumption. Models are deployed to embedded systems.

Compared with the von Neumann computer architecture, neural network computers use neurons as the basic computing and storage units to construct a new type of distributed storage and computing architecture. Field Programmable Gate Array (FPGA) not only has the programmability and flexibility of software, but also has the characteristics of high throughput and low latency of Application Specific Integrated Circuit (ASIC), which has become the current DNN hardware Popular vehicle for accelerator design applications. Most of the existing neural network model research focuses on the improvement of network recognition accuracy. Even the lightweight network model ignores the computational complexity of hardware acceleration, and the floating-point data representation leads to excessive consumption of hardware resources. Although there are many DNN accelerator solutions, there are still many problems that need to be solved or optimized, such as more efficient data multiplexing, pipeline design of hardware computing, and on-chip storage resource planning.

SUMMARY OF THE INVENTION

In order to overcome the above-mentioned deficiencies of the prior art, the purpose of the present invention is to propose an energy-efficient sparse convolutional neural network accelerator and an implementation method, so that the DNN connection weight has the characteristics of sparseness and hardware calculation friendliness.

To achieve the above object, the present invention is achieved through the following technical solutions:

A sparse convolutional neural network accelerator, comprising: off-chip DRAM, neuron input buffer, neuron decoding unit, neuron encoding unit, neuron output buffer, neuron on-chip global buffer, weight input buffer, Weight decoding unit, weight on-chip global buffer, PE computing unit array, activation unit and pooling unit; among them,

Off-chip DRAM for storing compressed neuron and connection weight data;

The neuron input buffer is used to buffer the compressed neuron data read from the off-chip DRAM and transmit it to the neuron decoding unit;

The neuron decoding unit is used to decode the compressed neuron data and transmit the decoded neuron to the neuron on-chip global buffer;

The neuron coding unit is used to compress and encode the data in the global buffer on the neuron chip and transmit it to the neuron output buffer;

Neuron output buffer, used to store the received compressed neuron data in off-chip DRAM;

The neuron on-chip global buffer is used to cache the decoded neurons and the intermediate results calculated by the PE array, and communicate data with the PE computing unit array;

The activation unit is used to activate the result calculated by the PE computing unit array and transmit it to the pooling unit or the global buffer on the neuron chip; the pooling unit is used to downsample the result after the activation of the activation unit;

The weight input buffer is used to cache the compressed connection weight data read from the off-chip DRAM and transmit it to the weight decoding unit;

The weight decoding unit is used to decode the compressed connection weight data, and transmit the decoded connection weight to the weight on-chip global buffer;

On-chip global buffer for weights, used to cache the decoded connection weights and communicate data with the PE computing unit array;

The PE computing unit array is used for computing in the convolution computing mode or the fully connected computing mode.

An implementation method of a sparse convolutional neural network accelerator. After preprocessing data, it is stored in DRAM, and the connection weight of the sparse network stored in the off-chip DRAM is read into the weight input buffer, and the weight decoding unit is passed through the weight decoding unit. After decoding, it is stored in the weight on-chip global buffer; the neurons stored in the off-chip DRAM are read into the neuron input buffer, and then the read neurons are decoded by the neuron decoding unit and stored on the neuron slice Global buffer; determine the calculation mode of the PE computing unit array according to the configuration parameters of the current layer of the neural network, and send the neurons and connection weights arranged after decoding to the corresponding PE computing units; the neurons and connection weight data enter PE separately Cached in Neuron_Buffer and Weight_Buffer. When Neuron_Buffer and Weight_Buffer are filled for the first time, the product of neuron and connection weight starts to be calculated; among them, Neuron_Buffer is the neuron buffer in PE, and Weight_Buffer is the weight buffer in PE;

If the result calculated by the PE computing unit array is an intermediate result, it will be temporarily stored in the global buffer on the neuron chip, and wait for the next accumulation with other intermediate results calculated by the PE computing unit array; if it is the final result of the convolution layer, then Send it to the activation unit, then send the activation result to the pooling unit, downsample and then save it back to the neuron on-chip global buffer; if it is the final result of the fully connected layer, it is sent to the activation unit, and then directly stored back to the neuron on-chip global buffer Buffer; the data stored in the global buffer on the neuron chip is compressed and encoded by the neuron coding unit according to the sparse matrix compression coding storage format, and then sent to the neuron output buffer, and then stored back to the off-chip DRAM, waiting for the next layer of calculation. Read in; loop in this way until the fully connected layer of the last layer outputs a one-dimensional vector whose length is the number of classification categories.

A further improvement of the present invention is that the specific process of preprocessing the data is:

1), prune the connection weights to make the neural network sparse;

2), 2 ⁿ connection weight clustering: First, according to the distribution of network parameters after pruning, select 2 ⁿ cluster centers in the form of quantized bit width within the range of the parameter set, and calculate the remaining non-zero parameters and cluster centers. Euclidean distance, which clusters non-zero parameters to the cluster center with the shortest Euclidean distance, where n is a negative integer, positive integer or zero;

3), for the sparse network after the clustering in step 2), the connection weight of the sparse network and the feature map neurons are compressed and stored in the off-chip DRAM using a sparse matrix compression coding storage format.

A further improvement of the present invention is that the sparse matrix compression coding storage format is specifically: the coding of the convolutional layer feature map and the fully connected layer neuron is divided into three parts, wherein each 5-bit represents the current non-zero neuron and the previous one. The number of 0s between non-zero neurons, each 16-bit represents the value of the non-zero element itself, and the last 1-bit is used to mark whether the non-zero elements in the feature map of the convolutional layer wrap or not in the fully-connected layer. When changing layers, the entire storage unit encodes three non-zero elements and their relative position information; if a storage unit contains the last non-zero element of a row of the feature map of the convolutional layer or the last non-zero element of an input neuron of a certain layer of the fully connected layer If the element is zero, then the following 5-bit and 16-bit are all filled with 0, and the last 1-bit flag is set to 1; the encoding of the connection weight is also divided into three parts, where each 4-bit represents the current non-zero connection The number of 0s between the weight and the previous non-zero connection weight, each 5-bit represents the non-zero connection weight index, and the last 1-bit is used to mark whether the non-zero connection weight in the convolutional layer corresponds to the same input feature map Or whether the non-zero connection weights in the fully connected layer correspond to the same output neuron, and the entire storage unit encodes seven non-zero connection weights and their relative position information; similarly, if a storage unit contains the same input in the convolutional layer The last non-zero connection weight of the feature map or the last non-zero connection weight corresponding to the same output neuron in a layer of the fully connected layer, then all the following 4-bit and 5-bit are filled with 0, and the last 1 -bit flag bit 1.

A further improvement of the present invention is that, when the calculation mode of the PE calculation unit array is convolution calculation, if the size of the convolution window is k×k, and the sliding step size of the convolution window is s, the filled Neuron_Buffer and Weight_Buffer will be calculated according to the first input. The first-out principle stores neurons 1~k and connection weights 1~k from first to last;

If the sliding step s of the convolution window is equal to 1, the neuron 1 is directly discarded after the product of the neuron 1 and the connection weight 1 is calculated in the first clock cycle, and the connection weight 1 is transferred to the end of the Weight_Buffer; the second After the clock cycle reads in the new neuron k+1 and calculates the product of neuron 2 and connection weight 2, dumps both neuron 2 and connection weight 2 to the end of their respective Buffers; the third clock cycle calculates neuron 3 After multiplying with connection weight 3, transfer neuron 3 and connection weight 3 to the end of their respective Buffers; and so on, until the kth clock cycle PE unit calculates and completes the multiplication and accumulation of one convolution window and one row, and outputs the result At the same time, adjust the order of the values in the Buffer according to the step size s, and wait for the calculation of the next convolution window;

If the step size s is greater than 1, after calculating the product of neuron 1 and connection weight 1 in the first clock cycle, neuron 1 is directly discarded, and the connection weight 1 is transferred to the end of Weight_Buffer; the second clock cycle reads Enter a new neuron k+1, calculate the product of neuron 2 and connection weight 2, and discard neuron 2 directly, and transfer connection weight 2 to the end of Weight_Buffer; and so on, until the sth clock cycle, read Enter the new neuron k+s-1, calculate the product of the neuron s and the connection weight s, and discard the neuron s directly, and transfer the connection weight s to the end of the Weight_Buffer; read in the s+1th clock cycle After the new neuron k+s calculates the product of the neuron s+1 and the connection weight s+1 at the same time, the neuron s+1 and the connection weight s+1 are dumped to the end of their respective Buffers; After calculating the product of neuron s+2 and connection weight s+2 for each clock cycle, transfer neuron s+2 and connection weight s+2 to the end of their respective Buffers; and so on, until the kth clock cycle The PE unit calculates the multiplication and accumulation of one line of convolution window, and adjusts the order of the values in the Buffer according to the step size s while outputting the result, waiting to calculate the next convolution window.

A further improvement of the present invention is that when the calculation mode of the PE calculation unit array is full connection calculation, if the number of input neurons calculated by each PE unit is m, then Neuron_Buffer and Weight_Buffer are filled up according to the principle of first-in, first-out. The neuron 1~m and the connection weight 1~m are stored first, then the connection weight 1~m is stored; the first clock cycle calculates the product of the neuron 1 and the connection weight 1, then directly discards the connection weight 1, and transfers the neuron 1 to the end of Neuron_Buffer ; The second clock cycle reads in the new connection weight m+1 and calculates the product of neuron 2 and connection weight 2, directly discards connection weight 2, and transfers neuron 2 to the end of Neuron_Buffer; and so on, Until the mth clock cycle, the PE unit calculates the intermediate result of an output neuron, adjusts the order of the values in the Buffer while outputting the result, and waits for the calculation of the next group of output neurons.

Compared with the prior art, the present invention has the following beneficial effects:

The invention adopts a sparse matrix compression coding storage format, so that the data of different layers of the neural network has a unified coding format, and considering the respective computing characteristics of the convolution layer and the fully connected layer in the neural network, the storage resources are saved to the greatest extent. At the same time, the complexity of the codec circuit in hardware design is simplified. The present invention also makes a corresponding optimization design for the data flow buffered by neurons and connection weights in the PE unit, and proposes a high data multiplexing rate in-PE data flow mode, which reduces storage space and makes computation more efficient. The invention has flexible operation, simple encoding and decoding, and good compression effect. Each input neuron is represented by 16-bit fixed point, and the connection weight is represented by 5-bit.

Further, after the data preprocessing process in the present invention, the remaining non-zero connection weights of the neural network all become 2 ⁿ or 2 ⁿ combinations, therefore, in the present invention, the multiplier in the PE computing unit is optimized to be more economical. Shifter for resources.

Further, the present invention includes three main steps of pruning, ²ⁿ clustering and compression coding. Compared with the original network model, the compressed LeNet, AlexNet, and VGG reduce the storage consumption by about 222 times, 22 times, and 68 times, respectively. In the accelerator of the present invention, the multipliers in the PE unit are all replaced by shifters, and all basic modules can be configured according to network computing and hardware resources, so it has the advantages of high speed, low power consumption, small resource occupation and data utilization. advantage of high rate.

Description of drawings

Figure 1 is the overall flow chart of neural network model preprocessing.

Figure 2 is a flow chart of 2 ⁿ connection weight clustering.

Figure 3 shows the compressed encoding storage format.

FIG. 4 is an overall architecture diagram of the neural network hardware accelerator of the present invention.

Figure 5 is a schematic diagram when the PE array is configured in the convolution calculation mode.

FIG. 6 is a schematic diagram of data flow allocation when the PE array is configured in the convolution calculation mode.

FIG. 7 is a schematic diagram when the PE array is configured in the fully connected computing mode.

FIG. 8 is a schematic diagram of data flow allocation when the PE array is configured in the fully connected computing mode.

FIG. 9 is a schematic structural diagram of the internal structure of the PE unit and the basic column unit of the PE array.

Figure 10a shows the data flow of neurons and connection weights in PE during convolution calculation.

Figure 10b shows the data flow of neurons and connection weights in PE during full connection calculation.

Figure 11 is a configurable design diagram of the present invention.

Detailed ways

The technical solutions of the present application will be clearly and completely described below with reference to the accompanying drawings.

The connection weight in the present invention is also called a weight.

Referring to FIG. 4, the neural network accelerator of the present invention includes: off-chip DRAM, neuron input buffer, neuron decoding unit, neuron encoding unit, neuron output buffer, weight input buffer, weight decoding unit, neural network On-chip global buffer, weight on-chip global buffer, PE computing unit array, activation unit and pooling unit.

Off-chip DRAM for storing compressed neuron and connection weight data.

The neuron input buffer is used to buffer the compressed neuron data read from the off-chip DRAM and transmit it to the neuron decoding unit.

The neuron decoding unit is used to decode the compressed neuron data and transmit the decoded neuron to the neuron on-chip global buffer.

The weight input buffer is used to buffer the compressed connection weight data read from the off-chip DRAM and transmit it to the weight decoding unit.

The weight decoding unit is used to decode the compressed connection weight data, and transmit the decoded connection weight to the weight on-chip global buffer.

The neuron coding unit is used to compress and encode the calculated data in the neuron on-chip global buffer and transmit it to the neuron output buffer.

The neuron output buffer is used to store the received compressed neuron data into off-chip DRAM.

The neuron on-chip global buffer is used to cache the decoded neuron and the intermediate result calculated by the PE array, and communicate data with the PE computing unit array.

The weight on-chip global buffer is used to cache the decoded connection weight and communicate data with the PE computing unit array.

The activation unit is used to activate the result calculated by the PE computing unit array and transmit it to the pooling unit or the global buffer on the neuron slice. Common activation functions include Sigmoid, Tanh, ReLU, etc. Among them, the Sigmoid function is the most commonly used in typical convolutional neural networks, and the ReLU function is currently the most used activation function, because it allows the training results to converge quickly, and A lot of zero values are output. In addition, the ReLu function does not need to be calculated. It only needs to compare the input data with zero, which saves a lot of computing resources than using other activation functions. The design of the relevant hardware modules is also relatively simple. A comparator is needed to achieve this. For the sigmoid activation function, the present invention selects the method of piecewise linear approximation to realize the hardware module design of the sigmoid function. The basic idea of piecewise linear approximation is to approximate the activation function itself with a series of polyline segments. During specific implementation, the parameters k and b of each broken line can be stored in the look-up table, and the value between the breakpoints can be calculated by substituting the corresponding parameters k and b according to formula (1).

y=kx+b (1)

It should be noted that the sigmoid function is symmetric about the center of the (0,0.5) point, so it is only necessary to store the fitted line segment parameters of the positive semi-axis of the x-axis, and then the negative semi-axis of the x-axis can be calculated by 1-f(x). the function value. Since the value of f(x) is very close to 1 when x>8, only the part of x from 0 to 8 is fitted in this design invention, and the output of the activation function is all 1 when x is greater than 8. First, the input x value is compared with the segmented polyline x interval to obtain the corresponding interval index, then the calculation parameters k and b of the corresponding line segment are obtained from the lookup table according to the index, and finally the final activation output result is calculated by the multiplier and the adder. .

The pooling unit is used to downsample the result after the activation unit is activated. In a convolutional neural network, usually after the activation function activates the convolution calculation result, it will enter the pooling layer for downsampling. Common pooling methods include average pooling, max pooling, and spatial pyramid pooling. The present invention designs hardware for the most common max pooling and average pooling. The maximum pooling only needs to send all the convolution results that need to be pooled into the comparator for comparison, and the maximum value obtained is the maximum pooling output result. In the calculation of average pooling, all convolution results that need to be pooled are first sent to the adder for accumulation, and then the accumulated result is divided by the number of all input convolution values to obtain the average pooling output result. In average pooling, the most common pooling window size is generally 2. When designing hardware circuits, the division operation in the last step of average pooling can actually be replaced by a shift operation, that is, when the pooling window size is 2 , just shift the accumulated result to the right by two digits.

The PE computing unit array can be configured in convolution computing mode or fully connected computing mode. The PE computing unit array includes several PE units, and each column of PE units is called a basic column unit. The main input port of each PE unit is the input neuron (16bits) and the connection weight (5bits) corresponding to the input neuron. After these two kinds of data enter the PE unit, they are buffered in Neuron_Buffer (neuron buffer in PE) and Weight_Buffer respectively. In (weight buffer in PE), when Neuron_Buffer and Weight_Buffer are filled for the first time, the product of neuron and connection weight starts to be calculated. After the data preprocessing process in the present invention, the remaining non-zero connection weights of the neural network are all changed to 2 ⁿ or 2 ⁿ combinations. Therefore, in the present invention, the multiplier in the PE computing unit is optimized to be a more resource-saving shifter. positioner.

Referring to Figure 9, the neuron and connection weights are read from Neuron_Buffer and Weight_Buffer respectively. First, determine whether the neuron and connection weights are 0. If one of the two data is 0 or both are 0, the enable signal will be calculated. en is set to 0, otherwise 1. If the enable signal en is 0, no shift operation is performed, and the result of the shift calculation is directly assigned to 0 and sent to the adder for accumulation. Count_out is used to count the number of shift-accumulation operations calculated by the PE unit. During convolution calculation, a PE unit calculates the shift-accumulation of one row of each convolution window. During full connection calculation, a PE unit calculates an output neuron and a set of Shift-accumulate connected input neurons. The output result calculated by the basic column unit of the PE calculation unit array is stored in the Regfile, and the addition operand is selected by the data selector (Select) and sent to the adder corresponding to the PE unit of the column for accumulation operation, and the corresponding offset value is added. bias, get the final column output result Colunm_out.

Since the data multiplexing and computing parallelism in the convolution calculation mode and the full connection calculation mode are different, the specific data flow methods inside the PE unit for the convolution calculation process and the full connection calculation process are slightly different.

In the process of convolution calculation, the convolutional neural network (CNN) is a forward propagation network structure composed of stacking convolution layers with similar structures. The operations between layers are independent and the operations of each layer are highly similar, so it can be abstracted. Integrate the characteristics of convolution layer calculation to design a general convolution calculation mode of PE calculation unit array. In the present invention, it is selected to split the output feature map in parallel and fuse the input feature map parallel and the same input feature map convolution window in parallel, so that the data on the chip is complex. Maximize it without putting more pressure on the on-chip cache.

In the process of full connection calculation, the present invention compares the calculation characteristics of the full connection layer and the convolution layer in the neural network, and also considers the full connection layer structure, data calculation pipeline and system clock in the convolutional neural network with different structures. After a series of practical situations such as cycle and hardware resources, a parametric and configurable fully connected computing mode of PE computing unit array is designed.

Referring to Figure 11, the size of the convolution kernel, the number of input feature maps, the number of output feature maps, and the number of input neurons and output neurons of different fully connected layers of various convolutional neural network models are different. Similarly, in order to make the operation unit more versatile, the present invention optimizes the configurable design of the neural network hardware accelerator from the convolution calculation mode, the full connection calculation mode, the selection of activation function and the selection of pooling mode. The configurable design of key modules such as PE computational unit arrays, activation units, and pooling units is achieved by parameterizing some key feature sizes of convolutional neural networks.

Table 1 lists the configuration parameters and parameter meanings of the basic units in some key modules when the code is implemented. When calculate_layer is 0, it means that the current PE array needs to calculate the first layer of the convolutional layer, 1 means that the current PE array needs to calculate the middle layer of the convolutional layer, 2 means the current calculation pooling layer, and 3 means that the current PE array needs to calculate the full Connection layer; when connet_mu_or_ad_pe is 0, it means the PE unit performs the multiplication operation, 1 means the PE unit performs the addition operation, and 2 means the PE unit performs the multiply-accumulate operation; when the active_type is 0, it means the Sigmoid activation function is selected, and 1 means the ReLU is selected Activation function; when pooling_type is 0, it means selecting the maximum pooling method, and 1 means selecting the average pooling method. These parameters can be configured in the top-level file of the program. Users can modify the configuration file according to the input and output size, activation function selection, and pooling method selection of different computing layers in the convolutional neural network, and then they can build the network model they want. Quickly generate neural network maps of the appropriate scale and needs on hardware.

Table 1 Configuration parameters and parameter meanings of basic units in some key modules during code implementation

An implementation method based on the above-mentioned sparse convolutional neural network accelerator, comprising the following steps:

Among them, data preprocessing includes the following steps:

Step 1), pruning the connection weights to make the neural network sparse;

Step 2), 2 ⁿ connection weight clustering: First, according to the distribution of network parameters after pruning, select the 2 ⁿ form of cluster centers according to the quantized bit width within the range of the parameter set, and calculate the remaining non-zero parameters and cluster centers. The Euclidean distance of , clusters non-zero parameters to the cluster center with the shortest Euclidean distance from it, where n can be a negative integer, a positive integer or zero;

Step 3), for the clustered sparse network in step 2), the connection weights of the sparse network and the feature map neurons are compressed and stored in the off-chip DRAM using a sparse matrix compression coding storage format. The specific storage format is: the encoding of the feature map of the convolutional layer and the neurons of the fully connected layer is divided into three parts, where each 5-bit represents the number of 0s between the current non-zero neuron and the previous non-zero neuron , each 16-bit represents the value of the non-zero element itself, and the last 1-bit is used to mark whether the non-zero element in the feature map of the convolutional layer wraps the line or whether the non-zero element in the fully connected layer changes the layer. The entire storage unit can encode three Non-zero element and its relative position information. If a storage unit contains the last non-zero element of a certain row of the feature map of the convolutional layer or the last non-zero element of the input neuron of a certain layer of the fully connected layer, the following 5-bit and 16-bit are all filled with 0, and the The last 1-bit flag bit is set to 1. The encoding of the connection weight is also divided into three parts, where each 4-bit represents the number of 0s between the current non-zero connection weight and the previous non-zero connection weight, each 5-bit represents the non-zero connection weight index, and finally The 1-bit is used to mark whether the non-zero connection weight in the convolutional layer corresponds to the same input feature map or whether the non-zero connection weight in the fully connected layer corresponds to the same output neuron. The entire storage unit can encode seven non-zero connection weights and its relative position information. Similarly, if a storage unit contains the last non-zero connection weight corresponding to the same input feature map in the convolutional layer or the last non-zero connection weight corresponding to the same output neuron in a layer of the fully connected layer, then the following 4-bit and 5-bit are all filled with 0, and the last 1-bit flag is set to 1.

Step 4), the connection weight of the sparse network stored in the off-chip DRAM is read into the weight input buffer, decoded by the weight decoding unit, and stored in the weight on-chip global buffer; The neuron reads the neuron input buffer, and then decodes the read neuron through the neuron decoding unit and stores it in the neuron on-chip global buffer; the PE is determined according to the configuration parameters of the current layer of the neural network (see Table 1). The calculation mode of the calculation unit array sends the decoded neurons and connection weights to the corresponding PE calculation unit; these two kinds of data are buffered in Neuron_Buffer and Weight_Buffer respectively after entering PE. When Neuron_Buffer and Weight_Buffer are filled for the first time After that, start calculating the product of neurons and connection weights.

When the calculation mode of the PE computing unit array is configured as convolution calculation, if the size of the convolution window is k×k, and the sliding step size of the convolution window is s, the filled Neuron_Buffer and Weight_Buffer are respectively changed from The neurons 1~k and the connection weights 1~k are stored on a first-come-first-served basis.

If the sliding step s of the convolution window is equal to 1, the neuron 1 is directly discarded after the product of the neuron 1 and the connection weight 1 is calculated in the first clock cycle, and the connection weight 1 is transferred to the end of the Weight_Buffer; the second After the clock cycle reads in the new neuron k+1 and calculates the product of neuron 2 and connection weight 2, dumps both neuron 2 and connection weight 2 to the end of their respective Buffers; the third clock cycle calculates neuron 3 After multiplying with connection weight 3, transfer neuron 3 and connection weight 3 to the end of their respective Buffers; and so on, until the kth clock cycle PE unit calculates and completes the multiplication and accumulation of one convolution window and one row, and outputs the result At the same time, adjust the order of the values in the Buffer according to the step size s, waiting to calculate the next convolution window.

If the step size s is greater than 1, after calculating the product of neuron 1 and connection weight 1 in the first clock cycle, neuron 1 is directly discarded, and the connection weight 1 is transferred to the end of Weight_Buffer; the second clock cycle reads Enter a new neuron k+1, calculate the product of neuron 2 and connection weight 2, and discard neuron 2 directly, and transfer connection weight 2 to the end of Weight_Buffer; and so on, until the sth clock cycle, read Enter the new neuron k+s-1, calculate the product of the neuron s and the connection weight s, and discard the neuron s directly, and transfer the connection weight s to the end of the Weight_Buffer; read in the s+1th clock cycle After the new neuron k+s calculates the product of the neuron s+1 and the connection weight s+1 at the same time, the neuron s+1 and the connection weight s+1 are dumped to the end of their respective Buffers; After calculating the product of neuron s+2 and connection weight s+2 for each clock cycle, transfer neuron s+2 and connection weight s+2 to the end of their respective Buffers; and so on, until the kth clock cycle The PE unit calculates the multiplication and accumulation of one line of convolution window, and adjusts the order of the values in the Buffer according to the step size s while outputting the result, waiting to calculate the next convolution window.

When the computing mode of the PE computing unit array is configured to be fully connected computing, if the number of input neurons calculated by each PE unit is m, the Neuron_Buffer and Weight_Buffer are filled and the neurons are stored from first to last according to the principle of first-in, first-out. Elements 1 to m and connection weights 1 to m. The first clock cycle calculates the product of neuron 1 and connection weight 1, then directly discards connection weight 1, and dumps neuron 1 to the end of Neuron_Buffer; the second clock cycle reads in the new connection weight m+1 and calculates at the same time After the product of neuron 2 and connection weight 2, the connection weight 2 is directly discarded, and neuron 2 is transferred to the end of Neuron_Buffer; and so on, until the mth clock cycle PE unit calculates the intermediate result of an output neuron , adjust the order of the values in the Buffer while outputting the results, and wait for the next set of output neurons to be calculated.

If the result calculated by the PE computing unit array is an intermediate result, it will be temporarily stored in the global buffer on the neuron chip, and wait for the next accumulation with other intermediate results calculated by the PE computing unit array; if it is the final result of the convolution layer, then Send it to the activation unit, then send the activation result to the pooling unit, downsample and then save it back to the neuron on-chip global buffer; if it is the final result of the fully connected layer, it is sent to the activation unit, and then directly stored back to the neuron on-chip global buffer buffer. The data stored in the global buffer on the neuron chip is compressed and encoded by the neuron coding unit according to the sparse matrix compression coding storage format, and then sent to the neuron output buffer, and then stored back to the off-chip DRAM, and read in after the next layer of calculation. This cycle is repeated until the fully connected layer of the last layer outputs a one-dimensional vector whose length is the number of classification categories.

The invention has flexible operation, simple encoding and decoding, and good compression effect. Each input neuron is represented by 16-bit fixed point, and the connection weight is represented by 5-bit.

Figure 1 is the overall flow chart of the preprocessing of the neural network model. The first step is called pruning of connection weights. The connections with smaller connection weights are removed from the trained model, and then the network recognition accuracy is restored to the initial state by retraining. The second step is called 2 ⁿ connection weight clustering, and the clustering operation of fixed cluster centers is performed on the model trained in the first step, and these cluster centers are used to replace all non-zero connection weights; the third step is called connection weights Compression encoding.

FIG. 2 is a specific flow chart of the 2 ⁿ connection weight clustering implemented by the present invention. After obtaining the pruned sparse network, perform training until the recognition accuracy no longer increases. Then, according to the parameter distribution at this time, select N cluster centers in the form of 2 ⁿ , and cluster all the weights to the one with the smallest Euclidean distance. On the clustering center, it is judged whether the network accuracy rate is lost at this time. If there is a loss, continue training until the accuracy rate no longer increases, and so on until the network accuracy rate is not lost after clustering.

Figure 3 shows the sparse matrix compression coding storage format of the present invention.

Figure 4 shows the overall architecture of the neural network hardware accelerator, including off-chip DRAM, neuron input buffer, neuron decoding unit, neuron encoding unit, neuron output buffer, weight input buffer, weight decoding unit , neuron on-chip global buffer, weight on-chip global buffer, PE computing unit array, activation unit and pooling unit.

Figure 5 shows a schematic diagram when the PE array is configured in the convolution calculation mode. The dotted box is a PE unit array with m rows and n columns. Each column of PE units is called a basic column unit and corresponds to an add operand selector. (SEL), used to select whether the calculation result corresponding to a column of PE units is sent to the accumulator for accumulation. All PE units in Figure 5 correspond to the same input feature map, in which the same row of PE units receives neurons in the same row of the input feature map and the same row of convolution kernel connection weights corresponding to different output feature maps, and the same column of PE units respectively receives The neurons in different lines of the input feature map that need convolution calculation and the connection weights of the same convolution kernel in different lines of the corresponding output feature map are finally calculated at the same time to obtain n continuous t neurons in the same position of the output feature map, t= m/k, where k is the size of the convolution kernel.

Figure 6 illustrates the specific data flow distribution method of the PE array in the convolution calculation mode of the present invention. The left side is the convolution kernel corresponding to the first output feature map and the input feature map that needs to be convolution calculated. It is assumed that the convolution kernel Size k=5, PE array size m=n=5. Each row of 5 connection weights corresponding to the convolution kernel of the first output feature map is used as a group, and the 5 groups of connection weights are respectively recorded as W _r1 ~ W _r5 , and these 5 groups of connection weights are sent to the first column of the PE array respectively. That is to say, the five columns of PE units in the example need to receive different convolution kernels corresponding to the five output feature maps respectively. The size of the input feature map is N×N, the neuron data of N lines are denoted as Row ₁ ～ Row _N , the neuron data that needs to be input to the PE unit for each convolution calculation are denoted as N _k1 ～N _k5 , and these 5 groups of input neurons are denoted as N k1 ～ N k5 . Data is sent to PE units in the same row of the PE array, respectively. That is to say, the PE units in the first row share N _k1 neuron data, the PE units in the second row share N _k2 neuron data, and so on. Finally, the 5×5 PE array in the example is calculated to obtain 5 output feature maps at the same time. 1 output neuron value at the same location.

Figure 7 shows a schematic diagram of a general PE array configured in the fully connected computing mode. Inside the dashed box is a PE unit array with m rows and n columns. Each column of PE units is called a basic column unit and corresponds to an add operand selection. The device (SEL) is used to select whether the calculation result corresponding to a column of PE units is sent to the accumulator for accumulation. The present invention finally selects a parallel manner in which all input neurons and output neurons are grouped and recalculated to achieve maximum data reuse. Therefore, all PE units in Figure 7 correspond to different parts of all input neurons, where the same row of PE units receive the same group of input neurons and the connection weights of different output neurons connected to it, and the same column of PE units receive different groups of The input neuron and the connection weights of different output neurons connected to it are finally calculated at the same time to obtain n consecutive output neurons.

Figure 8 illustrates the specific data flow distribution method of the PE array in the fully connected computing mode of the present invention. The left side is a fully connected layer. Assuming that there are 20 input neurons and 10 output neurons, this layer is fully connected The layer connection weight is 200 in total, and the PE array size is m=4 and n=5. All input neurons are divided into 4 groups, each group has 5 input neurons, which are respectively denoted as N ₁ -N ₄ , and each row of the PE array shares the same group of input neurons. That is to say, in the example, 4 rows of PE units need to receive corresponding 4 groups of input neurons, the first row of PE units share N ₁ neuron data, the second row of PE units share N ₂ neuron data, and so on, the mth Row PE units share N _m neuron data. Since the number of columns in the PE array is 5, all the output neurons are divided into two groups, which are respectively denoted as F ₁ and F ₂ , and 5 output neurons are calculated in each group. The connection weights of the input neurons of the N ₁ to N ₄ groups and the first output neuron of the F ₁ group are respectively recorded as W1 ₁₁ to W1 ₁₄ , and the connection weights of the connections to the second output neuron of the F ₁ group are respectively recorded as W1 ₂₁ to W1 ₂₄ , and so on, the connection weights with the fifth output neuron of the F2 group are respectively recorded as W2 ₅₁ to W2 ₅₄ , and there are 40 groups of connection weights, ₅ in each group. The 40 groups of connection weights are respectively sent to the computing units in the PE array. The PE units in the first column receive the connection weights from W1 ₁₁ to W1 ₁₄ respectively, and the PE units in the second column receive the connection weights from W1 ₂₁ to W1 ₂₄ respectively. , and so on, the fifth column of PE units respectively receive the connection weights of W1 ₅₁ to W1 ₅₄ . Finally, the 4×5 PE array in the example is calculated simultaneously to obtain the ₅ output neuron values of the F1 group.

Figure 9 shows the internal structure of a single PE computing unit and the data flow between basic column units consisting of m PE computing units in the PE array. The entire PE array is ultimately composed of n such basic column units. As can be seen from the figure, the main input port of each PE unit is the input neuron (16bits) and the corresponding connection weight (5bits). These two kinds of data are buffered in Neuron_Buffer and Weight_Buffer respectively after entering the PE unit. In addition, there are data input valid signals and PE unit calculation enable signals, etc., which are not marked in the figure.

Figures 10a and 10b show data flow examples of PE units in the convolution calculation mode and the fully connected calculation mode, respectively. Assume that the convolution window size configured in the convolution calculation mode is 5 × 5, the sliding step size is 1, and the input neurons calculated by each PE unit in the full connection calculation mode are 5, and the Neuron_Buffer and Weight_Buffer are filled and start the calculation.

An example of the data flow of the convolution calculation mode is shown in Figure 10a. In the first clock cycle, the product of neuron 1 and connection weight 1 is calculated, and neuron 1 is directly discarded, and the connection weight 1 is transferred to the end of Weight_Buffer; the second After the new neuron 6 is read in every clock cycle and the product of neuron 2 and connection weight 2 is calculated, both neuron 2 and connection weight 2 are dumped to the end of their respective Buffers; the third clock cycle calculates neuron 3 and After connecting the product of weight 3, transfer neuron 3 and connection weight 3 to the end of their respective Buffers; and so on, until the fifth clock cycle PE unit calculates and completes the multiplication and accumulation of one convolution window and one row, and outputs the result. At the same time, adjust the order of the values in the Buffer and wait for the calculation of the next convolution window.

An example of the data flow in the fully connected computing mode is shown in Figure 10b. The first clock cycle calculates the product of neuron 1 and connection weight 1, and then directly discards connection weight 1, and transfers neuron 1 to the end of Neuron_Buffer; the second After reading the new connection weight 6 and calculating the product of the neuron 2 and the connection weight 2, the connection weight 2 is directly discarded, and the neuron 2 is transferred to the end of Neuron_Buffer; and so on, until the fifth clock The periodic PE unit calculates the intermediate result of an output neuron, adjusts the order of the values in the Buffer while outputting the result, and waits for the calculation of the next group of output neurons.

Figure 11 is a configurable design of the present invention. In the convolution calculation mode, not only the size of the PE array can be configured according to the size of the convolution kernel and the number of output feature maps, but also the number of PE arrays that can be computed in parallel according to the available hardware resources and the number of input feature maps. Among them, each PE array corresponds to the convolution calculation of an input feature map, and the corresponding adder array is used to accumulate the output value of the PE unit and the corresponding bias value bias. When the number of input channels is not 1, the output neuron calculated by each PE array is the intermediate result of the corresponding output feature map, so an adder array is required outside the multiple PE arrays to accumulate all PE arrays to calculate the result. The output of the adder array is the final output of the corresponding output feature map neuron.

According to an effective neural network model compression algorithm and the characteristics of compressed sparse convolutional neural network data, the present invention designs a high-energy-efficiency, parameterized and configurable hardware accelerator. The hardware-friendly neural network model compression algorithm proposed by the invention and applied to the offline training of the model includes three main steps: pruning, ²ⁿ clustering and compression coding. Compared with the original network model, the compressed LeNet, AlexNet, and VGG reduce the storage consumption by about 222 times, 22 times, and 68 times, respectively. Then, in the hardware accelerator designed by the present invention, all multipliers in the PE unit are replaced by shifters, and all basic modules can be configured according to network computing and hardware resources. Therefore, it has the advantages of high speed, low power consumption, small resource occupation and high data utilization rate.

Claims (6)

1. a sparse convolutional neural network accelerator, is characterized in that, comprises: off-chip DRAM, neuron input buffer, neuron decoding unit, neuron coding unit, neuron output buffer, neuron on-chip global buffer, Weight input buffer, weight decoding unit, weight on-chip global buffer, PE computing unit array, activation unit and pooling unit; among them, Off-chip DRAM for storing compressed neuron and connection weight data; The neuron input buffer is used to buffer the compressed neuron data read from the off-chip DRAM and transmit it to the neuron decoding unit; The neuron decoding unit is used to decode the compressed neuron data and transmit the decoded neuron to the neuron on-chip global buffer; The neuron coding unit is used to compress and encode the data in the global buffer on the neuron chip and transmit it to the neuron output buffer; Neuron output buffer, used to store the received compressed neuron data in off-chip DRAM; The neuron on-chip global buffer is used to cache the decoded neurons and the intermediate results calculated by the PE array, and communicate data with the PE computing unit array; The activation unit is used to activate the result calculated by the PE computing unit array and transmit it to the pooling unit or the global buffer on the neuron chip; the pooling unit is used to downsample the result after the activation of the activation unit; The weight input buffer is used to cache the compressed connection weight data read from the off-chip DRAM and transmit it to the weight decoding unit; The weight decoding unit is used to decode the compressed connection weight data, and transmit the decoded connection weight to the weight on-chip global buffer; On-chip global buffer for weights, used to cache the decoded connection weights and communicate data with the PE computing unit array; The PE computing unit array is used for computing in the convolution computing mode or the fully connected computing mode. 2. an implementation method based on the sparse convolutional neural network accelerator described in claim 1, is characterized in that, after data is preprocessed, stored in DRAM, will be stored in the connection of the sparse network in off-chip DRAM The weight is read into the weight input buffer, decoded by the weight decoding unit, and stored in the weight on-chip global buffer; the neuron stored in the off-chip DRAM is read into the neuron input buffer, and is processed by the neuron decoding unit. After decoding, it is stored in the global buffer on the neuron chip; the calculation mode of the PE computing unit array is determined according to the configuration parameters of the current layer of the neural network, and the neurons and connection weights arranged after decoding are sent to the corresponding PE computing unit; neuron and After the connection weight data enters the PE, it is cached in Neuron_Buffer and Weight_Buffer respectively. When Neuron_Buffer and Weight_Buffer are filled for the first time, the product of neuron and connection weight starts to be calculated; among them, Neuron_Buffer is the neuron buffer in PE, and Weight_Buffer is in PE. weight buffer; If the result calculated by the PE computing unit array is an intermediate result, it will be temporarily stored in the global buffer on the neuron chip, and wait for the next accumulation with other intermediate results calculated by the PE computing unit array; if it is the final result of the convolution layer, then Send it to the activation unit, then send the activation result to the pooling unit, downsample and then save it back to the neuron on-chip global buffer; if it is the final result of the fully connected layer, it is sent to the activation unit, and then directly stored back to the neuron on-chip global buffer Buffer; the data stored in the global buffer on the neuron chip is compressed and encoded by the neuron coding unit according to the sparse matrix compression coding storage format, and then sent to the neuron output buffer, and then stored back to the off-chip DRAM, waiting for the next layer of calculation. Read in; loop in this way until the fully connected layer of the last layer outputs a one-dimensional vector whose length is the number of classification categories. 3. according to the realization method of the sparse convolutional neural network accelerator described in claim 2, it is characterized in that, the concrete process that data is preprocessed is: 1), prune the connection weights to make the neural network sparse; 2), 2 ⁿ connection weight clustering: First, according to the distribution of network parameters after pruning, select 2 ⁿ cluster centers in the form of quantized bit width within the range of the parameter set, and calculate the remaining non-zero parameters and cluster centers. Euclidean distance, which clusters non-zero parameters to the cluster center with the shortest Euclidean distance, where n is a negative integer, positive integer or zero; 3), for the sparse network after the clustering in step 2), the connection weight of the sparse network and the feature map neurons are compressed and stored in the off-chip DRAM using a sparse matrix compression coding storage format. 4. according to the realization method of the sparse convolutional neural network accelerator described in claim 2, it is characterised in that the sparse matrix compression coding storage format is specifically: the coding of the convolutional layer feature map and the fully connected layer neuron is divided into three parts , where each 5-bit represents the number of 0s between the current non-zero neuron and the previous non-zero neuron, each 16-bit represents the value of the non-zero element itself, and the last 1-bit is used to mark the convolution Whether the non-zero elements in the layer feature map wrap or whether the non-zero elements in the fully connected layer change layers, the entire storage unit encodes three non-zero elements and their relative position information; if a storage unit contains the convolutional layer feature map in the middle of a line at the end A non-zero element or the last non-zero element of the input neuron of a certain layer of the fully connected layer, then all the following 5-bit and 16-bit are filled with 0, and the last 1-bit flag is set to 1; the encoding of the connection weight is also Divided into three parts, each 4-bit represents the number of 0s between the current non-zero connection weight and the previous non-zero connection weight, each 5-bit represents the non-zero connection weight index, and the last 1-bit uses To mark whether the non-zero connection weight in the convolutional layer corresponds to the same input feature map or whether the non-zero connection weight in the fully connected layer corresponds to the same output neuron, the entire storage unit encodes seven non-zero connection weights and their relative position information ; Similarly, if a storage unit contains the last non-zero connection weight corresponding to the same input feature map in the convolutional layer or the last non-zero connection weight corresponding to the same output neuron in a layer of the fully connected layer, then the following The 4-bit and 5-bit are all filled with 0, and the last 1-bit flag is set to 1. 5. according to the realization method of the sparse convolutional neural network accelerator described in claim 2, it is characterized in that, when the calculation mode of PE calculation unit array is convolution calculation, if convolution window size is k × k, convolution If the window sliding step is s, the filled Neuron_Buffer and Weight_Buffer store neurons 1~k and connection weights 1~k respectively according to the principle of first-in, first-out; If the sliding step s of the convolution window is equal to 1, the neuron 1 is directly discarded after the product of the neuron 1 and the connection weight 1 is calculated in the first clock cycle, and the connection weight 1 is transferred to the end of the Weight_Buffer; the second After the clock cycle reads in the new neuron k+1 and calculates the product of neuron 2 and connection weight 2, dumps both neuron 2 and connection weight 2 to the end of their respective Buffers; the third clock cycle calculates neuron 3 After multiplying with connection weight 3, transfer neuron 3 and connection weight 3 to the end of their respective Buffers; and so on, until the kth clock cycle PE unit calculates and completes the multiplication and accumulation of one convolution window and one row, and outputs the result At the same time, adjust the order of the values in the Buffer according to the step size s, and wait for the calculation of the next convolution window; If the step size s is greater than 1, after calculating the product of neuron 1 and connection weight 1 in the first clock cycle, neuron 1 is directly discarded, and the connection weight 1 is transferred to the end of Weight_Buffer; the second clock cycle reads Enter a new neuron k+1, calculate the product of neuron 2 and connection weight 2, and discard neuron 2 directly, and transfer connection weight 2 to the end of Weight_Buffer; and so on, until the sth clock cycle, read Enter the new neuron k+s-1, calculate the product of the neuron s and the connection weight s, and discard the neuron s directly, and transfer the connection weight s to the end of the Weight_Buffer; read in the s+1th clock cycle After the new neuron k+s calculates the product of the neuron s+1 and the connection weight s+1 at the same time, the neuron s+1 and the connection weight s+1 are dumped to the end of their respective Buffers; After calculating the product of neuron s+2 and connection weight s+2 for each clock cycle, transfer neuron s+2 and connection weight s+2 to the end of their respective Buffers; and so on, until the kth clock cycle The PE unit calculates the multiplication and accumulation of one line of convolution window, and adjusts the order of the values in the Buffer according to the step size s while outputting the result, waiting to calculate the next convolution window. 6. according to the realization method of the sparse convolutional neural network accelerator described in claim 2, it is characterized in that, when the calculation mode of PE calculation unit array is full connection calculation, if the input neuron that each PE unit calculates is m After the Neuron_Buffer and Weight_Buffer are filled, neurons 1 to m and connection weights 1 to m are stored in accordance with the first-in-first-out principle, respectively; in the first clock cycle, the product of neuron 1 and connection weight 1 is calculated. Directly discard the connection weight 1, and dump the neuron 1 to the end of Neuron_Buffer; read the new connection weight m+1 in the second clock cycle and calculate the product of the neuron 2 and the connection weight 2, and directly discard the connection weight 2 , and transfer neuron 2 to the end of Neuron_Buffer; and so on, until the mth clock cycle PE unit calculates the intermediate result of an output neuron, adjusts the order of the values in the Buffer while outputting the result, and waits for the calculation of the next Group output neurons.

Images