CN108647779A - A kind of low-bit width convolutional neural networks Reconfigurable Computation unit - Google Patents

Abstract

The invention discloses a kind of low-bit width convolutional neural networks Reconfigurable Computation units.The unit includes：Several restructural displacement accumulator module, multi-channel gating device and quantification treatment modules；Restructural displacement accumulator module includes controller, the first register, the second register, third register and shifting accumulator；The present invention utilizes network discreteness structure controller, the first register, the second register, third register and shifting accumulator, it judges whether the fixed-point number data of current period and index weight are zero by controller, once the fixed-point number data and index weight that detect current period are zero, then the second trigger signal that the first trigger signal and the second register sent out according to the first register is sent out controls the third register output current period and shifts cumulative data；The present invention can realize that the flexible fixed point of 4 bits and 8 bits multiplies accumulating operation, moreover it is possible to improve displacement accumulating operation rate, reduce memory and power consumption that operation occupies.

Description

A low-bit-width convolutional neural network reconfigurable computing unit

technical field

The invention relates to the technical field of reconfigurable computing, in particular to a low-bit-width convolutional neural network reconfigurable computing unit.

Background technique

With the development of artificial intelligence, deep learning has achieved great success in fields such as speech recognition, computer vision, and autonomous driving, pushing these fields to further develop. The core technology that promotes the development of deep learning research is the convolutional neural network. The target recognition technology using Convolutional Neural Network (Convolutional Neural Network) defeated the traditional image recognition method in the large-scale image recognition competition ILSVRC2012 held in 2012, announcing the arrival of the deep academic era. With the continuous development of deep learning technology, the convolutional neural network structure has been continuously optimized, and the recognition performance has also been continuously improved. On the large-scale image recognition competition ILSVRC2015 held in 2015, the convolutional neural network surpassed human image recognition ability for the first time. This milestone marks the great success of deep learning technology.

With the continuous improvement of the performance of convolutional neural networks, the network structure has become more and more complex, corresponding to more computing and storage requirements. In order to support the calculation of convolutional neural network, the network processing process is usually run on the server and data center. When interacting with the data center, a large amount of data needs to be transmitted, which brings great delay and hinders the convolutional neural network. Applications in embedded devices such as smartphones and smart cars. In order to solve this problem, academia and industry have begun to study how to deploy convolutional neural networks to accelerators of embedded hardware systems, so many effective convolutional neural network accelerators have been designed with dedicated computing units (PEs), It is common to use fixed computational units for different convolutional neural network models. Due to the diversity of convolutional neural networks, fixed computing units may not be suitable when the network model changes, which will increase data movement and hurt power efficiency. Moreover, their convolutional mapping method is not very scalable for various convolution parameters, and a mismatch between network shape and computing resources will occur, thereby reducing resource utilization and performance. Therefore, how to design reconfigurable computing units for different networks has become an important research content in this field.

Existing reconfigurable computing units basically use proprietary DSP (Digital Signal Processing) for calculations, and DSP computing units are designed for floating-point operations. In ordinary floating-point convolutional neural networks In hardware design, DSP units are usually used for multiply-and-accumulate operations (Multiply-and-Accumulat, MAC), and a DSP can be used to complete a multiply-accumulate operation within one clock cycle. However, the DSP calculation unit is not suitable for multiplying and accumulating operations on low-bit widths. This shortcoming makes it unable to exert its full capabilities in low-bit-width hardware design.

In order to solve this problem, Xilinx has introduced a special DSP mapping technology, which is designed for the FPGA chip launched by Xilinx, so that each FPGA on-chip DSP calculation unit can realize two parallel 8-bit multiply-accumulate operations. This technology gives full play to the computing power of the DSP on the FPGA chip and improves the area and power consumption performance of the FPGA. However, the scope of application of this technology is too narrow, and it can only be used for eight-bit wide fixed-point multiply-accumulate operations, and cannot be applied to the special operation requirements of exponential convolutional neural networks. Based on the above problems, how to overcome the above problems has become an urgent problem to be solved in this field.

Contents of the invention

The purpose of the present invention is to provide a low-bit-width convolutional neural network reconfigurable computing unit to realize the computing requirements of the exponential convolutional neural network, which can not only realize 4-bit and 8-bit flexible fixed-point multiplication The bit accumulation operation rate reduces the memory and power consumption occupied by the operation.

In order to achieve the above object, the present invention provides a low-bit width convolutional neural network reconfigurable computing unit, which is applied to the displacement accumulation operation of the exponential convolutional neural network, which includes: Several reconfigurable shift accumulation modules, multiplexers and quantization processing modules;

The multiplexers are respectively connected to each of the reconfigurable shift-accumulation modules, and are used to select the shift-accumulation data of the current period output by the reconfigurable shift-accumulation modules; the quantization processing module and The multiplexers are connected to perform quantization processing according to the shift accumulation data of the current period to obtain quantization processing data; wherein:

The reconfigurable shift accumulation module includes a controller, a first register, a second register, a third register and a shift accumulator;

The controller is used to judge whether the index weight data of the current cycle is a negative number; if the index weight data of the current cycle is a negative number, no data shift and accumulation operation is required, and the index weight data of the next cycle is waited to be judged; if the index weight data of the current cycle If the data is not a negative number, then judge whether the index weight data of the current cycle is 0; if the index weight data of the current cycle is not 0, then control the first register to store the index weight data of the current cycle; the index weight data of the current cycle is 0, control, the first register sends a first trigger signal;

The controller is also used to judge whether the fixed-point number data of the current cycle is a negative number; if the fixed-point number data of the current cycle is a negative number, no data shift and accumulation operation is required, and the fixed-point number data of the next cycle is waited to be judged; if the fixed-point number data of the current cycle If the fixed-point number data is not a negative number, then judge whether the fixed-point number data of the current cycle is 0; if the fixed-point number data of the current cycle is not 0, then control the second register to store the fixed-point number data of the current cycle; if the fixed-point number data of the current cycle is 0, then control the second register to send the second trigger signal;

The third register is connected to the first register and the second register respectively, and the third register is used for triggering according to the first trigger signal sent by the first register or the second trigger signal sent by the second register. The signal controls the third register to output the shift accumulation data of the current cycle; the third register is also used to store the shift accumulation data of the previous cycle;

The shift accumulator is respectively connected to the first register, the second register and the third register, and the shift accumulator is used to store the index weight data of the previous cycle according to the first register, The fixed-point number data of the previous cycle stored in the second register and the first shift-accumulation data of the previous cycle stored in the third register determine the shift-accumulation data of the current cycle, and the shift-accumulation data of the current cycle stored in the third register.

Preferably, the shift accumulator includes:

A shifter, respectively connected to the first register and the second register, for determining shift data according to the index weight data stored in the first register and the fixed-point number data stored in the second register;

The accumulator is connected to the shifter and the third register respectively, and is used to determine the shift data determined by the shifter and the first shift accumulation data of the last cycle stored in the third register. Shift-accumulate data for the current cycle.

Preferably, the low-bit-width reconfigurable shift-accumulation module further includes:

The output register is connected to the third register and is used for storing the shift-accumulation data of the current period output by the third register.

Preferably, the index weight data is 4 bits.

Preferably, the fixed-point number data is 8 bits.

Preferably, the shift accumulation data is 18 bits.

Preferably, the quantized data is 8-bit data.

Compared with the prior art, the present invention has the following technical effects:

The present invention utilizes the discreteness of the network to construct a controller, a first register, a second register, a third register, and a shift accumulator, and judges whether the fixed-point number data and the index weight of the current cycle are zero values through the controller. Once the current cycle is detected, If the fixed-point number data and the index weight are zero, then the first trigger signal sent by the first register and the second trigger signal sent by the second register are used to control the third register to output the shift accumulation data of the current cycle; It can not only realize 4-bit and 8-bit flexible fixed-point multiply-accumulate operations, but also increase the speed of shift-accumulate operations, and reduce the memory and power consumption occupied by operations.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the accompanying drawings required in the embodiments. Obviously, the accompanying drawings in the following description are only some of the present invention. Embodiments, for those of ordinary skill in the art, other drawings can also be obtained according to these drawings without paying creative labor.

FIG. 1 is a structural diagram of a reconfigurable computing unit of a low-bit-width convolutional neural network according to an embodiment of the present invention;

FIG. 2 is a structural diagram of a low-bit-width reconfigurable shift-accumulation module according to an embodiment of the present invention.

Among them, 10, reconfigurable shift and accumulation module, 11, controller, 12, first register, 13, second register, 14, shifter, 15, accumulator, 16, third register, 17, output register , 20, a multiplexer, 30, a quantization processing module.

Detailed ways

The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

The purpose of the present invention is to provide a low-bit-width convolutional neural network reconfigurable computing unit to realize the computing requirements of the exponential convolutional neural network, which can not only realize 4-bit and 8-bit flexible fixed-point multiplication The bit accumulation operation rate reduces the memory and power consumption occupied by the operation.

In order to make the above objects, features and advantages of the present invention more comprehensible, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

Figure 1 is a structural diagram of a low-bit-width convolutional neural network reconfigurable computing unit according to an embodiment of the present invention; Figure 2 is a structural diagram of a low-bit-width reconfigurable shift-accumulation module according to an embodiment of the present invention, as shown in Figures 1-2, the present invention The invention provides a low-bit-width convolutional neural network reconfigurable computing unit, the low-bit-width convolutional neural network reconfigurable computing unit is applied to the displacement accumulation operation of the exponential convolutional neural network, and the low-bit-width convolutional neural network can be The reconstruction calculation unit includes: a plurality of reconfigurable shift accumulation modules 10, a multiplexer 20 and a quantization processing module 30;

The reconfigurable shift-accumulation module 10 includes a controller 11 , a first register 12 , a second register 13 , a third register 16 , and a shift-accumulator 15 .

The controller 11 is used to judge whether the index weight data of the current cycle is a negative number; if the index weight data of the current cycle is a negative number, no data shift and accumulation operation is required, and the index weight data of the next cycle is waited to be judged; if the index weight data of the current cycle The index weight data is not a negative number, then judge whether the index weight data of the current cycle is 0; if the index weight data of the current cycle is not 0, then control the first register 12 to store the index weight data of the current cycle; the index weight data of the current cycle is 0, the first register 12 is controlled to send out the first trigger signal.

Described controller 11 is also used for judging whether the fixed-point number data of current cycle is a negative number; If the fixed-point number data of current cycle is negative number, then need not data shift accumulation operation, wait to judge the fixed-point number data of next cycle; If current cycle If the fixed-point number data of the current cycle is not negative, it is judged whether the fixed-point number data of the current cycle is 0; if the fixed-point number data of the current cycle is not 0, then the second register 13 is controlled to store the fixed-point number data of the current cycle; If the point data is 0, the second register 13 is controlled to send a second trigger signal.

The third register 16 is connected to the first register 12 and the second register 13 respectively, and the third register 16 is used to send the first trigger signal according to the first register 12 or the second register The second trigger signal sent by 13 controls the third register 16 to output the shift accumulation data of the current cycle; the third register 16 is also used to store the shift accumulation data of the previous cycle.

The shift accumulator 15 is connected to the first register 12, the second register 13, and the third register 16 respectively, and the shift accumulator 15 is used for storing the upper register according to the first register 12. The index weight data of one period, the fixed-point number data of the previous period stored in the second register 13 and the first shift-accumulation data of the previous period stored in the third register 16 determine the shift-accumulation data of the current period, And store the shift accumulation data of the current cycle in the third register 16 .

Multiplexers 20 are respectively connected to the low-bit-width reconfigurable shift-accumulation modules 10 for selecting the shift-accumulation data of the current period output by the low-bit-width reconfigurable shift-accumulation modules 10 .

The quantization processing module 30 is connected with the multiplexer 20 and is used for performing quantization processing according to the shift-accumulation data of the current period to obtain quantization processing data. The quantized data is 8-bit data.

The shift accumulator 15 of the present invention includes:

The shifter 14 is connected to the first register 12 and the second register 13 respectively, and is used to determine the shift according to the index weight data stored in the first register 12 and the fixed-point number data stored in the second register 13. bit data.

The accumulator 15 is connected to the shifter 14 and the third register 16 respectively, and is used for shifting data determined by the shifter 14 and the first cycle of the last cycle stored by the third register 16. Shift-accumulate data determines the shift-accumulate data for the current cycle.

The low-bit-width convolutional neural network reconfigurable shift accumulation module 10 of the present invention also includes: an output register 17, connected to the third register 16, for storing the shift of the current cycle output by the third register 16 Accumulated data.

The index weight data in the present invention is 4 bits.

The fixed-point data in the present invention is 8 bits.

The shift accumulation data in the present invention is 18 bits.

Since the convolutional neural network contains a large part of discreteness, making full use of the discrete performance of the network can greatly improve the power performance of the hardware design. Therefore, in order to further improve the performance of the reconfigurable computing unit, the convolutional neural network of the present invention is extended. The discreteness of the network and the use of discreteness to improve the power consumption performance of the network. Studies have shown that about 40% to 60% of the input data in convolutional neural networks are zero-valued, and a large part of the small data in the weight data can be pruned without affecting the accuracy of the network, so the multiplication of zero values And addition is meaningless because it does not affect the output result, so once the present invention detects that the fixed-point number data and the index weight of the previous cycle are zero, then according to the first trigger signal sent by the first register 12 and the second The second trigger signal sent by the register 13 controls the third register 16 to output the shift accumulation data of the current period.

The quantization processing module 30 of the present invention quantizes the 18-bit shifted accumulation data to obtain 8-bit quantized processed data.

In the shift-accumulation calculation process of the present invention, the width of the shift-accumulation data of the previous cycle and the shift-accumulation data of the output current cycle is obviously greater than the width of the fixed-point number data and the index weight data, and this is because in the shift-accumulation Accumulation calculations require a large calculation range to avoid calculation overflow. The present invention sets the shift-accumulation data width of the current period to 18-bits, which can fully accommodate the shift-accumulation data of the current period obtained by all shift-accumulation operations.

The xc7z020clg400-2 experimental board used in the present invention has the following advantages for testing: (1) The reconfigurable computing unit designed in the present invention improves the shift and accumulation operation rate. After testing, the common neural network accelerator structure using the common reconfigurable multiply-accumulate unit occupies 95 LUTs, and the calculation power consumption is 1.658, while the common neural network accelerator structure adopting the reconfigurable computing unit designed by the present invention only occupies 46 LUTs , the computing power consumption is only 1W, obviously the reconfigurable computing unit designed by the present invention is close to twice the operating frequency of the ordinary multiply-accumulate unit. (2) The reconfigurable computing unit designed in the present invention fully utilizes reconfigurable performance, supports a network structure with multiple bit widths and multiple configurations, and realizes flexible bit width configurations of 4-8 bits. (3) The reconfigurable computing unit designed in the present invention fully utilizes the discreteness of the network to further improve hardware performance. (4) The present invention enables the exponential convolutional neural network to be effectively mapped on the embedded system, further reducing its area and power consumption.

Table 1 Comparison between reconfigurable computing unit and reconfigurable multiply-accumulate unit

Each embodiment in this specification is described in a progressive manner, each embodiment focuses on the difference from other embodiments, and the same and similar parts of each embodiment can be referred to each other.

In this paper, specific examples have been used to illustrate the principle and implementation of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; meanwhile, for those of ordinary skill in the art, according to the present invention Thoughts, there will be changes in specific implementation methods and application ranges. In summary, the contents of this specification should not be construed as limiting the present invention.

Claims (7)

1. A low-bit wide convolutional neural network reconfigurable computing unit, characterized in that the low-bit wide convolutional neural network reconfigurable computing unit is applied to the displacement accumulation operation of the exponential convolutional neural network, which includes: several A reconfigurable shift-accumulation module, a multiplexer, and a quantization processing module; the multiplexers are respectively connected to each of the reconfigurable shift-accumulation modules for selecting the reconfigurable shift-accumulation The shift accumulation data of the current cycle output by the module; the quantization processing module is connected to the multiplexer, and is used to perform quantization processing according to the shift accumulation data of the current cycle to obtain quantization processing data; wherein: The reconfigurable shift accumulation module includes a controller, a first register, a second register, a third register and a shift accumulator; The controller is used to judge whether the index weight data of the current cycle is a negative number; if the index weight data of the current cycle is a negative number, no data shift and accumulation operation is required, and the index weight data of the next cycle is waited to be judged; if the index weight data of the current cycle If the data is not a negative number, then judge whether the index weight data of the current cycle is 0; if the index weight data of the current cycle is not 0, then control the first register to store the index weight data of the current cycle; the index weight data of the current cycle is 0, control, the first register sends a first trigger signal; The controller is also used to judge whether the fixed-point number data of the current cycle is a negative number; if the fixed-point number data of the current cycle is a negative number, no data shift and accumulation operation is required, and the fixed-point number data of the next cycle is waited to be judged; if the fixed-point number data of the current cycle If the fixed-point number data is not a negative number, then judge whether the fixed-point number data of the current cycle is 0; if the fixed-point number data of the current cycle is not 0, then control the second register to store the fixed-point number data of the current cycle; if the fixed-point number data of the current cycle is 0, then control the second register to send the second trigger signal; The third register is connected to the first register and the second register respectively, and the third register is used for triggering according to the first trigger signal sent by the first register or the second trigger signal sent by the second register. The signal controls the third register to output the shift accumulation data of the current cycle; the third register is also used to store the shift accumulation data of the previous cycle; The shift accumulator is respectively connected to the first register, the second register and the third register, and the shift accumulator is used to store the index weight data of the previous cycle according to the first register, The fixed-point number data of the previous cycle stored in the second register and the first shift-accumulation data of the previous cycle stored in the third register determine the shift-accumulation data of the current cycle, and the shift-accumulation data of the current cycle stored in the third register. 2. The low-bit-width convolutional neural network reconfigurable computing unit according to claim 1, wherein the shift accumulator comprises: A shifter, respectively connected to the first register and the second register, for determining shift data according to the index weight data stored in the first register and the fixed-point number data stored in the second register; The accumulator is connected to the shifter and the third register respectively, and is used to determine the shift data determined by the shifter and the first shift accumulation data of the last cycle stored in the third register. Shift-accumulate data for the current cycle. 3. The low-bit-width convolutional neural network reconfigurable computing unit according to claim 1, wherein the reconfigurable shift-and-accumulate module further comprises: The output register is connected to the third register and is used for storing the shift-accumulation data of the current period output by the third register. 4. The low-bit-width convolutional neural network reconfigurable computing unit according to claim 1, wherein the index weight data is 4 bits. 5. The low-bit-width convolutional neural network reconfigurable computing unit according to claim 1, wherein the fixed-point number data is 8 bits. 6. The low-bit-width convolutional neural network reconfigurable computing unit according to claim 1, wherein the shift-accumulation data is 18 bits. 7. The low-bit-width convolutional neural network reconfigurable computing unit according to claim 1, wherein the quantized data is 8-bit data.