CN107103113A - Towards the Automation Design method, device and the optimization method of neural network processor - Google Patents

Abstract

The present invention proposes a neural network processor-oriented automatic design method, device and optimization method. The method includes step 1, obtaining a neural network model description file and hardware resource constraint parameters, wherein the hardware resource constraint parameters include hardware resource size and Target running speed; step 2, according to the neural network model description file and the hardware resource constraint parameters, search the unit library from the built neural network component library, and generate the corresponding neural network model according to the unit library The hardware description language code of the neural network processor; step 3, converting the hardware description language code into the hardware circuit of the neural network processor.

Description

Automatic design method, device and optimization method for neural network processor

technical field

The invention relates to the technical field of neural network processor architecture, in particular to an automatic design method, device and optimization method for neural network processors.

Background technique

The rapid development of deep learning and neural network technology provides new solutions for large-scale data processing tasks. Various new neural network models have excellent performance in dealing with complex and abstract problems. New applications in the field emerge in an endless stream.

At present, the real-time task analysis using deep neural networks mostly relies on large-scale high-performance processors or general-purpose graphics processors. These devices have high cost and high power consumption. When they are applied to portable smart devices, they have large circuit scale, high energy consumption and expensive products. And so on a series of questions. Therefore, for energy-efficient real-time processing applications in embedded devices and small low-cost data centers, it is a more effective solution to use dedicated neural network processors to accelerate neural network model calculations instead of software. The topology and parameter design of the network model will change according to different application scenarios. In addition, the development and change of the neural network model is very fast, providing a general and efficient neural network processing that can face various application scenarios and cover various neural network models. It is very difficult for high-level application developers to design hardware acceleration solutions for different application requirements.

Currently, existing neural network hardware acceleration technologies include two methods, Application Specific Integrated Circuit (ASIC) chip and Field Programmable Gate Array (Field Programmable Gate Array, FPGA). Under the same process conditions, ASIC chips run fast and consume low power consumption, but the design process is complicated, the casting cycle is long, and the development cost is high, which cannot adapt to the characteristics of rapid update of neural network models; FPGA has the characteristics of flexible circuit configuration and short development cycle. features, but the running speed is relatively low, and the hardware overhead and power consumption are relatively large. No matter which of the above hardware acceleration technologies is used, it is necessary for the neural network model and algorithm developers to understand the network topology and data flow mode while mastering the hardware development technology, including processor architecture design, hardware code writing, simulation verification, layout and wiring, etc. , these technologies are more difficult to develop for high-level application developers who focus on researching neural network models and structural design, but do not have hardware design capabilities. Therefore, in order to enable high-level developers to efficiently develop neural network technology applications, it is very urgent to provide an automatic design method and tool for neural network processors oriented to multiple neural network models.

Contents of the invention

Aiming at the deficiencies of the prior art, the present invention proposes an automatic design method, device and optimization method for neural network processors.

The present invention proposes an automatic design method for neural network processors, including:

Step 1, obtaining the neural network model description file and hardware resource constraint parameters, wherein the hardware resource constraint parameters include hardware resource size and target operating speed;

Step 2, according to the neural network model description file and the hardware resource constraint parameters, search the unit library from the built neural network component library, and generate the neural network processing corresponding to the neural network model according to the unit library The hardware description language code of the device;

Step 3, converting the hardware description language code into the hardware circuit of the neural network processor.

The neural network processor includes a storage structure, a control structure, and a calculation structure.

The neural network model description file includes three parts: basic attributes, parameter descriptions, and connection information, wherein the basic attributes include layer names and layer types, and the parameter descriptions include the number of output layers, convolution kernel size, and step size. The above connection information includes connection name, connection direction and connection type.

The neural network reusable unit library includes two parts: a hardware description file and a configuration script.

The neural network reusable unit library includes a neuron unit, an accumulator unit, a pooling unit, a classifier unit, a local response normalization unit, a lookup table unit, an address generation unit, and a control unit.

The neural network processor includes a main address generation unit, a data address generation unit and a weight address generation unit.

It also includes determining the data path according to the user-specified neural network model and hardware resource constraint parameters, and determining the data resource sharing method according to the characteristics of the middle layer of the neural network;

Generate an address access flow of the memory according to the hardware configuration and network characteristics, and the address access flow is described by means of a finite state machine;

Generate a hardware description language code, and then convert it into a hardware circuit of the neural network processor.

It also includes generating a data storage map and a control instruction flow according to the neural network model, the hardware resource constraint parameters, and the hardware description language code.

The present invention also includes a neural network processor-oriented automatic design device, comprising:

Obtaining a data module, configured to obtain a neural network model description file and hardware resource constraint parameters, wherein the hardware resource constraint parameters include hardware resource size and target operating speed;

Generate a hardware description language code module, which is used to search for a unit library from the constructed neural network component library according to the neural network model description file and the hardware resource constraint parameters, and generate a module corresponding to the neural network according to the unit library. The hardware description language code of the neural network processor of the network model;

A hardware circuit module is generated for converting the hardware description language code into a hardware circuit of the neural network processor.

The neural network processor includes a storage structure, a control structure, and a calculation structure.

The neural network model description file includes three parts: basic attributes, parameter descriptions, and connection information, wherein the basic attributes include layer names and layer types, and the parameter descriptions include the number of output layers, convolution kernel size, and step size. The above connection information includes connection name, connection direction and connection type.

The neural network reusable unit library includes two parts: a hardware description file and a configuration script.

The neural network reusable unit library includes a neuron unit, an accumulator unit, a pooling unit, a classifier unit, a local response normalization unit, a lookup table unit, an address generation unit, and a control unit.

The neural network processor includes a main address generation unit, a data address generation unit and a weight address generation unit.

It also includes determining the data path according to the user-specified neural network model and hardware resource constraint parameters, and determining the data resource sharing method according to the characteristics of the middle layer of the neural network;

Generate an address access flow of the memory according to the hardware configuration and network characteristics, and the address access flow is described by means of a finite state machine;

Generate a hardware description language code, and then convert it into a hardware circuit of the neural network processor.

It also includes generating a data storage map and a control instruction flow according to the neural network model, the hardware resource constraint parameters, and the hardware description language code.

The present invention also proposes an optimization method based on a neural network processor-oriented automatic design method as described, including:

Step 1, define the convolution kernel size as k*k, stepping as s, memory width as d, number of data images as t, if k^2=d^2, divide the data into k*k size data blocks , the data width is consistent with the memory width, ensuring that the data is stored continuously in the memory;

Step 2, if k^2! =d^2, and stepping s is the greatest common divisor of k and memory width d, divide the data into data blocks of s*s size, and ensure that the data in a data map is continuously stored in the memory;

Step 3, if the above two items are not satisfied, find the greatest common divisor f of step s, k, and memory width d, divide the data into data blocks of size f*f, and store t data maps alternately.

As can be seen from the above scheme, the present invention has the advantages of:

The present invention can map the neural network model into a hardware circuit, automatically optimize the circuit structure and data storage mode according to hardware resource constraints and network characteristics, and generate corresponding control instruction streams at the same time, realizing the hardware and software automatic collaborative design of the neural network hardware accelerator, While shortening the design cycle of the neural network processor, the computing energy efficiency of the neural network processor is improved.

Description of drawings

Fig. 1 is the FPGA automatic implementation tool work flow chart of neural network processor provided by the present invention;

Fig. 2 is a schematic diagram of a neural network processor system that can be automatically generated by the present invention;

Fig. 3 is a schematic diagram of the neural network reusable unit library used in the present invention;

Fig. 4 is a schematic diagram of the interface of the address generation circuit used in the present invention.

detailed description

In order to make the object, technical solution, design method and advantages of the present invention clearer, the present invention will be further described in detail through specific embodiments in conjunction with the accompanying drawings. It should be understood that the specific embodiments described here are only used to explain the present invention, and It is not intended to limit the invention.

The present invention aims to provide an automatic design method, device and optimization method for neural network processors, the device includes a hardware generator and a compiler, and the hardware generator can automatically generate a neural network according to the neural network type and hardware resource constraints The hardware description language code of the processor, and then the designer uses the existing hardware circuit design method to generate the processor hardware circuit through the hardware description language; the compiler can generate the control and data scheduling instruction flow according to the neural network processor circuit structure.

Fig. 1 is a schematic diagram of the neural network processor automatic generation technology provided by the present invention, and the specific steps are:

Step 1, the device of the present invention reads the neural network model description file, which includes the network topology and the definitions of each computing layer;

Step 2, the device of the present invention reads in hardware resource constraint parameters, which include hardware resource size and target operating speed, etc., and the device of the present invention can generate a corresponding circuit structure according to the hardware constraint parameters;

Step 3, the device of the present invention indexes a suitable unit library from the constructed neural network component library according to the neural network model description script and hardware resource constraints, and the hardware circuit generator contained in the tool uses the above-mentioned unit library to generate a corresponding Neural Network Processor Hardware Description Language code for the neural network model;

Step 4, the compiler included in the device of the present invention generates data storage mapping and control instruction flow according to the neural network model, logic resource constraints and generated hardware description language code;

Step 5, convert the hardware description language into a hardware circuit through the existing hardware design method.

The automatically generated neural network processor of the present invention is based on the storage-control-computation structure;

The storage structure is used to store data involved in calculations, neural network weights and processor operation instructions;

The control structure includes a decoding circuit and a control logic circuit, which are used to analyze operation instructions and generate control signals, which are used to control the scheduling and storage of on-chip data and the neural network calculation process;

The computing structure includes computing units for participating in neural network computing operations in the processor.

Fig. 2 is a schematic diagram of the neural network processor system 101 that can be automatically generated in the present invention. The architecture of the neural network processor system 101 is composed of seven parts, including an input data storage unit 102, a control unit 103, an output data storage unit 104, and a weight storage unit. Unit 105 , instruction storage unit 106 , calculation unit 107 .

The input data storage unit 102 is used to store the data involved in the calculation, which includes the original feature map data and the data involved in the calculation of the intermediate layer; the output data storage unit 104 stores the calculated neuron response value; the instruction storage unit 106 stores the data involved in the calculation Instruction information, the instruction is parsed into a control flow to schedule neural network calculations; the weight storage unit 105 is used to store trained neural network weights;

The control unit 103 is connected to the output data storage unit 104, the weight storage unit 105, the instruction storage unit 106, and the calculation unit 107 respectively. The control unit 103 obtains the instruction stored in the instruction storage unit 106 and analyzes the instruction. The control unit 103 can analyze the instruction according to the The control signal obtained from the instruction controls the calculation unit to perform neural network calculation.

The calculation unit 107 is configured to perform corresponding neural network calculations according to the control signal generated by the control unit 103 . The calculation unit 107 is associated with one or more storage units, and the calculation unit 107 can obtain data from the data storage components in the input data storage unit 102 associated with it to perform calculations, and can send data to the output data storage unit 104 associated with it data input. The calculation unit 107 completes most of the operations in the neural network algorithm, that is, vector multiplication and addition operations and the like.

The present invention describes the characteristics of the neural network model by providing the neural network description file format. The content of the description file includes three parts: basic attributes, parameter descriptions, and connection information, wherein the basic attributes include layer names and layer types, and the parameter descriptions include the number of output layers. , convolution kernel size and step size, connection information includes connection name, connection direction, connection type.

In order to adapt to the hardware implementation design of various neural network models, the neural network reusable unit library provided by the present invention is shown in Figure 3. The unit library includes two parts: a hardware description file and a configuration script. The reusable unit library provided by the present invention includes but not limited to: neuron unit, accumulator unit, pooling unit, classifier unit, local response normalization unit, lookup table unit, address generation unit, control unit, etc.

When the present invention utilizes the reusable unit library to form a neural network processor system, it rationally optimizes and calls the unit library by reading the neural network model description file and hardware resource constraints.

During the working process of the neural network processor, the processor needs to automatically obtain the address stream of on-chip and off-chip memory data. In the present invention, the memory address stream is determined and generated by the compiler, and the memory access mode determined by the memory address stream is passed through text interaction. To the hardware generator, the memory access mode includes master access mode, data access mode and weight access mode, etc.

Hardware generators address generation unit (AGU) according to the memory access pattern.

The neural network processor circuit designed by the neural network processor automatic design tool provided by the present invention includes three types of address generation units, including: a main address generation unit, a data address generation unit and a weight address generation unit, wherein the main address generation unit The unit is responsible for the data exchange between the on-chip memory and the off-chip memory. The data address generation unit is responsible for reading data from the on-chip memory to the computing unit and storing the intermediate calculation results and final calculation results of the computing unit to the storage unit. The weight address generating unit is responsible for reading weight data from the on-chip memory to the computing unit.

In the present invention, the hardware circuit generator and the compiler work together to realize the design of the address generation circuit, and the specific design algorithm steps are:

Step 1, the device of the present invention determines the data path according to the neural network model and hardware constraints specified by the designer, and determines the data resource sharing mode according to the characteristics of the middle layer of the neural network;

Step 2, the compiler generates a memory address access flow according to the hardware configuration and network characteristics, and the address access flow is described by the compiler through a finite state machine;

Step 3, the finite state machine is mapped into an address generation circuit hardware description language by a hardware generator, and then mapped into a hardware circuit through a hardware circuit design method.

FIG. 4 is a schematic diagram of the general structure of the address generation circuit provided by the present invention. The address generation circuit of the present invention has a general signal interface, and the interface signals included in the interface are:

The starting address signal, that is, the first address of the data;

Data block size signal, the amount of data to fetch once;

The memory flag bit signal determines the number of the memory where the data is stored;

The working mode signal is divided into large convolution kernel data acquisition mode, small convolution kernel data acquisition mode, pooling mode, full convolution mode, etc.;

The convolution kernel size signal defines the size of the convolution kernel;

The length signal defines the size of the output image;

input layer number signal, marking the number of input layers;

Output layer number signal, marking the number of output layers;

Reset signal, when the signal is 1, initialize the address generation circuit;

Write enable signal, specifying the accessed memory for write operation;

Read enable signal, specifying the accessed memory for read operation;

Address signal, giving the access memory address;

End signal, access end signal.

The parameters ensure that the AGU supports multiple working modes and ensures that correct read and write address streams can be generated during different working modes and neural network propagation.

For different target networks, the tool selects the necessary parameters from the template to build an address generator and provides on-chip and off-chip memory access modes.

The neural network processor provided by the present invention constructs a processor architecture in a data-driven manner, so the address generation circuit not only provides access addresses but also drives the execution of different neural layers and layer data blocks.

Due to resource constraints, when the neural network model is mapped to a hardware circuit, it cannot be completely expanded according to its model description form. Therefore, the automatic design tool proposed by the present invention optimizes the data storage and access mechanism by using software and hardware to work together, including two parts. : First, the compiler analyzes the computing throughput and on-chip memory size of the neural network processor, divides the neural network feature data and weight data into appropriate data blocks for centralized storage and access; secondly, according to the calculation unit scale, memory and data bit width Data partitioning is performed within a data block.

The present invention proposes an optimization method for data storage and access based on the above-mentioned optimization mechanism, and the specific implementation steps are:

Step 1, define the convolution kernel size as k*k, stepping as s, memory width as d, number of data images as t, if k^2=d^2, divide the data into k*k size data blocks , the data width is consistent with the memory width, ensuring that the data is stored continuously in the memory;

Step 2, if k^2! =d^2, and s is the greatest common divisor of k and d, divide the data into data blocks of s*s size, and ensure that the data in a data map can be continuously stored in the memory;

Step 3, if the above two items are not satisfied, find the greatest common divisor f of s, k, and d, divide the data into data blocks with a size of f*f, and store t data graphs alternately.

The calculation data of the neural network includes input feature data and trained weight data. Through a good data storage layout, the internal data bandwidth of the processor can be reduced and the storage space utilization efficiency can be improved. The automatic design tool provided by the invention improves the calculation efficiency of the processor by increasing the processor data storage locality.

In summary, the present invention provides an automatic design tool for neural network processors, which has the functions of mapping from neural network models to describe the hardware codes of neural network processors, optimizing processor architecture according to hardware resource constraints, and automatically generating control Streaming instructions and other functions realize the automatic design of neural network processors, reduce the design cycle of neural network processors, and adapt to the application characteristics of neural network technology with fast update of network models, high computing speed requirements, and high energy efficiency requirements.

It should be understood that although this description is described according to various embodiments, not each embodiment only includes an independent technical solution, and this description of the description is only for clarity, and those skilled in the art should take the description as a whole , the technical solutions in the various embodiments can also be properly combined to form other implementations that can be understood by those skilled in the art.

The present invention also proposes an automatic design device for neural network processors, including:

Obtaining a data module, configured to obtain a neural network model description file and hardware resource constraint parameters, wherein the hardware resource constraint parameters include hardware resource size and target operating speed;

Generate a hardware description language code module, which is used to search for a unit library from the constructed neural network component library according to the neural network model description file and the hardware resource constraint parameters, and generate a module corresponding to the neural network according to the unit library. The hardware description language code of the neural network processor of the network model;

A hardware circuit module is generated for converting the hardware description language code into a hardware circuit of the neural network processor.

The neural network processor includes a storage structure, a control structure, and a calculation structure.

The neural network model description file includes three parts: basic attributes, parameter descriptions, and connection information, wherein the basic attributes include layer names and layer types, and the parameter descriptions include the number of output layers, convolution kernel size, and step size. The above connection information includes connection name, connection direction and connection type.

The neural network processor includes a main address generation unit, a data address generation unit and a weight address generation unit.

It also includes determining the data path according to the user-specified neural network model and hardware resource constraint parameters, and determining the data resource sharing method according to the characteristics of the middle layer of the neural network;

Generate an address access flow of the memory according to the hardware configuration and network characteristics, and the address access flow is described by means of a finite state machine;

The neural network reusable unit library includes two parts: a hardware description file and a configuration script.

The neural network reusable unit library includes a neuron unit, an accumulator unit, a pooling unit, a classifier unit, a local response normalization unit, a lookup table unit, an address generation unit, and a control unit.

The finite state machine is mapped to an address, and a hardware description language code is generated, and then converted into a hardware circuit of the neural network processor.

It also includes generating a data storage map and a control instruction flow according to the neural network model, the hardware resource constraint parameters, and the hardware description language code.

The above descriptions are only illustrative specific implementations of the present invention, and are not intended to limit the scope of the present invention. Any equivalent changes, modifications and combinations made by those skilled in the art without departing from the concept and principle of the present invention shall fall within the protection scope of the present invention.

Claims (17)

1. an automated design method for neural network processors, characterized in that, comprising: Step 1, obtaining the neural network model description file and hardware resource constraint parameters, wherein the hardware resource constraint parameters include hardware resource size and target operating speed; Step 2, according to the neural network model description file and the hardware resource constraint parameters, search the unit library from the built neural network component library, and generate the neural network processing corresponding to the neural network model according to the unit library The hardware description language code of the device; Step 3, converting the hardware description language code into the hardware circuit of the neural network processor. 2. a kind of neural network processor-oriented automatic design method as claimed in claim 1, is characterized in that, described neural network processor comprises storage structure, control structure, computing structure. 3. a kind of neural network processor-oriented automatic design method as claimed in claim 1, is characterized in that, described neural network model description file comprises three parts of basic attribute, parameter description and connection information, wherein said basic attribute includes Layer name and layer type, the parameter description includes the number of output layers, convolution kernel size and step size, and the connection information includes connection name, connection direction, and connection type. 4. A neural network processor-oriented automatic design method according to claim 1, wherein the neural network reusable unit library includes two parts, a hardware description file and a configuration script. 5. a kind of automatic design method facing neural network processor as claimed in claim 1, is characterized in that, described neural network reusable unit storehouse comprises neuron unit, accumulator unit, pooling unit, classifier unit , a partial response normalization unit, a lookup table unit, an address generation unit, and a control unit. 6. A neural network processor-oriented automatic design method according to claim 1, characterized in that said neural network processor comprises a main address generation unit, a data address generation unit and a weight address generation unit. 7. A kind of neural network processor-oriented automatic design method as claimed in claim 1, is characterized in that, also comprises determining the data path according to the neural network model specified by the user and the hardware resource constraint parameter, and according to the neural network middle layer characteristic Determine the data resource sharing method; Generate an address access flow of the memory according to the hardware configuration and network characteristics, and the address access flow is described by means of a finite state machine; Generate a hardware description language code, and then convert it into a hardware circuit of the neural network processor. 8. a kind of neural network processor-oriented automatic design method as claimed in claim 1, is characterized in that, also comprises according to described neural network model, described hardware resource constraint parameter, described hardware description language code, generates data Memory mapping and control instruction flow. 9. An automatic design device for neural network processors, characterized in that it comprises: Obtaining a data module, configured to obtain a neural network model description file and hardware resource constraint parameters, wherein the hardware resource constraint parameters include hardware resource size and target operating speed; Generate a hardware description language code module, which is used to search for a unit library from the constructed neural network component library according to the neural network model description file and the hardware resource constraint parameters, and generate a module corresponding to the neural network according to the unit library. The hardware description language code of the neural network processor of the network model; A hardware circuit module is generated for converting the hardware description language code into a hardware circuit of the neural network processor. 10 . The neural network processor-oriented automatic design device according to claim 9 , wherein the neural network processor comprises a storage structure, a control structure, and a calculation structure. 11 . 11. A neural network processor-oriented automatic design device according to claim 9, wherein the neural network model description file includes three parts: basic attributes, parameter descriptions and connection information, wherein the basic attributes include Layer name and layer type, the parameter description includes the number of output layers, convolution kernel size and step size, and the connection information includes connection name, connection direction, and connection type. 12. The neural network processor-oriented automatic design device according to claim 9, wherein the neural network reusable unit library includes two parts: a hardware description file and a configuration script. 13. A kind of automatic design device for neural network processor as claimed in claim 9, is characterized in that, described neural network reusable unit storehouse comprises neuron unit, accumulator unit, pooling unit, classifier unit , a partial response normalization unit, a lookup table unit, an address generation unit, and a control unit. 14 . The neural network processor-oriented automatic design device according to claim 9 , wherein the neural network processor comprises a main address generation unit, a data address generation unit and a weight address generation unit. 15 . 15. A neural network processor-oriented automatic design device as claimed in claim 9, further comprising determining the data path according to the user-specified neural network model and hardware resource constraint parameters, and determining the data path according to the characteristics of the neural network middle layer Determine the data resource sharing method; Generate an address access flow of the memory according to the hardware configuration and network characteristics, and the address access flow is described by means of a finite state machine; Generate a hardware description language code, and then convert it into a hardware circuit of the neural network processor. 16. A kind of neural network processor-oriented automatic design device as claimed in claim 9, is characterized in that, also comprises according to described neural network model, described hardware resource constraint parameter, described hardware description language code, generates data Memory mapping and control instruction flow. 17. A kind of optimization method based on a kind of automatic design method for neural network processor as described in any one of claim 1-8, it is characterized in that, comprising: Step 1, define the convolution kernel size as k*k, stepping as s, memory width as d, number of data images as t, if k^2=d^2, divide the data into k*k size data blocks , the data width is consistent with the memory width, ensuring that the data is stored continuously in the memory; Step 2, if k^2! =d^2, and stepping s is the greatest common divisor of k and memory width d, divide the data into data blocks of s*s size, and ensure that the data in a data map is continuously stored in the memory; Step 3, if the above two items are not satisfied, find the greatest common divisor f of step s, k, and memory width d, divide the data into data blocks of size f*f, and store t data maps alternately.