CN114462582A - Data processing method, device and equipment based on convolutional neural network model - Google Patents

Abstract

The embodiment of the application can be applied to various scenes such as cloud technology, artificial intelligence, intelligent traffic, auxiliary driving and the like, and particularly discloses a data processing method, device and equipment based on a convolutional neural network model. The method comprises the following steps: acquiring a convolutional neural network model to be processed and a pruning configuration file matched with the convolutional neural network model; identifying a network structure of the convolutional neural network model based on an identification algorithm to obtain a target network structure, and determining a target pruning algorithm based on a pruning configuration file, wherein the target pruning algorithm is independent of the identification algorithm; pruning the target network structure based on a target pruning algorithm to obtain a pruned convolutional neural network model; and processing the data to be processed based on the pruned convolutional neural network model to obtain a processing result. According to the technical scheme of the embodiment of the application, the data processing efficiency based on the convolutional neural network model is improved, and the data processing scheme based on the convolutional neural network model is greatly optimized.

Description

Data processing method, device and equipment based on convolutional neural network model

technical field

The present application relates to the field of computer technology, and in particular, to a data processing method based on a convolutional neural network model, a data processing apparatus based on a convolutional neural network model, an electronic device, and a computer-readable medium.

Background technique

The data processing scheme based on the convolutional neural network model in the related art usually firstly trains the convolutional neural network model, then performs pruning and optimization based on the trained convolutional neural network model, and then performs pruning and optimization based on the convolutional neural network model after the pruning optimization. The neural network model performs corresponding data processing. Among them, in the process of pruning optimization, relevant workers usually manually select a suitable pruning algorithm for pruning optimization, but such a pruning optimization method is inefficient, which makes the convolutional neural network based on pruning optimization. The network model is also less efficient for corresponding data processing.

It can be seen that how to improve the data processing efficiency based on the convolutional neural network model is an urgent problem to be solved.

SUMMARY OF THE INVENTION

In order to solve the above technical problems, the embodiments of the present application provide a data processing method, device, equipment, and medium based on a convolutional neural network model, thereby improving the data processing efficiency based on the convolutional neural network model at least to a certain extent. .

According to an aspect of the embodiments of the present application, the embodiments of the present application provide a data processing method based on a convolutional neural network model, the method comprising: acquiring a convolutional neural network model to be processed and combining with the convolutional neural network model A pruning configuration file for model matching; the network structure of the convolutional neural network model is identified based on an identification algorithm to obtain a target network structure, and a target pruning algorithm is determined based on the pruning configuration file, wherein the target pruning The pruning algorithm is independent of the identifying algorithm; the target network structure is pruned based on the target pruning algorithm to obtain a pruned convolutional neural network model; the data to be processed is based on the pruned convolutional neural network model Process and get the processing result.

According to an aspect of the embodiments of the present application, the embodiments of the present application provide a data processing apparatus based on a convolutional neural network model, the apparatus comprising: an acquisition module configured to acquire a convolutional neural network model to be processed and a The pruning configuration file matched with the convolutional neural network model; the identification and determination module is configured to identify the network structure of the convolutional neural network model based on the recognition algorithm, obtain the target network structure, and based on the pruning configuration file Determine a target pruning algorithm, wherein the target pruning algorithm is independent of the identification algorithm; a pruning module is configured to prune the target network structure based on the target pruning algorithm to obtain a pruned volume The convolutional neural network model; the processing module is configured to process the data to be processed based on the pruned convolutional neural network model to obtain the processing result.

In an embodiment of the present application, based on the foregoing solution, the pruning configuration file includes a first pruning rate; the pruning module includes: a first reading unit configured to read the pruning configuration file The first cropping rate contained in the target network structure; a first determining unit, configured to determine, based on the first cropping rate, a second cropping rate corresponding to each network layer included in the target network structure; a first pruning unit, configured In order to prune the network layers respectively through the target pruning algorithm according to the second clipping rate corresponding to the network layers, to obtain the network layers after the pruning; the generating unit is configured to be based on the pruning Each layer of the network generates a pruned convolutional neural network model.

In an embodiment of the present application, based on the foregoing solution, the first determining unit is specifically configured to: if the first cropping rate represents a global cropping rate, then based on the pruning prior knowledge information and the first cropping rate rate, and determine the second cropping rate corresponding to each network layer included in the target network structure; if the first cropping rate represents the cropping rate of the specified network layer contained in the target network structure, then based on the first cropping rate determining a second cropping rate corresponding to the specified network layer, and determining other networks other than the specified network layer included in the target network structure based on the pruning prior knowledge information and the first cropping rate The second cropping rate corresponding to the layer.

In an embodiment of the present application, based on the foregoing solution, the first pruning unit is specifically configured to: if the target pruning algorithm is a structured pruning algorithm, then based on the structured pruning algorithm Delete the feature maps corresponding to the network layers respectively, so as to prune the network layers to obtain the network layers after structured pruning; if the target pruning algorithm is an unstructured pruning algorithm based on the unstructured pruning algorithm, the eigenvalues contained in the feature maps corresponding to the network layers are respectively zeroed, so as to prune the network layers to obtain an unstructured pruning algorithm. Each network layer after the branch.

In an embodiment of the present application, based on the foregoing solution, the pruning configuration file includes processing information of a hopping structure; the pruning module includes: a second reading unit configured to read the pruning configuration processing information of the skip structure contained in the file; the second pruning unit is configured to prune all network layers of the skip structure included in the target network structure if the processing information indicates to process the skip structure, obtaining the pruned convolutional neural network model; the third pruning unit is configured to obtain the skippable network layer of the skipping structure included in the target network structure if the processing information indicates that the skipping structure is not to be processed, and The skippable network layer is pruned to obtain a pruned convolutional neural network model.

In an embodiment of the present application, based on the foregoing solution, the pruning module includes: a second determining unit configured to determine a network node for characterizing the output from the target network structure; a fourth pruning unit is configured to prune other network nodes included in the target network structure except the network nodes used to characterize the output based on the target pruning algorithm, to obtain a pruned convolutional neural network model.

In an embodiment of the present application, based on the foregoing solution, the pruning configuration file includes identification information of the target pruning algorithm; the identification and determination module includes: a third reading unit configured to read The identification information of the target pruning algorithm contained in the pruning configuration file; the third determining unit is configured to determine the target matching the identification information from at least two preset pruning algorithms according to the identification information Pruning algorithm.

In an embodiment of the present application, based on the foregoing solution, the pruning configuration file includes identification information of the target pruning algorithm; the identification and determination module includes: an identification unit, configured to use the identification algorithm to The convolutional neural network model is identified, the dependencies of the operators contained in the convolutional neural network models are obtained, and the dependencies of the operators are mapped to a directed acyclic graph; the obtaining unit is configured to be based on the The directed acyclic graph is used to obtain the target network structure.

In an embodiment of the present application, based on the foregoing solution, the data to be processed includes at least one of image data, voice data, and text data; the processing module includes: a first processing unit configured to: If the data to be processed is image data, then the image data is processed based on the pruned convolutional neural network model to obtain an image processing result; the second processing unit is configured to, if the data to be processed is speech data, process the image data based on the pruning The subsequent convolutional neural network model processes the voice data to obtain a voice processing result; the third processing unit is configured to, if the data to be processed is text data, perform processing on the text data based on the pruned convolutional neural network model. Process to get the word processing result.

In an embodiment of the present application, based on the foregoing solution, the pruning framework for processing the convolutional neural network model includes a user layer, an intermediate layer and a pruning layer; the user layer corresponds to a user module, and the user layer for obtaining the convolutional neural network model to be processed and a pruning configuration file matching the convolutional neural network model; the middle layer corresponds to the middle module, and the middle layer is used to determine based on the pruning configuration file Target pruning algorithm; the pruning layer corresponds to the underlying module, and the pruning layer is used to identify the network structure of the convolutional neural network model based on the recognition algorithm to obtain the target network structure, and based on the target pruning algorithm The target network structure is pruned to obtain a pruned convolutional neural network model.

In an embodiment of the present application, based on the foregoing solution, the apparatus further includes: an obtaining module configured to obtain an identification algorithm for identifying the network structure of the convolutional neural network model, and at least two algorithms for identifying the network structure of the convolutional neural network model. A pruning algorithm for pruning the target network structure; a construction and generation module is configured to construct and generate the pruning framework based on the identification algorithm and at least two pruning algorithms.

In an embodiment of the present application, based on the foregoing solution, the construction and generation module includes: a construction unit, configured to construct a user layer, an intermediate layer and a pruning layer; a preset unit, configured to call the intermediate layer a first interface for presetting the at least two pruning algorithms in the middle layer, and calling a second interface of the pruning layer to preset the identification algorithm in the pruning layer; a generating unit, It is configured to generate the pruning framework through the user layer, the middle layer with preset pruning algorithms, and the pruning layer with preset recognition algorithms.

According to an aspect of the embodiments of the present application, the embodiments of the present application provide an electronic device, including one or more processors; and a storage device for storing one or more programs, when the one or more programs are stored When executed by the one or more processors, the electronic device implements the data processing method based on the convolutional neural network model as described above.

According to an aspect of the embodiments of the present application, the embodiments of the present application provide a computer-readable medium on which a computer program is stored, and when the computer program is executed by a processor, causes the computer to perform the above-mentioned convolution-based Data processing methods for neural network models.

According to an aspect of the embodiments of the present application, the embodiments of the present application provide a computer program product, including computer instructions, which, when executed by a processor, implement the above-mentioned data processing method based on a convolutional neural network model.

In the technical solutions provided by the embodiments of this application:

On the one hand, based on the recognition algorithm and the target pruning algorithm, the automatic pruning optimization of the convolutional neural network model to be processed is realized, so that the relevant workers do not need to manually select the appropriate pruning algorithm for pruning optimization, which not only improves the pruning efficiency , it also improves the pruning accuracy to a certain extent, and greatly reduces the workload of the relevant workers, bringing a lot of convenience to the relevant workers; correspondingly, due to the improvement of the convolutional neural network to be processed The pruning efficiency and accuracy of the model, then the processing efficiency and accuracy of the data to be processed based on the pruned-optimized convolutional neural model can also be improved.

On the one hand, the target pruning algorithm is independent of the recognition algorithm, that is, the target pruning algorithm and the recognition algorithm are isolated; therefore, in terms of the target pruning algorithm including the recognition algorithm in the related art, it is no longer necessary to be included in the target pruning algorithm. When the recognition algorithm cannot recognize the convolutional neural network model, the recognition algorithm included in the target pruning algorithm needs to be modified every time to support the recognition of the convolutional neural network model. The coupling is low and can be applied to a wider range of convolutional neural networks. In the data processing application scenario of the model.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not limiting of the present application.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description serve to explain the principles of the application. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative effort.

FIG. 1 is a schematic diagram of an exemplary implementation environment to which the technical solutions of the embodiments of the present application can be applied.

FIG. 2 is a flowchart of a data processing method based on a convolutional neural network model according to an exemplary embodiment of the present application.

FIG. 3 is a flowchart of step S202 in the embodiment shown in FIG. 2 in an exemplary embodiment.

FIG. 4 is a schematic diagram of a directed acyclic graph according to an exemplary embodiment of the present application.

FIG. 5 is a flowchart of step S202 in the embodiment shown in FIG. 2 in an exemplary embodiment.

FIG. 6 is a flowchart of step S203 in the embodiment shown in FIG. 2 in an exemplary embodiment.

FIG. 7 is a flowchart of step S203 in the embodiment shown in FIG. 2 in an exemplary embodiment.

FIG. 8 is a flowchart of step S203 in the embodiment shown in FIG. 2 in an exemplary embodiment.

FIG. 9 is a schematic diagram of a pruning frame shown in an exemplary embodiment of the present application.

FIG. 10 is a schematic diagram of correlation dependencies between nodes of a convolutional neural network model according to an exemplary embodiment of the present application.

FIG. 11 is a schematic diagram of a jump structure shown in an exemplary embodiment of the present application.

FIG. 12 is a block diagram of a data processing apparatus based on a convolutional neural network model according to an exemplary embodiment of the present application.

FIG. 13 is a schematic structural diagram of a computer system suitable for implementing the electronic device of the embodiment of the present application.

Detailed ways

The description will now be made in detail of exemplary embodiments, examples of which are illustrated in the accompanying drawings. Where the following description refers to the drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the following exemplary examples are not intended to represent all implementations identical to the present application. Rather, they are merely examples of apparatus and methods that have some of the same aspects of the present application as recited in the appended claims.

The block diagrams shown in the figures are merely functional entities and do not necessarily necessarily correspond to physically separate entities. That is, these functional entities may be implemented in software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices entity.

The flowcharts shown in the figures are only exemplary illustrations and do not necessarily include all contents and operations/steps, nor do they have to be performed in the order described. For example, some operations/steps can be decomposed, and some operations/steps can be combined or partially combined, so the actual execution order may be changed according to the actual situation.

It should be noted that the "plurality" mentioned in this application refers to two or more. "And/or" describes the association relationship between associated objects, indicating that there can be three kinds of relationships, for example, A and/or B can indicate that A exists alone, A and B exist at the same time, and B exists alone. The character "/" generally indicates that the associated objects are an "or" relationship.

Artificial intelligence (AI) is a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technique of computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that the machines have the functions of perception, reasoning and decision-making.

Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, including both hardware-level technology and software-level technology. The basic technologies of artificial intelligence generally include technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation/interaction systems, and mechatronics. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning/deep learning, autonomous driving, and smart transportation.

Machine Learning (ML) in artificial intelligence technology is a multi-domain interdisciplinary subject involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in how computers simulate or realize human learning behaviors to acquire new knowledge or skills, and to reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent, and its applications are in all fields of artificial intelligence. Machine learning and deep learning usually include neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, teaching learning and other techniques.

The data processing scheme based on the convolutional neural network model in the related art usually firstly trains the convolutional neural network model, then performs pruning and optimization based on the trained convolutional neural network model, and then performs pruning and optimization based on the convolutional neural network model after the pruning optimization. The neural network model performs corresponding data processing. Among them, in the process of pruning optimization, relevant workers usually manually select a suitable pruning algorithm for pruning optimization, but such a pruning optimization method is inefficient, which makes the convolutional neural network based on pruning optimization. The network model is also less efficient for corresponding data processing.

Therefore, in a data processing scenario based on a convolutional neural network model, an embodiment of the present application proposes a data processing solution based on a convolutional neural network model. Please refer to FIG. 1 , which is a schematic diagram of an exemplary implementation environment of the present application. The implementation environment includes a terminal device 101 and a server 102, and the terminal device 101 and the server 102 communicate through a wired or wireless network.

It should be understood that the numbers of terminal devices 101 and servers 102 in FIG. 1 are only illustrative. According to actual needs, there may be any number of terminal devices 101 and servers 102 .

Wherein, the terminal device 101 corresponds to a client, which can be any electronic device with a user input interface, including but not limited to smart phones, tablets, laptops, computers, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, aircraft, etc., The user input interface includes, but is not limited to, a touch screen, a keyboard, physical keys, an audio pickup device, and the like.

The server 102 corresponds to a server, which can be a server providing various services, an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud service, cloud Basic cloud computing services such as databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms Cloud server, which is not limited here.

In some embodiments of the present application, the data processing method based on the convolutional neural network model may be executed by the terminal device 101 , and accordingly, the data processing apparatus based on the convolutional neural network model is configured in the terminal device 101 . Optionally, the terminal device 101 can obtain the convolutional neural network model to be processed and the pruning configuration file matching the convolutional neural network model, and then identify the network structure of the convolutional neural network model based on the identification algorithm to obtain the target network. The target pruning algorithm is determined based on the pruning configuration file, in which the target pruning algorithm is independent of the recognition algorithm, and then the target network structure is pruned based on the target pruning algorithm to obtain the pruned convolutional neural network model. The data to be processed is processed based on the pruned convolutional neural network model, and the processing result is obtained.

In some embodiments of the present application, the data processing method based on the convolutional neural network model may be executed by the server 102 , and accordingly, the data processing apparatus based on the convolutional neural network model is configured in the server 102 . Optionally, the server 102 can obtain the convolutional neural network model to be processed and the pruning configuration file matching the convolutional neural network model, and then identify the network structure of the convolutional neural network model based on the identification algorithm to obtain the target network structure. , and determine the target pruning algorithm based on the pruning configuration file, where the target pruning algorithm is independent of the recognition algorithm, and then prune the target network structure based on the target pruning algorithm to obtain the pruned convolutional neural network model. The pruned convolutional neural network model processes the data to be processed to obtain the processing result.

By implementing the technical solutions of the embodiments of the present application, automatic pruning and optimization of the convolutional neural network model to be processed is realized, the pruning efficiency and accuracy of the convolutional neural network model to be processed are improved, and correspondingly, the The processing efficiency and accuracy of the data to be processed based on the convolutional neural model optimized by pruning; at the same time, the target pruning algorithm and the recognition algorithm are isolated, and the coupling is low, which can greatly reduce the amount of modification, thereby reducing related work It brings a lot of convenience to related workers and can be applied to a wider range of data processing application scenarios based on the convolutional neural network model.

Various implementation details of the technical solutions of the embodiments of the present application are described in detail below:

Please refer to FIG. 2. FIG. 2 is a flowchart of a data processing method based on a convolutional neural network model shown in an embodiment of the present application. The data processing method based on a convolutional neural network model can be performed by the terminal shown in FIG. 1 device 101 or server 102 to execute. As shown in FIG. 2, the data processing method based on the convolutional neural network model includes at least steps S201 to S204, and the details are as follows:

Step S201, acquiring a convolutional neural network model to be processed and a pruning configuration file matching the convolutional neural network model.

The convolutional neural network model to be processed in the embodiment of the present application refers to the convolutional neural network model that has completed training; the completion of the training may refer to the completion of one or more rounds (two or more rounds) of training, which can be understood The point is, to complete multiple rounds of training is usually to perform multiple rounds of training on the convolutional neural network model until it converges to obtain a trained convolutional neural network model.

The pruning configuration file matched by the convolutional neural network model in the embodiment of the present application refers to the pruning configuration file obtained by the relevant administrator/user/developer who needs to perform pruning and optimization on the convolutional neural network model, thereby configuring. For example, if manager 1 needs to prune and optimize the convolutional neural network model A, it can be configured to obtain a pruning configuration file 1. The pruning configuration file 1 is called the pruning that matches the convolutional neural network model A. configuration file.

Step S202, the network structure of the convolutional neural network model is identified based on the identification algorithm to obtain the target network structure, and the target pruning algorithm is determined based on the pruning configuration file, wherein the target pruning algorithm is independent of the identification algorithm.

In the embodiment of the present application, the convolutional neural network model to be processed and the pruning configuration file matching the convolutional neural network model are obtained, and then the network structure of the convolutional neural network model can be identified based on the identification algorithm to obtain the target network structure, And the target pruning algorithm is determined based on the pruning configuration file, wherein the target pruning algorithm is independent of the recognition algorithm.

It can be understood that when pruning and optimizing the convolutional neural network model, it is based on the network structure of the convolutional neural network model; therefore, in the embodiment of the present application, it is necessary to prune the network structure of the convolutional neural network model based on the recognition algorithm. For identification, in order to facilitate understanding, the identified network structure of the convolutional neural network model is referred to as the target network structure in the embodiments of the present application. It should be noted that the network structure of the convolutional neural network model includes, but is not limited to, which network layers the convolutional neural network model corresponds to, how each network layer is connected, and the number of convolution kernels in the convolutional layer.

In an embodiment of the present application, please refer to FIG. 3 , in step S202, the network structure of the convolutional neural network model is identified based on the identification algorithm, and the process of obtaining the target network structure may include the following steps S301 to S302. The detailed introduction as follows:

Step S301, identify the convolutional neural network model by an identification algorithm, obtain the dependencies of the operators contained in the convolutional neural network model, and map the dependencies of the operators into a directed acyclic graph;

In step S302, the target network structure is obtained based on the directed acyclic graph.

That is, in an optional embodiment, the convolutional neural network model can be identified by an identification algorithm, the dependencies of the operators contained in the convolutional neural network model can be obtained, and the dependencies of the operators can be mapped to a directed acyclic graph. , and then the target network structure is obtained based on the directed acyclic graph.

Among them, it is understandable that a large number of operators (also called network nodes) are usually included in the convolutional neural network model. In fact, the operator can be simply regarded as part of the algorithm process in the convolutional neural network model. When a function is mapped to a function, or a function is mapped to a number, the operators have mutual dependencies, and the dependencies can be simply understood as data dependencies.

For example, in the convolutional neural network model, there are operator a, operator b, operator c, operator d, operator e, operator f, operator g and operator h. Among them, let the convolutional neural network model define that the input data of operator d is the output data of operator b, and the input data of operator b is the output data of operator a, and operator a has only data output; then it can be It is known that the operation of operator d requires the output data of operator b, that is, operator d depends on operator b. Similarly, the operation of operator b requires the output data of operator a, then calculate Sub b is dependent on operator a. At the same time, the convolutional neural network model also defines that the output data of operator c is the input data of operator f, the input data of operator e is the output data of operator a, and the input data of operator f is derived from the operator f. In addition to operator c, it also comes from operator b, the input data of operator g comes from operator b, operator c and operator e, and the input data of operator h comes from operator d and operator f; then we can get It is known that operator f depends on operator b and operator c at the same time, operator e depends on operator a, operator g depends on operator b, operator c and operator e at the same time, and operator h also depends on operator Sub d and operator f.

Therefore, in an optional embodiment, the convolutional neural network model may be identified by an identification algorithm, so as to obtain the dependencies of the operators contained in the convolutional neural network model.

Among them, it can be understood that a directed acyclic graph (Directed Acyclic Graph) refers to a directed graph in graph theory that cannot start from a vertex and go back to the vertex through several edges. Like the blockchain, it is also a data structure.

By way of example, the directed acyclic graph shown in FIG. 4 can be obtained by following the dependency relationship shown in the foregoing example. There are node 0, node 1, node 2, node 3, node 4, node 5, node 6 and node 7 in the directed acyclic graph; where node 0 represents operator a, node 1 represents operator b, and node 2 represents operator b Operator c, node 3 represents operator d, node 4 represents operator e, node 5 represents operator f, node 6 represents operator g, and node 7 represents operator h. It is understood that the node to which the arrow points depends on the starting node of the arrow.

Therefore, in an optional embodiment, the target network structure may be obtained based on the obtained directed acyclic graph.

In this way, by implementing the optional embodiment, the target network structure can be obtained quickly and easily, thereby improving the efficiency of obtaining the pruned convolutional neural network model based on the target network structure, and treating the pruned convolutional neural network model based on the Data processing provides support.

It can be understood that when pruning and optimizing the convolutional neural network model, it is based on the pruning algorithm; therefore, in the embodiment of the present application, the target pruning algorithm needs to be determined based on the pruning configuration file. The pruning algorithm for pruning and optimizing the convolutional neural network model in the embodiment is called a target pruning algorithm.

In an embodiment of the present application, the identification information of the target pruning algorithm may be included in the pruning configuration file; please refer to FIG. 5 , the process of determining the target pruning algorithm based on the pruning configuration file in step S202 may include the following steps S501 To step S502, the details are as follows:

Step S501, read the identification information of the target pruning algorithm contained in the pruning configuration file;

Step S502: Determine a target pruning algorithm matching the identification information from at least two preset pruning algorithms according to the identification information.

That is, in an optional embodiment, the identification information of the target pruning algorithm may be read from the pruning configuration file first, and then the target pruning algorithm that matches the identification information is determined from at least two preset pruning algorithms according to the identification information. branch algorithm.

Wherein, the identification information of the target pruning algorithm in the optional embodiment is used to uniquely identify the pruning algorithm, which includes but is not limited to the name of the target pruning algorithm, a specific identification number, and the like.

Among them, it can be understood that, because different pruning algorithms have their own advantages and disadvantages; therefore, in an optional embodiment, relevant managers/users/developers can specify in the pruning configuration file to perform pruning optimization on the convolutional neural network model pruning algorithm (namely target pruning algorithm) to achieve autonomous selection.

For example, suppose that the read identification information of the target pruning algorithm is "XX", and 10 kinds of pruning algorithms 0-9 are preset at the same time, then according to the identification information "XX", 10 kinds of pruning algorithms 0-9 are set. Determine the target pruning algorithm that matches the identification information "XX", and suppose that the pruning algorithm 2 of the 10 pruning algorithms 0-9 matches the identification information "XX", then the pruning algorithm 2 is the target pruning algorithm.

In this way, by implementing the optional embodiment, a target pruning algorithm can be obtained quickly and easily, thereby improving the efficiency of obtaining a pruned convolutional neural network model based on the target pruning algorithm, which is a pruned-based convolutional neural network model. Provide support for processing data to be processed.

In an embodiment of the present application, the pruning configuration file may also include a target pruning algorithm, that is, the pruning configuration file contains the target pruning algorithm itself, then the target pruning algorithm can be directly read from the pruning configuration file. The pruning algorithm is then used to prune the target network structure based on the target pruning algorithm to obtain the pruned convolutional neural network model.

It should be noted that the target pruning algorithm in the embodiment of the present application is independent of the identification algorithm, that is, the target pruning algorithm and the identification algorithm are isolated. In the related art, the recognition algorithm is included in the pruning algorithm. It can be understood that due to the different network structures of different convolutional neural network models, only the recognition algorithm that matches the network structure can recognize the network structure; so the actual situation Yes, if the recognition algorithm contained in the pruning algorithm cannot recognize the network structure of the convolutional neural network model, then the recognition algorithm contained in the pruning algorithm needs to be modified, and the modification will also affect the pruning to a certain extent. Therefore, it should bring some trouble to the relevant managers/users/developers, and if the number of pruning algorithms is larger, the identification algorithms contained in it must be able to identify the network structure of the convolutional neural network model, then The amount of modification is very large. Therefore, in the embodiment of the present application, considering the existing problems, the target pruning algorithm and the identification algorithm are isolated, so that when using or modifying, it will bring a lot of convenience to the relevant managers/users/developers, reducing the Its workload and corresponding troubles save time and cost, thereby improving the data processing efficiency based on the pruned convolutional neural network model.

Step S203 , prune the target network structure based on the target pruning algorithm to obtain a pruned convolutional neural network model.

In the embodiment of the present application, the convolutional neural network model to be processed and the pruning configuration file matching the convolutional neural network model are obtained, and then the network structure of the convolutional neural network model can be identified based on the identification algorithm to obtain the target network structure, And the target pruning algorithm is determined based on the pruning configuration file, wherein the target pruning algorithm is independent of the recognition algorithm.

In an embodiment of the present application, the pruning configuration file includes a first pruning rate; please refer to FIG. 6 , in step S203, the target network structure is pruned based on the target pruning algorithm to obtain a pruned convolutional neural network The model process may include the following steps S601 to S604, which are described in detail as follows:

Step S601, read the first trimming rate contained in the pruning configuration file;

Step S602, determining the second trimming rate corresponding to each network layer included in the target network structure based on the first trimming rate;

Step S603, according to the second pruning rate corresponding to each network layer, respectively prune each network layer through the target pruning algorithm to obtain each network layer after pruning;

Step S604, a pruned convolutional neural network model is generated based on the pruned network layers.

That is, in an optional embodiment, the first trimming rate contained in the pruning configuration file may be read first, and then the second trimming rate corresponding to each network layer included in the target network structure is determined based on the first trimming rate, and then the second trimming rate corresponding to each network layer included in the target network structure may be determined based on the first trimming rate. The second pruning rate corresponding to the network layer is to prune each network layer through the target pruning algorithm to obtain the pruned network layers, and then generate the pruned convolutional neural network based on the pruned network layers. Model.

Wherein, in an optional embodiment, the target network structure includes multiple (two or more) network layers, such as but not limited to multiple convolution layers, batch normalization layers, fully connected layers, and splicing layers, etc. .

Wherein, in an optional embodiment, the relevant manager/user/developer may specify the pruning rate (also called pruning rate or compression rate) for pruning and optimizing the convolutional neural network model in the pruning configuration file. It is understood that, in an optional embodiment, the pruning rate contained in the pruning configuration file is referred to as the first pruning rate. The cropping rate refers to the cropping ratio, for example, it may take a value of 50%, 30%, etc., and the specific cropping rate (ie, the first cropping rate) read from the pruning configuration file shall prevail.

Wherein, each network layer included in the target network structure in the optional embodiment has a corresponding cropping rate. For ease of understanding, the cropping rate corresponding to each network layer is referred to as the second cropping rate in the optional embodiment.

In this way, by implementing the optional embodiment, each network layer can be pruned quickly and easily, and a pruned convolutional neural network model can be generated based on each pruned network layer, which is based on the pruned convolutional neural network. The network model provides support for processing the data to be processed.

In an embodiment of the present application, the process of determining the second cropping rate corresponding to each network layer included in the target network structure based on the first cropping rate in step S602 may include the following two situations:

In case 1, if the first cropping rate represents the global cropping rate, the second cropping rate corresponding to each network layer included in the target network structure is determined based on the pruning prior knowledge information and the first cropping rate.

That is, in an optional embodiment, if the first cropping rate represents the global cropping rate, then each network layer included in the target network structure may be determined based on the prior knowledge information of pruning and the first cropping rate representing the global cropping rate. The corresponding second cropping rate.

It can be understood that, in the optional embodiment, based on the prior knowledge information of pruning and the first trimming rate representing the global trimming rate, the second trimming rate corresponding to all network layers included in the target network structure is determined. It can be seen as fully automated pruning.

For example, for example, the first cropping rate is set to 50%, and each network layer included in the target network structure is set to be convolutional layer 1, convolutional layer 2, convolutional layer 3, batch normalization layer, fully connected layer and splicing layer; then the second cropping rate corresponding to each network layer can be determined based on the pruning prior knowledge information and the first cropping rate of 50%. For example, the second cropping rate corresponding to convolutional layer 1 can be determined to be 55%, The second cropping rate corresponding to the accumulation layer 2 is 50%, the second cropping rate corresponding to the convolutional layer 3 is 45%, the second cropping rate corresponding to the batch normalization layer is 55%, and the second cropping rate corresponding to the fully connected layer The rate is 50%, and the second cropping rate corresponding to the stitching layer is 60%.

In case 2, if the first cropping rate represents the cropping rate of the specified network layer included in the target network structure, the second cropping rate corresponding to the specified network layer is determined based on the first cropping rate, and the first cropping rate is determined based on the pruning prior knowledge information and the first cropping rate. , and determine the second cropping rate corresponding to other network layers except the specified network layer included in the target network structure.

That is, in an optional embodiment, if the first cropping rate represents the cropping rate of the specified network layer included in the target network structure, then the specified network can be determined based on the first cropping rate representing the cropping rate of the specified network layer contained in the target network structure. The second cropping rate corresponding to the layer, and the first cropping rate based on the pruning prior knowledge information and characterizing the specified network layer cropping rate contained in the target network structure, determine other than the specified network layer contained in the target network structure. The second cropping rate corresponding to the network layer.

It can be understood that, in the optional embodiment, based on the pruning prior knowledge information and the first clipping rate representing the designated network layer clipping rate included in the target network structure, what is determined is a part of the network layer ( That is, the second pruning rate corresponding to other network layers (other than the specified network layer), which can be regarded as semi-automatic pruning.

Wherein, in an optional embodiment, the first cropping rate represents the cropping rate of a specified network layer included in the target network structure, and the specific form may be "layer identification-cropping rate", and the layer identification refers to the specific and unique identification of the network layer, which includes But not limited to the network layer name, layer identification number, etc.

For example, for example, the first cropping rate is set to 2-50% of the convolutional layer, and the network layers included in the target network structure are set to be convolutional layer 1, convolutional layer 2, convolutional layer 3, and batch normalization layer respectively. , the fully connected layer and the splicing layer; then it can be known that the convolution layer 2 is the designated convolution layer. At this time, the second cropping rate corresponding to the convolutional layer 2 can be determined based on the first cropping rate of the convolutional layer 2-50%. 50%. Correspondingly, it is also necessary to determine the second cropping rate corresponding to each network layer except the convolutional layer 2 based on the pruning prior knowledge information and the first cropping rate convolutional layer 2-50%. The second cropping rate corresponding to layer 1 is 45%, the second cropping rate corresponding to convolutional layer 3 is 45%, the second cropping rate corresponding to batch normalization layer is 55%, and the second cropping rate corresponding to fully connected layer. is 50%, and the second cropping rate corresponding to the stitching layer is 55%.

In this way, by implementing the optional embodiment, the second pruning rate corresponding to each network layer can be quickly and easily determined, so as to realize fully automatic or semi-automatic pruning.

In an embodiment of the present application, in step S603, according to the second clipping rate corresponding to each network layer, the target pruning algorithm is used to prune each network layer respectively, and the process of obtaining the pruned network layers may include: The following two situations:

In case 1, if the target pruning algorithm is a structured pruning algorithm, the feature maps corresponding to each network layer are deleted based on the structured pruning algorithm, so as to prune each network layer and obtain a structured pruning algorithm. Each network layer after the branch.

That is, in an optional embodiment, if the target pruning algorithm is a structured pruning algorithm, the feature maps corresponding to each network layer may be deleted based on the structured pruning algorithm, so as to realize the pruning of each network layer. Pruning to obtain each network layer after structured pruning.

Among them, it can be understood that in the optional embodiment, structured pruning (structured pruning) simply means removing a certain neuron; therefore, if structured pruning is to be performed in the optional embodiment, it is based on the structure The optimized pruning algorithm deletes the feature maps (corresponding to the convolution kernels) corresponding to each network layer, which is equivalent to deleting a certain neuron.

Wherein, the structured pruning algorithm in the optional embodiment includes, but is not limited to, the L1 norm pruning algorithm, the FPGM (Filter Pruning via Geometric Median) pruning algorithm, the network slimming pruning algorithm, and the like.

In case 2, if the target pruning algorithm is an unstructured pruning algorithm, then based on the unstructured pruning algorithm, the eigenvalues contained in the feature maps corresponding to each network layer are set to zero respectively, so as to perform zero processing on each network layer. Pruning to get each network layer after unstructured pruning.

That is, in an optional embodiment, if the target pruning algorithm is an unstructured pruning algorithm, the feature values contained in the feature maps corresponding to each network layer may be reset to zero based on the unstructured pruning algorithm, respectively. Thus, the pruning of each network layer is realized to obtain each network layer after unstructured pruning.

Among them, it can be understood that in the optional embodiment, unstructured pruning simply means removing the connections (ie weights) between neurons; therefore, in the optional embodiment, if the unstructured pruning is to be Structural pruning, then based on the unstructured pruning algorithm, the eigenvalues contained in the feature map corresponding to each network layer (which corresponds to the convolution kernel) are zeroed, which is equivalent to removing the neuron. connection between.

In this way, by implementing the optional embodiment, it is possible to flexibly choose whether to perform structured pruning or unstructured pruning on the convolutional neural network model, with higher flexibility.

In an embodiment of the present application, the pruning configuration file includes the processing information of the skip structure; please refer to FIG. 7 , in step S203, the target network structure is pruned based on the target pruning algorithm, and the pruned convolutional neural network is obtained. The process of the network model may include the following steps S701 to S703, which are described in detail as follows:

Step S701, read the processing information of the jump structure contained in the pruning configuration file;

Step S702, if the processing information indicates to process the jump structure, then prune all network layers of the jump structure included in the target network structure to obtain a pruned convolutional neural network model;

Step S703, if the processing information indicates that the skipping structure is not to be processed, obtain the skippable network layer of the skipping structure included in the target network structure, and prune the skippable network layer to obtain a pruned convolutional neural network model.

That is, in an optional embodiment, the processing information of the skip structure contained in the pruning configuration file may be read first, and then corresponding pruning is performed according to the situation indicated by the processing information, so as to obtain a pruned convolutional neural network model.

Among them, in an optional embodiment, if the situation indicated by the processing information is to process the skip structure, then all network layers of the skip structure included in the target network structure may be pruned, so as to obtain the pruned convolutional neural network model. . It can be understood that if all network layers of the skip structure included in the target network structure are pruned, compared to pruning only the skippable network layers of the skip structure included in the target network structure, the pruned volume The product neural network model will take up less space and the processing speed will be faster.

Among them, in an optional embodiment, if the situation indicated by the processing information is that the skip structure is not processed, then the skippable network layer of the skip structure included in the target network structure can be obtained, and the skippable network layer can be pruned, so as to obtain the pruned network layer. A post-branch convolutional neural network model. It can be understood that if only the skippable network layers of the skip structure included in the target network structure are pruned, compared to pruning all the network layers of the skip structure included in the target network structure, the pruned volume The accuracy of the product neural network model will be higher.

In this way, by implementing the optional embodiment, it is possible to flexibly choose to process or not to process the jumping structure included in the convolutional neural network model, and the degree of flexibility is higher.

In an embodiment of the present application, the pruning configuration file includes the processing information of the jump structure; please refer to FIG. 8 , in step S203, the target network structure is pruned based on the target pruning algorithm, and the pruned convolutional neural network is obtained. The process of the network model may include the following steps S801 to S802, which are described in detail as follows:

Step S801, determining the network node used to characterize the output from the target network structure;

Step S802 , prune other network nodes included in the target network structure except the network nodes used to characterize the output based on the target pruning algorithm, to obtain a pruned convolutional neural network model.

That is, in an optional embodiment, the network nodes used to characterize the output may be determined from the target network structure first, and then other networks included in the target network structure except the network nodes used to characterize the output may be determined based on the target pruning algorithm. The nodes are pruned to obtain the pruned convolutional neural network model.

Wherein, in an optional embodiment, the network node used to characterize the output is determined from the target network structure, which may be determined through the directed acyclic graph described above, and other arbitrary methods may also be used in other embodiments.

Wherein, in an optional embodiment, the network node used for characterizing the output is determined from the target network structure, and the network node characterizing the output can be marked at the same time, so as to distinguish the network node characterizing the output from other network nodes.

For example, if there are 50 network nodes included in the target network structure, which are 0-49 respectively, and there are 5 network nodes determined to represent the output, which are network nodes 10-15, then other networks The nodes are 0-9 and 16-49. At this time, based on the target pruning algorithm, other network nodes are pruned from 0-9 and 16-49 to obtain the pruned convolutional neural network model.

In this way, by implementing the optional embodiment, since the network nodes representing the output are not pruned, the output of the pruned convolutional neural network model is guaranteed to be correct and complete, and the pruning of the network nodes representing the output is avoided. The phenomenon that the pruned convolutional neural network model cannot be output or the output is wrong, so that the accuracy of the pruned convolutional neural network model is not greatly affected, which balances the pruned convolutional neural network model. Footprint, processing speed, and precision.

Step S204, processing the data to be processed based on the pruned convolutional neural network model to obtain a processing result.

In the embodiment of the present application, the target network structure is pruned based on the target pruning algorithm to obtain a pruned convolutional neural network model, and then the data to be processed can be processed based on the pruned convolutional neural network model to obtain a processing result .

The data to be processed in this embodiment of the present application includes, but is not limited to, image data, voice data, and text data; optionally, the data to be processed may include any one or more of image data, voice data, and text data.

In an embodiment of the present application, if the data to be processed is image data, the image data is processed based on the pruned convolutional neural network model, thereby obtaining an image processing result.

In an embodiment of the present application, if the data to be processed is speech data, the speech data is processed based on the pruned convolutional neural network model, thereby obtaining a speech processing result.

In an embodiment of the present application, if the data to be processed is text data, the text data is processed based on the pruned convolutional neural network model, thereby obtaining a text processing result.

In this way, by implementing the optional embodiment, a variety of data to be processed can be processed based on the pruned convolutional neural network model, which is suitable for many application scenarios and has a wide range of applications.

In the embodiment of the present application, automatic pruning and optimization of the convolutional neural network model to be processed is realized, the pruning efficiency and accuracy of the convolutional neural network model to be processed are improved, and correspondingly, the pruning-based optimization is also improved. The processing efficiency and accuracy of the data to be processed by the convolutional neural model; at the same time, the target pruning algorithm and the recognition algorithm are isolated, and the coupling is low, which can greatly reduce the amount of modification, thereby reducing the workload of related workers, It brings many conveniences to related workers and can be applied to a wider range of data processing application scenarios based on convolutional neural network models.

An embodiment of the present application proposes a pruning framework for processing a convolutional neural network model; please refer to FIG. 9 , which is a schematic structural diagram of a pruning framework according to an embodiment of the present application. As shown in Figure 9, the pruning framework includes a user layer 901, an intermediate layer 902 and a pruning layer 903, wherein:

User layer 901, used to obtain the convolutional neural network model to be processed, and the pruning configuration file matched with the convolutional neural network model;

The middle layer 902 is used to determine the target pruning algorithm based on the pruning configuration file;

The pruning layer 903 is used to identify the network structure of the convolutional neural network model based on the identification algorithm to obtain the target network structure, and to prune the target network structure based on the target pruning algorithm to obtain the pruned convolutional neural network Model.

In an embodiment of the present application, the data processing method based on the convolutional neural network model may further include: acquiring an identification algorithm for identifying the network structure of the convolutional neural network model, and at least two methods for identifying the target network A pruning algorithm for structure pruning; a pruning framework is constructed and generated based on the recognition algorithm and at least two pruning algorithms.

That is, in the optional embodiment, the identification algorithm is used to identify the network structure of the convolutional neural network model, and the pruning algorithm is used to prune the target network structure, that is, the pruning algorithm and the identification algorithm are separated from each other, respectively. Obtain the recognition algorithm and pruning algorithm, and build and generate a pruning framework based on the recognition algorithm and pruning algorithm.

In an embodiment of the present application, a pruning framework is constructed and generated based on an identification algorithm and at least two pruning algorithms, which may specifically include: constructing a user layer, an intermediate layer, and a pruning layer;

Call the first interface of the middle layer to preset at least two pruning algorithms in the middle layer, and call the second interface of the pruning layer to preset the recognition algorithm in the pruning layer; The middle layer of the algorithm and the pruning layer preset with the recognition algorithm generate the pruning framework.

That is, in an optional embodiment, at least two pruning algorithms can be preset in the middle layer by calling the first interface provided by the middle layer, and the recognition algorithm can be preset by calling the second interface of the pruning layer. in the pruning layer.

Before using the pruning framework in the embodiment of the present application, the pruning framework may be packaged as a library, and then the pruning framework is installed and imported through the PIP protocol/tool of the Python language.

For each layer in the pruning framework, the details are as follows:

Among them, in the optional embodiment, the user layer 901 refers to the input part of the entire pruning framework. In order to use the pruning framework, the relevant manager/user/developer needs to provide the trained convolutional neural network model file and the pruning framework. stick configuration file.

Optionally, the pruning configuration file may have a total of three configuration items, which are the identification information of the target pruning algorithm, the processing information of the jump structure, and the first trimming rate (the pruning strategy can be determined by the representation of the first trimming rate, Alternatively, the pruning configuration file may contain a pruning strategy in addition to the aforementioned three configuration items); specifically: the identification information of the target pruning algorithm is to inform the pruning framework to prune and optimize the convolutional neural network model. The target pruning algorithm that should be used. The pruning of the skip structure will reduce the accuracy of the convolutional neural network model to a certain extent. Therefore, the pruning framework selection in this embodiment of the present application allows the relevant managers/users/developers to choose whether to process the skip structure. The pruning strategy is divided into fully automatic pruning strategy and semi-automatic pruning strategy. The fully automatic strategy means that the first pruning rate represents the global pruning rate, and the second pruning rate corresponding to each network layer is automatically specified by the pruning framework; The pruning strategy allows the relevant managers/users/developers to specify the pruning rate of the network layer using key-value pairs such as "layer ID-pruning rate", while the second pruning rate corresponding to the unspecified network layer is still determined by the pruning rate. The branch frame is automatically specified.

Wherein, in an optional embodiment, the middle layer 902 is provided with a reader, a policyr, and a pruning algorithm library.

Optionally, after the model file and the pruning configuration file are sent to the reader, the reader can obtain the pruning strategy by reading the pruning configuration file, for example, by reading the pruning rate contained in the pruning configuration file. The pruning strategy is determined by characterizing the situation, or obtained by reading the pruning strategy contained in the pruning configuration file. It can be understood that the pruning strategy read by the reader may have a fully automatic pruning strategy or a semi-automatic pruning strategy.

Optionally, the policer may determine whether the current pruning strategy is a fully automatic pruning strategy or a semi-automatic pruning strategy according to the content in the reader. Among them, if it is a fully automatic pruning strategy, the strategy device analyzes the network structure of the convolutional neural network model (here is just a simple analysis to determine which network layers are in the convolutional neural network model), and combines the previous pruning priors. The knowledge information and the first cropping rate are used to set the second cropping rate for each network layer. Among them, if it is a semi-automatic pruning strategy, the strategy device performs basically the same operation as the fully automatic pruning strategy, the difference is that the second pruning rate of some network layers (that is, the specified network layer) is specified in the pruning configuration file. of.

Optionally, when the second cropping rate of each network layer is determined, the target pruning algorithm that matches the identification information of the target pruning algorithm contained in the pruning configuration file can be selected from the pruning algorithm library to the convolutional neural network. The network model is pruned and optimized. Among them, structured pruning algorithms such as L1 norm pruning algorithm, FPGM pruning algorithm, and network slimming pruning algorithm are encapsulated in the pruning algorithm library. In other embodiments, unstructured pruning algorithms may be encapsulated simultaneously or separately ; It is understandable that the input of these structured pruning algorithms is the specified network layer, and the output is the operator index to be removed in the network layer. Correspondingly, the intermediate layer obtains the operator index to be removed, and then can send the index content (ie, the operator index to be removed) to the pruning layer, and then the pruning layer performs the real pruning operation.

Optionally, the pruning algorithm library also reserves a first interface (that is, a developer interface) for developers to encapsulate pruning algorithms for the pruning framework, including but not limited to adding pruning algorithms, deleting pruning algorithms, modifying Pruning algorithm and query pruning algorithm, etc.

Among them, in an optional embodiment, the pruning layer 903 can mainly realize four functions of operator support, calculation graph construction, jump structure identification, and pruning protection.

Optionally, operator support refers to support for pruning each network layer of the current convolutional neural network model, including multiple convolutional layers, batch normalization layers, fully connected layers, and splicing and other network layers. It can be understood that, in order to implement structured pruning, the implementation supported by the operator may specifically be to create a new network layer and copy the parameters of the non-redundant operators in the original network layer to the new network layer.

Optionally, the construction of the computational graph (ie, directed acyclic graph) can utilize the identification algorithm of the chain rule of the convolutional neural network. Please refer to Figure 10. The convolutional neural network model includes input nodes, hidden nodes, and output nodes. There is a dependency relationship between each node. Based on the dependency relationship, a computational graph can be constructed, and then the target network structure can be obtained based on the computational graph. Optionally, the pruning layer 903 further reserves a second interface (ie, a developer interface) for the developer to encapsulate the identification algorithm for the pruning framework.

Optionally, hopping structure identification, please refer to FIG. 11 , which is an exemplary hopping structure, in which path A and path B intersect at the same point in the recursive process, specifically including network layers 1-3, network layer 1 can jump to After the network layer 3, then the network layer 2 is the skippable network layer. If the processing information of the hopping structure contained in the pruning configuration file indicates that the hopping structure is to be processed, the network layers 1-3 are pruned, and it is ensured that the pruned network layers 1-3 can still perform the addition operation; If the processing information of the jump structure contained in the pruning configuration file indicates that the jump structure is not to be processed, the network layer 2 is pruned, but the network layer 1 and network 3 do not need to be pruned. Similarly, ensure that the pruning is performed. The added network layer 2 and the unpruned network layer 1 and network 3 can still perform the addition operation. In this way, by identifying the jumping structure, it is possible to flexibly choose to process or not to process the jumping structure included in the convolutional neural network model, and the flexibility is higher.

Optionally, for pruning protection, the network node used to characterize the output can be determined from the target network structure, and the network node representing the output can be marked. Correspondingly, the marked network node does not need to be pruned, Instead, only unlabeled network nodes are pruned. In this way, the correct pruning result can be obtained, which ensures that the output of the pruned convolutional neural network model is correct and complete.

Optionally, for the specific implementation process of each layer in the pruning framework, please refer to the technical solutions in the foregoing embodiments, which will not be repeated here.

It should be noted that the pruning framework in the embodiment of the present application is not only suitable for simple classification tasks, but also for complex target detection tasks and visual tasks such as semantic segmentation, and can provide high support for visual tasks. Among them, in the classification task, the pruning framework in the embodiment of the application supports ResNet (residual network, Residual Network) and MobileNet (separable convolution) and other networks; in the target detection task, the pruning in the embodiment of the application The pruning framework supports the whole series of YOLO (object detection) and CenterNet and other networks; in the semantic segmentation task, the pruning framework in the embodiment of this application supports PSPNet (Pyramid Scene Parsing Network, semantic segmentation) and U-Net (Convolutional Networks for Biomedical Image). Segmentation) and other networks. Please refer to Table 1. In the PASCAL VOC2012+2017 dataset, the pruning framework in this example can reduce the parameters of the YOLO-V5 model by 80.1%, and mAP by only 0.4%.

Model parameter quantity Accuracy (mAP@0.5) YOLO-V5 model 21114417 0.824 YOLO-V5 model after pruning 4201768 0.820

Table 1

In the embodiment of the present application, the pruning optimization of the convolutional neural network model is realized by applying the pruning framework, which not only has high pruning and optimization efficiency, but also can be applied to many application scenarios and has higher flexibility.

FIG. 12 is a block diagram of a data processing apparatus based on a convolutional neural network model according to an embodiment of the present application. As shown in Figure 12, the data processing device based on the convolutional neural network model includes:

Obtaining module 1201, configured to obtain a convolutional neural network model to be processed and a pruning configuration file matching the convolutional neural network model;

The identification and determination module 1202 is configured to identify the network structure of the convolutional neural network model based on the identification algorithm, obtain the target network structure, and determine the target pruning algorithm based on the pruning configuration file, wherein the target pruning algorithm is independent of the identification algorithm ;

The pruning module 1203 is configured to prune the target network structure based on the target pruning algorithm to obtain the pruned convolutional neural network model;

The processing module 1204 is configured to process the data to be processed based on the pruned convolutional neural network model to obtain a processing result.

In an embodiment of the present application, the pruning configuration file includes a first pruning rate; the pruning module 1203 includes:

a first reading unit, configured to read the first pruning rate contained in the pruning configuration file;

a first determining unit, configured to determine a second trimming rate corresponding to each network layer included in the target network structure based on the first trimming rate;

The first pruning unit is configured to prune each network layer through the target pruning algorithm according to the second pruning rate corresponding to each network layer, and obtain each network layer after pruning;

The generating unit is configured to generate a pruned convolutional neural network model based on the pruned network layers.

In an embodiment of the present application, the first determining unit is specifically configured as:

If the first cropping rate represents the global cropping rate, then the second cropping rate corresponding to each network layer included in the target network structure is determined based on the pruning prior knowledge information and the first cropping rate;

If the first cropping rate represents the cropping rate of the specified network layer included in the target network structure, the second cropping rate corresponding to the specified network layer is determined based on the first cropping rate, and the target is determined based on the pruning prior knowledge information and the first cropping rate The second cropping rate corresponding to other network layers except the specified network layer included in the network structure.

In an embodiment of the present application, the first pruning unit is specifically configured as:

If the target pruning algorithm is a structured pruning algorithm, the feature maps corresponding to each network layer are deleted based on the structured pruning algorithm, so as to prune each network layer, and the structured pruning algorithm is obtained. each network layer;

If the target pruning algorithm is an unstructured pruning algorithm, then based on the unstructured pruning algorithm, the eigenvalues contained in the feature maps corresponding to each network layer are set to zero, so as to prune each network layer. Get each network layer after unstructured pruning.

In an embodiment of the present application, the pruning configuration file includes processing information of the skip structure; the pruning module 1203 includes:

The second reading unit is configured to read the processing information of the jump structure contained in the pruning configuration file;

The second pruning unit is configured to prune all network layers of the jump structure included in the target network structure if the processing information indicates that the skip structure is processed, to obtain a pruned convolutional neural network model;

The third pruning unit is configured to obtain the skippable network layer of the skip structure included in the target network structure if the processing information indicates that the skip structure is not to be processed, and prune the skippable network layer to obtain the pruned volume A neural network model.

In an embodiment of the present application, the pruning module 1203 includes:

a second determining unit, configured to determine a network node for characterizing the output from the target network structure;

The fourth pruning unit is configured to prune other network nodes included in the target network structure except the network nodes used to characterize the output based on the target pruning algorithm, to obtain a pruned convolutional neural network model.

In an embodiment of the present application, the pruning configuration file includes identification information of the target pruning algorithm; the identification and determination module 1202 includes:

The third reading unit is configured to read the identification information of the target pruning algorithm contained in the pruning configuration file;

The third determining unit is configured to determine, according to the identification information, a target pruning algorithm matching the identification information from at least two preset pruning algorithms.

In an embodiment of the present application, the pruning configuration file includes identification information of the target pruning algorithm; the identification and determination module 1202 includes:

The identification unit is configured to identify the convolutional neural network model through the identification algorithm, obtain the dependencies of the operators contained in the convolutional neural network model, and map the dependencies of the operators into a directed acyclic graph;

The obtaining unit is configured to obtain the target network structure based on the directed acyclic graph.

In an embodiment of the present application, the data to be processed includes at least one of image data, voice data, and text data; the processing module 1204 includes:

The first processing unit is configured to, if the data to be processed is image data, process the image data based on the pruned convolutional neural network model to obtain an image processing result;

The second processing unit is configured to, if the data to be processed is voice data, process the voice data based on the pruned convolutional neural network model to obtain a voice processing result;

The third processing unit is configured to, if the data to be processed is text data, process the text data based on the pruned convolutional neural network model to obtain a text processing result.

In an embodiment of the present application, the pruning framework for processing the convolutional neural network model includes a user layer, an intermediate layer and a pruning layer;

The user layer corresponds to the user module, and the user layer is used to obtain the convolutional neural network model to be processed and the pruning configuration file matching the convolutional neural network model;

The middle layer corresponds to the middle module, and the middle layer is used to determine the target pruning algorithm based on the pruning configuration file;

The pruning layer corresponds to the underlying module, and the pruning layer is used to identify the network structure of the convolutional neural network model based on the recognition algorithm to obtain the target network structure, and to prune the target network structure based on the target pruning algorithm to obtain the pruned The convolutional neural network model.

In an embodiment of the present application, the device further includes:

an obtaining module, configured to obtain an identification algorithm for identifying the network structure of the convolutional neural network model, and at least two pruning algorithms for pruning the target network structure;

The construction and generation module is configured to construct and generate a pruning framework based on the recognition algorithm and at least two pruning algorithms.

In an embodiment of the present application, the building and generating module includes:

The building unit is configured to build the user layer, the middle layer and the pruning layer;

The preset unit is configured to call the first interface of the middle layer, preset at least two pruning algorithms in the middle layer, and call the second interface of the pruning layer, and preset the identification algorithm in the pruning layer;

The generating unit is configured to generate a pruning framework through a user layer, an intermediate layer with a preset pruning algorithm, and a pruning layer with a preset identification algorithm.

It should be noted that the apparatuses provided in the foregoing embodiments belong to the same concept as the methods provided in the foregoing embodiments, and the specific manners in which each module and unit performs operations have been described in detail in the method embodiments, which will not be repeated here.

Embodiments of the present application also provide an electronic device, including: one or more processors; and a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, The electronic device is made to realize the previous data processing method based on the convolutional neural network model.

FIG. 13 is a schematic structural diagram of a computer system suitable for implementing the electronic device of the embodiment of the present application.

It should be noted that the computer system 1300 of the electronic device shown in FIG. 13 is only an example, and should not impose any limitations on the functions and scope of use of the embodiments of the present application.

As shown in FIG. 13 , the computer system 1300 includes a central processing unit (Central Processing Unit, CPU) 1301, which can be loaded into a random device according to a program stored in a read-only memory (Read-Only Memory, ROM) 1302 or from a storage part 1308 Various appropriate actions and processes are performed by accessing a program in a memory (Random Access Memory, RAM) 1303, for example, performing the methods in the above-mentioned embodiments. In the RAM 1303, various programs and data necessary for system operation are also stored. The CPU 1301 , the ROM 1302 , and the RAM 1303 are connected to each other through a bus 1304 . An Input/Output (I/O) interface 1305 is also connected to the bus 1304 .

The following components are connected to the I/O interface 1305: an input section 1306 including a keyboard, a mouse, etc.; an output section 1307 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc. ; a storage section 1308 including a hard disk and the like; and a communication section 1309 including a network interface card such as a LAN (Local Area Network) card, a modem, and the like. The communication section 1309 performs communication processing via a network such as the Internet. Drivers 1310 are also connected to I/O interface 1305 as needed. A removable medium 1311, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is mounted on the drive 1310 as needed so that a computer program read therefrom is installed into the storage section 1308 as needed.

In particular, according to embodiments of the present application, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising a computer program for performing the method illustrated in the flowchart. In such an embodiment, the computer program may be downloaded and installed from the network via the communication portion 1309, and/or installed from the removable medium 1311. When the computer program is executed by the central processing unit (CPU) 1301, various functions defined in the system of the present application are executed.

It should be noted that the computer-readable medium shown in the embodiments of the present application may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the above two. The computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or a combination of any of the above. More specific examples of computer readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable Erasable Programmable Read Only Memory (EPROM), flash memory, optical fiber, portable Compact Disc Read-Only Memory (CD-ROM), optical storage device, magnetic storage device, or any suitable of the above The combination. In this application, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, however, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program therein. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device . A computer program embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. Wherein, each block in the flowchart or block diagram may represent a module, program segment, or part of code, and the above-mentioned module, program segment, or part of code contains one or more executables for realizing the specified logical function instruction. It should also be noted that, in some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It is also noted that each block of the block diagrams or flowchart illustrations, and combinations of blocks in the block diagrams or flowchart illustrations, can be implemented in special purpose hardware-based systems that perform the specified functions or operations, or can be implemented using A combination of dedicated hardware and computer instructions is implemented.

The units involved in the embodiments of the present application may be implemented in software or hardware, and the described units may also be provided in a processor. Among them, the names of these units do not constitute a limitation on the unit itself under certain circumstances.

Another aspect of the present application also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, implements the aforementioned data processing method based on a convolutional neural network model. The computer-readable storage medium may be included in the electronic device described in the above embodiments, or may exist alone without being assembled into the electronic device.

Another aspect of the present application also provides a computer program product or computer program comprising computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the data processing method based on the convolutional neural network model provided in each of the foregoing embodiments.

The above contents are only preferred exemplary embodiments of the present application, and are not intended to limit the embodiments of the present application. Those of ordinary skill in the art can easily make corresponding changes or modifications according to the main concept and spirit of the present application, Therefore, the protection scope of the present application shall be subject to the protection scope required by the claims.

Claims (15)

1. A data processing method based on a convolutional neural network model is characterized by comprising the following steps:

acquiring a convolutional neural network model to be processed and a pruning configuration file matched with the convolutional neural network model;

identifying a network structure of the convolutional neural network model based on an identification algorithm to obtain a target network structure, and determining a target pruning algorithm based on the pruning configuration file, wherein the target pruning algorithm is independent of the identification algorithm;

pruning the target network structure based on the target pruning algorithm to obtain a pruned convolutional neural network model;

and processing the data to be processed based on the pruned convolutional neural network model to obtain a processing result.

2. The method of claim 1, wherein the pruning profile includes a first pruning rate; the pruning of the target network structure based on the target pruning algorithm to obtain the pruned convolutional neural network model comprises the following steps:

reading the first cutting rate contained in the pruning configuration file;

determining a second clipping rate corresponding to each network layer contained in the target network structure based on the first clipping rate;

according to the second cutting rate corresponding to each network layer, the target pruning algorithm is used for pruning each network layer to obtain each network layer after pruning;

and generating a pruned convolutional neural network model based on each pruned network layer.

3. The method of claim 2, wherein the determining a second clipping rate for each network layer included in the target network structure based on the first clipping rate comprises:

if the first clipping rate represents the global clipping rate, determining a second clipping rate corresponding to each network layer included in the target network structure based on the prior clipping knowledge information and the first clipping rate;

if the first clipping rate represents a designated network layer clipping rate contained in the target network structure, determining a second clipping rate corresponding to the designated network layer based on the first clipping rate, and determining a second clipping rate corresponding to other network layers except the designated network layer contained in the target network structure based on the pruning priori knowledge information and the first clipping rate.

4. The method according to claim 2, wherein the pruning, according to the second pruning rate corresponding to each network layer, each network layer through the target pruning algorithm to obtain each pruned network layer comprises:

if the target pruning algorithm is a structured pruning algorithm, deleting the feature maps corresponding to the network layers based on the structured pruning algorithm to prune the network layers to obtain the network layers after structured pruning;

and if the target pruning algorithm is an unstructured pruning algorithm, respectively carrying out zero setting processing on characteristic values contained in the characteristic graphs corresponding to the network layers based on the unstructured pruning algorithm so as to prune the network layers to obtain the network layers subjected to unstructured pruning.

5. The method of claim 1, wherein the pruning profile includes processing information for a skip structure; the pruning of the target network structure based on the target pruning algorithm to obtain the pruned convolutional neural network model comprises the following steps:

reading processing information of a jump structure contained in the pruning configuration file;

if the processing information indicates that the jump structure is processed, pruning all network layers of the jump structure contained in the target network structure to obtain a pruned convolutional neural network model;

and if the processing information indicates that the hopping structure is not processed, acquiring a jumpable network layer of the hopping structure contained in the target network structure, and pruning the jumpable network layer to obtain a pruned convolutional neural network model.

6. The method of claim 1, wherein said pruning the target network structure based on the target pruning algorithm to obtain a pruned convolutional neural network model comprises:

determining a network node for characterizing an output from the target network structure;

and pruning other network nodes except the network node used for representing the output in the target network structure based on the target pruning algorithm to obtain a pruned convolutional neural network model.

7. The method of claim 1, wherein the pruning configuration file includes identification information for the target pruning algorithm; the determining a target pruning algorithm based on the pruning configuration file comprises:

reading the identification information of the target pruning algorithm contained in the pruning configuration file;

and determining a target pruning algorithm matched with the identification information from at least two preset pruning algorithms according to the identification information.

8. The method of claim 1, wherein the identifying the network structure of the convolutional neural network model based on an identification algorithm to obtain a target network structure comprises:

identifying the convolutional neural network model through an identification algorithm to obtain the dependency relationship of operators contained in the convolutional neural network model, and mapping the dependency relationship of the operators into a directed acyclic graph;

and obtaining the target network structure based on the directed acyclic graph.

9. The method of claim 1, wherein the data to be processed comprises at least one of image data, voice data, and text data; the method for processing the data to be processed based on the pruned convolutional neural network model to obtain a processing result comprises the following steps:

if the data to be processed is image data, processing the image data based on the pruned convolutional neural network model to obtain an image processing result;

if the data to be processed is voice data, processing the voice data based on the pruned convolutional neural network model to obtain a voice processing result;

and if the data to be processed is character data, processing the character data based on the pruned convolutional neural network model to obtain a character processing result.

10. The method of any one of claims 1 to 9, wherein a pruning framework for processing the convolutional neural network model comprises a user layer, an intermediate layer, and a pruning layer;

the user layer is used for acquiring the convolutional neural network model to be processed and a pruning configuration file matched with the convolutional neural network model;

the middle layer is used for determining a target pruning algorithm based on the pruning configuration file;

the pruning layer is used for identifying the network structure of the convolutional neural network model based on an identification algorithm to obtain a target network structure, and pruning the target network structure based on the target pruning algorithm to obtain a pruned convolutional neural network model.

11. The method of claim 10, wherein the method further comprises:

acquiring an identification algorithm for identifying the network structure of the convolutional neural network model and at least two pruning algorithms for pruning the target network structure;

and constructing and generating the pruning frame based on the identification algorithm and at least two pruning algorithms.

12. The method of claim 11, wherein said constructing and generating the pruning frame based on the recognition algorithm and at least two pruning algorithms comprises:

constructing a user layer, a middle layer and a pruning layer;

calling a first interface of the middle layer, presetting the at least two pruning algorithms in the middle layer, calling a second interface of the pruning layer, and presetting the identification algorithm in the pruning layer;

and generating the pruning frame through the user layer, the middle layer preset with the pruning algorithm and the pruning layer preset with the identification algorithm.

13. A data processing device based on a convolution neural network model is characterized in that,

the system comprises an acquisition module, a processing module and a pruning module, wherein the acquisition module is configured to acquire a convolutional neural network model to be processed and a pruning configuration file matched with the convolutional neural network model;

the identification and determination module is configured to identify a network structure of the convolutional neural network model based on an identification algorithm to obtain a target network structure, and determine a target pruning algorithm based on the pruning configuration file, wherein the target pruning algorithm is independent of the identification algorithm;

the pruning module is configured to prune the target network structure based on the target pruning algorithm to obtain a pruned convolutional neural network model;

and the processing module is configured to process the data to be processed based on the pruned convolutional neural network model to obtain a processing result.

14. An electronic device, comprising:

one or more processors;

storage means for storing one or more programs that, when executed by the electronic device, cause the electronic device to implement the convolutional neural network model-based data processing method of any one of claims 1 to 12.

15. A computer-readable medium, on which a computer program is stored which, when being executed by a processor, carries out a method of data processing based on a convolutional neural network model as claimed in any one of claims 1 to 12.

Images