CN119127228A - Image recognition model deployment method and system - Google Patents

Abstract

The invention belongs to the technical field of algorithm models, and particularly relates to an image recognition model deployment method and system. Aiming at the defect that the existing model deployment method cannot or cannot well realize the deployment of the models to different chips, the image recognition model deployment method adopts the following technical scheme that the image recognition model deployment method comprises the steps of converting a model to be deployed into a standard model with a unified format, loading the standard model under the environment of the chip to be deployed, restoring to obtain all information of the model to be deployed, dividing a network according to whether the chip to be deployed supports the network, compiling operators supported by the chip to be deployed by adopting a compiling tool of a chip manufacturer to be deployed after converting the operators supported by the chip to be deployed to obtain a hard calculation result, calculating the unsupported operators by adopting a processor, inputting the hard calculation result into an reasoning tool of the chip manufacturer to be deployed, and inputting data to be inferred into the reasoning tool or the processor to be inferred to obtain a reasoning result. The invention has the beneficial effect of realizing the deployment of the same algorithm model on different chips.

Description

Image recognition model deployment method and system

Technical Field

The invention belongs to the technical field of algorithm models, and particularly relates to an image recognition model deployment method and system.

Background

In the current artificial intelligence industry, the types of chips used for carrying image recognition models are various, including Injeam, hua Ji, ganman, ruifeng micro, horizon and the like, the requirements of each type of chips on algorithm interfaces are different and incompatible, so that the algorithms and the chips are bound, the algorithms need to be redeveloped when the chips are replaced, and the existing chips can only be selected and adapted when the algorithms are updated. The fragmentation phenomenon which is mutually not universal directly causes higher application and development cost of the algorithm, and indirectly causes the inexact field of artificial intelligence, so that the innovative power of algorithm developers is reduced, and finally the practical process of the artificial intelligence algorithm is hindered.

CN114880995A discloses an algorithm scheme deployment method, a related device, equipment and a storage medium, wherein the algorithm scheme deployment method comprises the steps of responding to a newly created scheme project on a scheme platform, obtaining a plurality of first algorithm models suitable for the scheme project based on an algorithm model library arranged in the scheme platform, obtaining a format conversion tool suitable for a target platform based on a conversion tool library arranged in the scheme platform, wherein the target platform characterizes a chip platform deployed after the scheme project is built, respectively carrying out format conversion on the plurality of first algorithm models based on the format conversion tool to obtain a second algorithm model matched with the target platform, and carrying out scheme deployment based on the second algorithm model to obtain the algorithm scheme.

The scheme can realize the deployment application of the algorithm scheme on various chip platforms, but has the following problems that 1, the algorithm is developed from zero and the developed algorithm cannot be deployed on various chip platforms, 2, a built-in algorithm model library needs to be configured with models suitable for various chip platforms, a conversion tool library needs to be configured with format conversion tools suitable for various chip platforms, 3, the models can only be selected from the built-in algorithm model library, and 4, the same complete flow is needed to be carried out on various chip platforms, namely, the different chip platforms need to be redeveloped.

Disclosure of Invention

Aiming at the defect that the existing model deployment method can not or can not well realize the deployment of the model to different chips, the invention provides the image recognition model deployment method for realizing the deployment of the developed algorithm on various chip platforms. The invention also provides an image recognition model deployment system.

In order to achieve the purpose, the invention adopts the following technical scheme that the image recognition model deployment method comprises the following steps:

Step S1, converting a model to be deployed into a standard model in a unified format, wherein the standard model keeps all information of the model to be deployed;

s2, loading a standard model in a chip environment to be deployed, and recovering after analysis to obtain all information of the model to be deployed;

s3, inquiring an operator information base of a chip manufacturer to be deployed, and judging whether the chip to be deployed supports network segmentation or not;

s4, compiling an operator supported by the chip to be deployed by adopting a compiling tool of a chip manufacturer to be deployed to obtain executable content;

And S5, inputting the executable content into an inference tool of a chip manufacturer to be deployed, and inputting the data to be inferred into the inference tool or a processor to be inferred, so as to obtain an inference result.

In step S1, the operator of the model to be deployed is represented by the node, and all information of the model to be deployed includes the node name, the connection relationship between the nodes, and the input and output of the nodes.

In step S1, a development framework of the model to be deployed is identified by a model conversion tool, and tensors and node names of each layer of the model to be deployed are described by using the same rule.

As an improvement, in step S1, the model structure is optimized by a model conversion tool, and the optimization method includes:

Operator fusion, i.e. the fusion of two or more operators that can be combined into one operator.

In step S1, a model conversion tool is used to output node names, node-to-node connection relationships, and node input and output in a serialized manner for a standard model.

In step S2, the deserialization module is used to analyze the sequence of the standard model, and each parameter required by each layer of calculation is extracted, wherein the parameters include tensor, data type arrangement and operator.

In step S3, the standard model after deserialization is marked in a layering way, and a compiling tool or a processor is selected for different layers according to the marks;

in step S4, according to the content of the operator information base, the correct operators in the operator pool are called to perform layer-by-layer calculation, and a handle representing the network structure is obtained after compiling is completed;

In step S4, according to the content of the operator information base, a plurality of operators of the operator pool are called once, fusion compiling is finished in advance, and calculation is carried out by utilizing the fused operators;

in step S5, the inference result is represented by a matrix, and the matrix includes the complete calculation result information.

As an improvement, the image recognition model deployment method further comprises:

And S6, releasing the reasoning resources.

The image recognition model deployment system adopts the image recognition model deployment method, and comprises the following steps:

the model conversion tool is used for converting the model to be deployed into a standard model in a unified format, and the standard model contains all information of the algorithm to be deployed;

The model analysis module is used for loading a standard model in the chip environment to be deployed, and recovering after analysis to obtain all information of the standard model;

The operator information base module is used for storing operators of all chip manufacturers;

The inquiring and splitting module is used for inquiring an operator information base of a chip manufacturer to be deployed and splitting the network according to whether the chip to be deployed supports the splitting of the network;

The network compiling module is used for compiling operators supported by the chip to be deployed by adopting a compiling tool of a chip manufacturer to be deployed to obtain executable content;

The network computing module is used for inputting executable content into an inference tool of a chip manufacturer to be deployed, and inputting data to be inferred into the inference tool or a processor to be inferred, so as to obtain an inference result;

And the resource release module is used for releasing the reasoning resources after the reasoning is completed.

As an improvement of an image recognition model deployment system, the model conversion tool comprises an analysis module, an operator fusion module and a serialization module;

the model analysis module, the operator information base module, the query segmentation module, the network compiling module and the network computing module are integrated in the SDK.

The image recognition model deployment method comprises the steps of converting a model to be deployed into a standard model with a unified format, loading the standard model under the environment of a chip to be deployed, analyzing and restoring to obtain all information of the model to be deployed, inquiring an operator information base of a chip manufacturer to be deployed, compiling operators supported by the chip to be deployed by adopting a compiling tool of the chip manufacturer to be deployed according to whether the chip to be deployed supports the splitting of a network to obtain executable content, inputting the executable content into an reasoning tool of the chip manufacturer to be deployed, inputting data to be inferred into the reasoning tool or a processor to infer to obtain an reasoning result, and deploying the developed algorithm model on various chip platforms by the steps.

Drawings

FIG. 1 is a block diagram of a model conversion tool of an image recognition model deployment system of an embodiment of the present invention.

FIG. 2 is a block diagram of the architecture of an SDK (Software Development Kit ) of an image recognition model deployment system of an embodiment of the present invention.

FIG. 3 is a flow chart of an image recognition model deployment method of an embodiment of the present invention.

Fig. 4 is a model structure diagram obtained after a model is read by a model conversion tool of the image recognition model deployment system according to the embodiment of the present invention.

FIG. 5 is a model structure diagram obtained by fusing operators by a model conversion tool of the image recognition model deployment system according to the embodiment of the invention.

FIG. 6 is a representation of the model serialization output after the conversion of the model conversion tool of the image recognition model deployment system of an embodiment of the present invention.

FIG. 7 is a block diagram of a segmented model of an image recognition model deployment system in accordance with an embodiment of the present invention.

FIG. 8 is a compiled model structure diagram of an image recognition model deployment system of an embodiment of the present invention.

Detailed Description

The following description of the technical solutions of the inventive embodiments of the present invention is provided only for the preferred embodiments of the invention, but not all. Based on the examples in the implementation manner, other examples obtained by a person skilled in the art without making any inventive effort fall within the scope of protection created by the present invention.

Referring to fig. 1 to 8, an image recognition model deployment system of an embodiment of the present invention includes:

the model conversion tool is used for converting the model to be deployed into a standard model in a unified format, and the standard model contains all information of the algorithm to be deployed;

The model analysis module is used for loading a standard model in the chip environment to be deployed, and recovering after analysis to obtain all information of the standard model;

The operator information base module is used for storing operators of all chip manufacturers;

The inquiring and splitting module is used for inquiring an operator information base of a chip manufacturer to be deployed and splitting the network according to whether the chip to be deployed supports the splitting of the network;

The network compiling module is used for compiling operators supported by the chip to be deployed by adopting a compiling tool of a chip manufacturer to be deployed to obtain executable content;

The network computing module is used for inputting executable content into an inference tool of a chip manufacturer to be deployed, and inputting data to be inferred into the inference tool or a processor to be inferred, so as to obtain an inference result;

And the resource release module is used for releasing the reasoning resources after the reasoning is completed.

Referring to fig. 1 and 2, in the present embodiment, the image recognition model deployment system includes two main parts, a model conversion tool and an SDK. The model conversion tool comprises an analysis module, an operator fusion module and a serialization module. The model analysis module, the operator information base module, the query segmentation module, the network compiling module and the network computing module are integrated in the SDK.

In this embodiment, the entire image recognition model deployment system is provided to the user (e.g., algorithm developer). The user deploys the SDK on the hardware device in a library manner to provide an environment for the converted model to run.

In this embodiment, the parsing module of the model conversion tool has parsing sub-modules for each development framework such as caffe, tensorflow, darkent. Through the analysis module, the received algorithm model can be known based on what development framework is developed.

In this embodiment, the operator fusion module of the model conversion tool may fuse nodes with consistent calculation forms, so as to greatly increase the reasoning speed.

In this embodiment, the model conversion tool expresses the model in a serialized manner.

In this embodiment, the deserializing module of the SDK analyzes to obtain all standardized information. The standardization means that the algorithm models developed by all development frameworks adopt the same expression mode.

In this embodiment, the network segmentation module of the SDK queries the operator information base, and segments the network according to whether the chip to be deployed supports the segmentation of the network.

In this embodiment, after the network is segmented, the network compiling module invokes a compiling tool of a chip manufacturer to compile the supported operators individually, and the compiling result is binary machine code content, which is stored in the memory, and forms a processor (such as CPU and GPU) computing network for the operators that are not supported.

In this embodiment, after compiling is completed, the network computing module performs computation. Specifically, an inference tool of a chip manufacturer is called to calculate the supported operators. And calculating unsupported operators by adopting a processor.

In this embodiment, after the reasoning is completed, the resource releasing module releases the reasoning resource. The user (such as algorithm developer) can extract information from the calculation result matrix according to the own needs, and then carry out subsequent operation.

In this embodiment, an operator fusion module of a model conversion tool is used to fuse part of operators in the model, so as to obtain a standard model. The model conversion tool has an operator fusion function (the operators are uniformly represented by node), and when the calculation forms of a plurality of operators are consistent, fusion can be performed. The model structure after optimization can greatly accelerate the reasoning speed. The converted second algorithm model is stored in a file mode.

Referring to fig. 3 to 8, an image recognition model deployment method includes:

Step S1, converting a model to be deployed into a standard model in a unified format, wherein the standard model keeps all information of the model to be deployed;

s2, loading a standard model in a chip environment to be deployed, and recovering after analysis to obtain all information of the model to be deployed;

s3, inquiring an operator information base of a chip manufacturer to be deployed, and judging whether the chip to be deployed supports network segmentation or not;

Step S4, compiling operators supported by the chip to be deployed by adopting a compiling tool of a chip manufacturer to be deployed to obtain executable content, wherein the operators not supported have no compiling step, and directly adopt a processor to perform calculation processing in the subsequent steps;

And S5, inputting the executable content into an inference tool of a chip manufacturer to be deployed, and inputting the data to be inferred into the inference tool or a processor to be inferred, so as to obtain an inference result.

In this embodiment, in step S1, the operator of the model to be deployed is represented by a node, and all information of the model to be deployed includes a node name, a connection relationship between nodes, and input and output of the node.

In this embodiment, in step S1, a development framework of the model to be deployed is identified through a model conversion tool, and tensors and node names of each layer of the model to be deployed are described by using the same rule.

In this embodiment, in step S1, the model structure is optimized by the model conversion tool, and the optimization method includes:

Operator fusion, i.e. the fusion of two or more operators that can be combined into one operator.

In the embodiment, in step S1, a model conversion tool is used to output node names, connection relationships between nodes, and input and output of nodes in a serialized manner for a standard model.

In this embodiment, in step S2, a deserialization module is used to analyze the sequence of the standard model, and each parameter required for calculation of each layer is extracted, where the parameters include tensor, data type arrangement, and operator.

In the embodiment, in step S3, layering labeling is performed on the deserialized standard model, and a compiling tool or a processor is selected for different layers according to the labeling;

in step S4, according to the content of the operator information base, the correct operators in the operator pool are called to perform layer-by-layer calculation, and a handle representing the network structure is obtained after compiling is completed;

in step S4, according to the content of the operator information base, a plurality of operators of the operator pool are called at a time, fusion compiling is finished in advance, and calculation is carried out by using the fused operators.

In step S4, the supported operators are converted before compiling, that is, all information (including input information, output information, arrangement information, operator names, etc.) of the operators are represented by the data structure according to the requirements of the chip manufacturer, and then compiling is started.

In step S5, the inference result is represented by a matrix, and the matrix includes the complete calculation result information.

The following describes the procedure of the deployment method according to the embodiment of the present invention in more detail with reference to fig. 1 to 8.

Assuming that the structure of a certain image recognition model to be deployed is shown in fig. 4, the original model is obtained through caffe framework training. The image recognition model has 6 nodes (nodes for representing operators) and tensors 0-7 represent different tensors, respectively. After Tensor0 inputs Node0, tensor1 to Node1 and Tensor to Node2 are output. Node1 outputs Tensor3 to Node3, node2 outputs Tensor4 to Node3, node3 outputs Tensor5 to Node4, node4 outputs Tensor6 to Node5, and finally Node5 outputs Tensor7.

First, this original model of fig. 4 is converted using a model conversion tool, resulting in a standard model. The main structure of the model conversion tool is shown in fig. 1. The parsing module of the model conversion tool has caffe submodules corresponding to caffe frameworks, and model network information of an original model (set as a first model file) is read through caffe submodules to obtain the information of fig. 4.

The model conversion tool has an operator fusion function (these operators are collectively represented by node), and since the calculation forms of node4 and node5 are identical, fusion can be performed, so that the model structure shown in fig. 5 can be obtained. The model structure after optimization can greatly accelerate the reasoning speed.

The model conversion tool has a serialization module, and the converted model is represented in a serialized form (see fig. 6), stored in a file manner, and set as a second model file. The second model file is also a file for inputting the SDK.

Then, the SDK analyzes the content of the second model file through an internal deserializing module, and all information of the standard model is obtained after analysis.

And secondly, the SDK queries an operator information base of a chip manufacturer to be deployed through an internal network segmentation module, and segments the network according to whether the chip to be deployed supports the network. In this example, node0, node1, node2, and Node3 are AI chip supported, and Node4 is not supported. The sub-network will be split into the case of fig. 7.

And thirdly, after the network segmentation is completed, the SDK invokes a compiling tool of a chip manufacturer through an internal network compiling module, independently compiles the supported operators, and the compiling result is binary machine code content and is stored in a memory. As shown in fig. 8, the chip computing network of node0, node1, node2, node3 has been compiled into a piece of machine code content that can be run directly. And forming a CPU computing network for the Node4 which is not supported, and adopting the CPU to compute.

Finally, each sub-network is calculated. Firstly, a chip computing network is calculated, and the computing mode of the network is that an inference tool of a chip manufacturer is called for computing. The result of the calculation is given to the CPU calculation network, namely node4, and the calculation process of node4 is realized on the CPU. The matrix result stored in tensor as form is finally obtained.

And after the reasoning is completed, releasing the reasoning resources. The user (such as algorithm developer) can extract information from the calculation result matrix according to the own needs, and then carry out subsequent operation.

When the same algorithm model needs to be deployed to different chip platforms, the steps S2 to S6 are repeated only according to the corresponding chips.

While the invention has been described in terms of specific embodiments, it will be apparent to those skilled in the art that the invention is not limited to the specific embodiments described. Any modifications which do not depart from the functional and structural principles of the present invention are intended to be included within the scope of the appended claims.

Claims (10)

1. The image recognition model deployment method is characterized by comprising the following steps of:

Step S1, converting a model to be deployed into a standard model in a unified format, wherein the standard model keeps all information of the model to be deployed;

s2, loading a standard model in a chip environment to be deployed, and recovering after analysis to obtain all information of the model to be deployed;

S3, inquiring an operator information base of a chip manufacturer to be deployed, and judging whether the operator is supported by the chip to be deployed or not;

s4, compiling an operator supported by the chip to be deployed by adopting a compiling tool of a chip manufacturer to be deployed to obtain executable content;

And S5, inputting the executable content into an inference tool of a chip manufacturer to be deployed, and inputting the data to be inferred into the inference tool or a processor to be inferred, so as to obtain an inference result.

2. The method for deploying the image recognition model according to claim 1, wherein in step S1, the operator of the model to be deployed is represented by the node, and all information of the model to be deployed comprises the node name, the connection relationship between the nodes, and the input and output of the node.

3. The method for deploying the image recognition model according to claim 2, wherein in step S1, a development framework of the model to be deployed is recognized through a model conversion tool, and tensors and node names of layers of the model to be deployed are described by using the same rule.

4. The method for deploying an image recognition model according to claim 3, wherein in step S1, the model structure is optimized by a model conversion tool, and the optimizing method comprises:

Operator fusion, i.e. the fusion of two or more operators that can be combined into one operator.

5. The method of claim 4, wherein in step S1, the model conversion tool is used to output node names, node-to-node connection relationships, and node inputs and outputs in a serialized manner for the standard model.

6. The method of claim 5, wherein in step S2, the de-serialization module is used to parse the sequence of the standard model, and extract parameters required by each layer of calculation, the parameters including tensor, data type arrangement, and operator.

7. The method for deploying image recognition models according to claim 6, wherein in step S3, the de-serialized standard model is marked in layers, and a compiling tool or a processor is selected for different layers according to the marking;

In step S4, the supported operators are expressed in a data structure according to the requirements of chip manufacturers before compiling;

in step S4, according to the content of the operator information base, the correct operators in the operator pool are called to perform layer-by-layer calculation, and a handle representing the network structure is obtained after compiling is completed;

In step S4, according to the content of the operator information base, a plurality of operators of the operator pool are called once, fusion compiling is finished in advance, and calculation is carried out by utilizing the fused operators;

in step S5, the inference result is represented by a matrix, and the matrix includes the complete calculation result information.

8. The image recognition model deployment method according to claim 1, wherein the image recognition model deployment method further comprises:

And S6, releasing the reasoning resources.

9. An image recognition model deployment system adopting the image recognition model deployment method according to any one of claims 1 to 8, characterized in that the image recognition model deployment system comprises:

the model conversion tool is used for converting the model to be deployed into a standard model in a unified format, and the standard model contains all information of the algorithm to be deployed;

The model analysis module is used for loading a standard model in the chip environment to be deployed, and recovering after analysis to obtain all information of the standard model;

The operator information base module is used for storing operators of all chip manufacturers;

the query segmentation module is used for querying an operator information base of a chip manufacturer to be deployed and segmenting operators according to whether the chip to be deployed supports the operator;

The network compiling module is used for compiling operators supported by the chip to be deployed by adopting a compiling tool of a chip manufacturer to be deployed to obtain executable content;

The network computing module is used for inputting executable content into an inference tool of a chip manufacturer to be deployed, and inputting data to be inferred into the inference tool or a processor to be inferred, so as to obtain an inference result;

And the resource release module is used for releasing the reasoning resources after the reasoning is completed.

10. The image recognition model deployment system of claim 9, wherein the model transformation tool comprises an parsing module, an operator fusion module, and a serialization module;

the model analysis module, the operator information base module, the query segmentation module, the network compiling module and the network computing module are integrated in the SDK.