CN116560674A - A method and device for exporting a deep learning model compatible with multiple frameworks to ONNX - Google Patents

Abstract

The embodiment of the present application discloses a method and device for exporting a deep learning model compatible with multiple frameworks to ONNX. The method includes: obtaining layer information of each layer of the target model, and breadth-first search rules. The target model is a unified deep learning model. The programming framework model, the target model can be compatible with multiple frameworks using different conversion rules; according to the layer information of each layer of the target model and the breadth-first traversal rules, each layer of the target deep learning model is traversed sequentially to obtain The corresponding model topology; according to the model topology, map each layer of the target model in order to map the model node to the open neural network exchange ONNX; and according to the mapping result of mapping the model node to ONNX and multiple key Function to generate model files including ONNX models by creating ONNX calculation graphs and creating ONNX models.

Description

A method and device for exporting a deep learning model compatible with multiple frameworks to ONNX

technical field

The invention relates to the field of computer technology, in particular to a method and device for exporting a deep learning model compatible with multiple frameworks to ONNX.

Background technique

At present, the deployment of deep learning models is mostly implemented by converting to an intermediate representation ONNX (Open NeuralNetwork Exchange, Open Neural Network Exchange), and then inferring through the inference engine ONNX Runtime, and deploying it through TensorRT. However, ONNX has occupied most of the deep learning deployment market, and various deep learning framework manufacturers have switched their adopted frameworks to ONNX.

Currently, there are many tools for sequentially converting each framework to ONNX, for example, ensorFlow2ONNX, Keras2ONNX, Paddle2ONNX. Each of the above-mentioned tools has a specific set of conversion rules corresponding to it. For example, TensorFlow2ONNX is to map TensorFlow operators to ONNX operators one by one, and then convert the entire TensorFlow model calculation graph into the corresponding ONNX graph, and then export the ONNX model.

Since operators and operator parameters are different among different frameworks, when converting any framework in each framework to ONNX, it is necessary to configure a specific set of conversion rules corresponding to the framework.

At present, each deep learning framework has its corresponding specific set of conversion tools in terms of model deployment (for example, paddle has paddle2onnx conversion tool, mindspore has mindspore2onnx conversion tool). Since each of the above-mentioned deep learning frameworks has its corresponding specific front-end interface, and there are differences between each front-end interface, each deep learning framework has its corresponding set of conversion rules; therefore, for different deep learning with different conversion rules Framework, the user needs to implement the code of the corresponding framework in turn, and deploy and migrate the model through the code of the corresponding framework, which will make the deployment and migration process of the model difficult.

Contents of the invention

Based on this, it is necessary to provide a method, device, Storage media, electronic devices and computer program products.

In the first aspect, the embodiment of the present application provides a method for exporting a deep learning model compatible with multiple frameworks to ONNX, the method comprising:

Obtaining layer information and breadth-first search rules of each layer of the target model, the target model is a deep learning unified programming framework model, and the target model is compatible with multiple frameworks using different conversion rules;

According to the layer information of each layer of the target model and the breadth-first traversal rule, each layer of the target deep learning model is sequentially traversed to obtain a corresponding model topology;

According to the model topology, each layer of the target model is mapped in order to map the model nodes to the open neural network exchange ONNX;

According to the mapping result of mapping model nodes to ONNX and multiple key functions, by creating an ONNX calculation graph and creating an ONNX model, a model file including the ONNX model is generated.

In one embodiment, the obtaining layer information of each layer of the target model includes:

Obtain the attribute information of the module node. The attribute information of the module node includes the neural network layer, the input node of the neural network layer of the current layer, the output node of the neural network layer of the current layer, the input value of the neural network layer of the current layer, the neural network layer of the current layer Output value, and the name of the current network layer node;

By calling the module node to perform a forward propagation, obtain the layer information of each layer of the target model, the layer information of each layer of the target model includes the target neural network layer, any layer as the current layer corresponding to the current layer Neural network layer input node, any layer as the current layer neural network layer output node corresponding to the current layer, any layer as the current layer neural network layer input value corresponding to the current layer, any layer as the current layer corresponding to the current layer Layer neural network layer output value and any layer as the current network layer node name corresponding to the current layer.

In one embodiment, according to the model topology, each layer of the target model is sequentially mapped to map model nodes to ONNX, including:

Obtain the model topology;

determining a corresponding topological sorting manner according to the model topology;

According to the topological sorting method, each layer is mapped sequentially to obtain the corresponding ONNX operator combination to map the model nodes to ONNX.

In one embodiment, the plurality of key functions at least include a first function for creating an ONNX calculation graph and a second function for creating an ONNX model, and according to the mapping result of mapping model nodes to ONNX and multiple A key function, by creating an ONNX calculation graph and creating an ONNX model, a model file including the ONNX model is generated, including:

Obtain the mapping result of mapping model nodes to ONNX;

Create a corresponding ONNX calculation graph and a corresponding ONNX model according to the mapping result, the first function and the second function;

Verify whether the ONNX model is an effective model, and generate a model file including the ONNX model under the valid situation of verifying the ONNX model.

In one implementation, the verification of whether the ONNX model is an effective model includes:

In the case that each layer used from TensorLayerX can be mapped to the ONNX unified format, verify that the ONNX model is a valid model.

In one embodiment, the verification of whether the ONNX model is an effective model also includes:

In the case where the TensorLayerX includes at least one layer that cannot be mapped to the ONNX unified format, verify that the ONNX model is an invalid model.

In one embodiment, the plurality of frameworks using different conversion rules compatible with the target model include at least one of the following:

TneosFlow, MindSpore, PaddlePaddle, and PyTorch.

In the second aspect, the embodiment of the present application provides a device for exporting a deep learning model compatible with multiple frameworks to ONNX, and the device includes:

An acquisition module, configured to acquire layer information of each layer of the target model, and breadth-first search rules, the target model is a deep learning unified programming framework model, and the target model can be compatible with multiple frameworks using different conversion rules;

A traversal module, configured to sequentially perform layer traversal on each layer of the target deep learning model to obtain a corresponding model topology according to the layer information of each layer of the target model and the breadth-first traversal rule;

The mapping module is used to map each layer of the target model in sequence according to the model topology, so as to map the model nodes to the open neural network exchange ONNX;

The generation module is used to generate a model file including the ONNX model by creating an ONNX calculation graph and creating an ONNX model according to the mapping result of mapping the model nodes to ONNX and a plurality of key functions.

In a third aspect, the embodiments of the present application provide a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and the computer program is used to execute the above method steps.

In a fourth aspect, an embodiment of the present application provides an electronic device, and the electronic device includes:

processor;

memory for storing said processor-executable instructions;

The processor is configured to read the executable instruction from the memory, and execute the executable instruction to implement the above method steps.

In a fifth aspect, the embodiments of the present application provide a computer program product, including a computer program, and when the computer program is executed by a processor, the above method steps are implemented.

In the embodiment of the present application, the layer information of each layer of the target model and the breadth-first search rules are obtained, the target model is a deep learning unified programming framework model, and the target model can be compatible with multiple frameworks using different conversion rules; according to the target model The layer information of each layer of the target deep learning model and the breadth-first traversal rules are used to perform layer traversal on each layer of the target deep learning model in order to obtain the corresponding model topology; according to the model topology, each layer of the target model is sequenced Mapping is performed to map the model nodes to the open neural network exchange ONNX; and according to the mapping result and a plurality of key functions of mapping the model nodes to ONNX, by creating an ONNX calculation graph and creating an ONNX model, a model file including the ONNX model is generated. The method for exporting a deep learning model compatible with multiple frameworks to ONNX provided by the embodiment of the present application, since the target model is a unified programming framework model for deep learning, and the target model can be compatible with multiple frameworks using different conversion rules, it is compatible with the depth of multiple frameworks The method of exporting the learning model to ONNX can support a variety of deep learning framework backends (for example, paddle, mindspore, tensorflow, pytorch), and the computing engine can be changed by switching the backend. Therefore, users do not need to deploy and migrate models. Implement the corresponding framework code for deployment, just switch the backend. Using the method of exporting the deep learning model compatible with multiple frameworks to ONNX in the embodiment of the present application can achieve: for different deep learning frameworks using different conversion rules, the process of model deployment and migration has nothing to do with the computing backend used, thus This avoids the need to design multiple mapping methods due to differences in operator interfaces.

Description of drawings

A more complete understanding of the exemplary embodiments of the present invention can be had by referring to the following drawings. The accompanying drawings are used to provide a further understanding of the embodiments of the present application, and constitute a part of the specification, and are used together with the embodiments of the present application to explain the present invention, and do not constitute limitations to the present invention. In the drawings, the same reference numerals generally represent the same components or steps.

Figure 1 is a schematic diagram of the overall design of a deep learning model compatible with multiple frameworks exported to ONNX in a specific application scenario;

Fig. 2 is the flow chart of the method for exporting the deep learning model of compatible multi-frame according to an exemplary embodiment of the present application to ONNX;

Figure 3 is a schematic diagram of the model topology collected through the ModuleNode class in a specific application scenario;

Figure 4 is a schematic diagram of the mapping of model nodes to ONNX in specific application scenarios;

Figure 5 is a schematic diagram of ONNX composition and storage models in specific application scenarios;

FIG. 6 is a schematic structural diagram of a device 600 for exporting a multi-framework-compatible deep learning model to ONNX according to an exemplary embodiment of the present application;

Fig. 7 shows a schematic diagram of an electronic device provided by an exemplary embodiment of the present application;

Fig. 8 shows a schematic diagram of a computer-readable medium provided by an exemplary embodiment of the present application.

Detailed ways

Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided for more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

It should be noted that, unless otherwise specified, technical terms or scientific terms used in this application shall have the usual meanings understood by those skilled in the art to which this application belongs.

In addition, the terms "first" and "second", etc. are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "include" and "have", as well as any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes unlisted steps or units, or optionally further includes For other steps or units inherent in these processes, methods, products or apparatuses.

Embodiments of the present application provide a method and device for exporting a deep learning model compatible with multiple frameworks to ONNX, an electronic device, and a computer-readable medium, which will be described below with reference to the accompanying drawings.

As shown in Figure 1, it is a schematic diagram of the overall design of exporting a deep learning model compatible with multiple frameworks to ONNX in a specific application scenario.

As shown in Figure 1, based on the high-level interface provided by the deep learning unified programming framework, a model that supports multiple computing backends can be obtained; given a model with one input and one forward propagation, all the information of the high-level interface layer can be collected , and then use breadth-first search to obtain the model topology; traverse the access topology, and convert the interface layer information to ONNX operator information one by one according to the mapping; finally use the make_graph function and make_model function provided by ONNX to create the ONNX calculation graph, and use check_model Check the ONNX model, save the model after passing the test and get the model.onnx file; at this point, you can deploy the model to the target device for inference.

It should be noted that the above calculation graph is a directed acyclic graph used to describe operations. The calculation graph has two main elements: node Node and edge Edge; nodes represent data, such as vectors, matrices, and tensors. Edges represent operations, such as addition, subtraction, multiplication, division, convolution, etc.

The process shown in the overall design diagram shown in Figure 1 mainly includes four parts, and the details of the four parts are as follows:

Model topology (for traversal access, and then map the neural network layer of the unified framework to ONNX operators), model node mapping to ONNX (for generating ONNX operator information), ONNX composition and ONNX model (according to ONNX operator information export ONNX file).

Please refer to FIG. 2 , which shows a flowchart of a method for exporting a multi-framework-compatible deep learning model to ONNX provided by some embodiments of the present application. As shown in FIG. 2 , a multi-framework-compatible deep learning model export A method to ONNX may include the following steps:

Step S201: Obtain layer information and breadth-first search rules of each layer of the target model. The target model is a deep learning unified programming framework model, and the target model can be compatible with multiple frameworks using different conversion rules.

In the method for exporting a multi-framework compatible deep learning model to ONNX provided in the embodiment of the present application, the target model is compatible with multiple frameworks using different conversion rules including at least one of the following: TneosFlow, MindSpore, PaddlePaddle, and PyTorch.

In a possible implementation manner, obtaining layer information of each layer of the target model includes the following steps:

Obtain the attribute information of the module node. The attribute information of the module node includes the neural network layer, the input node of the neural network layer of the current layer, the output node of the neural network layer of the current layer, the input value of the neural network layer of the current layer, the neural network layer of the current layer Output value, and the name of the current network layer node;

By calling the module node to perform a forward propagation, the layer information of each layer of the target model is obtained. The layer information of each layer of the target model includes the target neural network layer, and any layer is used as the input of the current layer neural network layer corresponding to the current layer. Node, any layer as the current layer neural network layer output node corresponding to the current layer, any layer as the current layer neural network layer input value corresponding to the current layer, any layer as the current layer neural network layer corresponding to the current layer The output value and any layer are used as the current network layer node name corresponding to the current layer.

Step S202: According to the layer information of each layer of the target model and the breadth-first traversal rule, each layer of the target deep learning model is traversed sequentially to obtain the corresponding model topology.

As shown in Figure 3, it is a schematic diagram of the model topology collected through the ModuleNode class in a specific application scenario.

As shown in Figure 3, the ModuleNode object is constructed, which contains the current layer node information, the current layer input and output layer node information, and input and output data. Because the deep learning model is composed of layers of neural network components, the ModuleNode module node class is encapsulated into a function, and ModuleNode is called in the forward propagation method of each Layer to collect the information of each Layer. Because the input and output are directional, all node information can be collected as long as one forward propagation is performed; finally, the model topology of a deep learning model can be obtained by breadth-first access to the information of each Node node.

As shown in Figure 3, the specific process of collecting information of each layer by calling ModuleNode is as follows:

As shown in Figure 3, the ModuleNode class shows the properties and methods of ModuleNode in the leftmost class diagram. Among them, the attributes include layer (neural network layer), in_nodes (current layer neural network layer input node), out_nodes (current layer neural network layer output node), in_tensors (current layer neural network layer input value), out_tensors (current layer neural network layer layer output value), and node_name (current network layer node name); wherein, the method adopted is layer(inputs), that is, a forward propagation is performed.

As shown in Figure 3, the second module in Figure 3 is a deep learning model. The deep learning model is a network structure connected by Layer stacking. ModuleNode is called in the forward propagation method of each Layer. Then, when the model performs forward propagation, each Layer will call ModuleNode once. At this time, the information of each Layer can be recorded, that is, the attributes described by ModuleNode: layer, in_nodes, out_nodes, in_tensors, out_tensors, node_name. Thus, the third module in Figure 3 is obtained: all node information of the model.

In the method for exporting a multi-framework-compatible deep learning model to ONNX provided in the embodiment of this application, each layer of the target deep learning model is sequentially processed according to the layer information of each layer of the target model and the breadth-first traversal rule Through layer traversal, the corresponding model topology structure is obtained as follows:

The above breadth-first traversal rules are hierarchical traversals. Firstly, the first layer is traversed, and then the second layer is traversed, and the same layer is visited in order from left to right until the last layer is traversed. In Figure 3, all node information of the model has been known.

For example: layer=Layer1, in_nodes=[input], out_nodes=[layer1_node], in_tensors=[a], out_tensors=[b], node_name=layer1;

layer=Layer2, in_nodes=[input], out_nodes=[layer2_node], in_tensors=[a], out_tensors=[c], node_name=layer2;

layer=Layer3, in_nodes=[layer1_node, layer2_node], out_nodes=[layer3_node], in_tensors=[b,c], out_tensors=[d], node_name=layer3;

layer=Layer4, in_nodes=[layer3_node], out_nodes=[layer4_node], in_tensors=[d], out_tensors=[output], node_name=layer4;

Traversing using the breadth-first traversal rule above is: traversing according to the level, specifically: the first layer has layer1 and layer2, then, from left to right layer2, layer1 (the input is input, the output is b, c). The second layer has only layer3 (its input is the node of b, c, and the output is d). The third layer is layer4 (its input is d and its output is output). Check that the output flag is output, and the traversal ends. The model topology obtained through breadth-first traversal above is: a->[b, c]->[d]->[e]->[output].

Step S203: according to the model topology, each layer of the target model is sequentially mapped, so as to map the model nodes to ONNX.

As shown in Figure 4, it is a schematic diagram of the mapping of model nodes to ONNX in specific application scenarios. With the above-mentioned model topology shown in Figure 3, it is only necessary to convert each Layer to ONNX operator combination in turn according to the topology sorting, and then the model node mapping can be completed.

In the actual application scenario, the mapping rules adopted by the model node mapping to ONNX are based on the standard provided by ONNX (an open format for expressing deep learning models, which defines a set of standard formats independent of the environment and platform.)

For example, 2d convolution layer: conv2d in tensorlayerx, then, mapping to onnx needs to be described in onnx format. If conv2d is detected in Layer, then it needs to be mapped to onnx, use make_node('Transpose') to convert the input, and then call make_node('Conv') to perform convolution.

Linear layer: It is linear in tensorlayerx, so mapping to onnx needs to be described in onnx format. If the linear operation is detected in the Layer, it cannot be directly mapped and needs to be split into multiplication and addition. Then, it needs to be mapped to onnx, use make_node('matmul') to perform multiplication, and then call make_node('add') to perform addition.

The 'Transpose', 'Conv', 'matmul', and 'add' provided by ONNX mentioned above are all standard formats.

In a possible implementation, each layer of the target model is sequentially mapped according to the model topology, so as to map the model nodes to the open neural network exchange ONNX, including the following steps:

Get model topology;

Determine the corresponding topological sorting method according to the model topology;

According to the topological sorting method, each layer is mapped sequentially to obtain the corresponding ONNX operator combination to map the model nodes to ONNX.

Step S204: According to the mapping result of mapping model nodes to ONNX and multiple key functions, by creating an ONNX calculation graph and creating an ONNX model, a model file including the ONNX model is generated.

In a possible implementation, the multiple key functions include at least a first function for creating an ONNX calculation graph and a second function for creating an ONNX model, according to the mapping result of mapping model nodes to ONNX and multiple key The function generates a model file including an ONNX model by creating an ONNX calculation graph and creating an ONNX model, including the following steps:

Obtain the mapping result of mapping model nodes to ONNX;

According to the mapping result, the first function and the second function, create the corresponding ONNX calculation graph and the corresponding ONNX model;

Verify whether the ONNX model is a valid model, and generate a model file including the ONNX model if the ONNX model is verified to be valid.

In the actual application scenario, in addition to the first function for creating ONNX calculation graph and the second function for creating ONNX model, the multiple key functions also include: the third function for verifying whether the model is valid.

As shown in Figure 5, it is a schematic diagram of ONNX composition and saving models in specific application scenarios. After obtaining the mapping result of mapping model nodes to ONNX as shown in Figure 4, use the functions that come with ONNX to complete ONNX composition and ONNX export. The above process uses the make_graph (first function) and make_model (second function) functions to create the ONNX calculation graph, and uses check_model (the third function) to check the model. After the check is passed, save the model to get the model.onnx file.

In the actual application scenario, according to the mapping result of mapping model nodes to ONNX and multiple key functions (the first function and the second function), the process of creating ONNX calculation graph is as follows:

As shown in Figure 3, the model topology a->[b,c]->[d]->[e]->[output];

layer=linear, in_nodes=[input], out_nodes=[layer1_node], in_tensors=[a], out_tensors=[b], node_name=linear1;

layer=linear, in_nodes=[input], out_nodes=[layer2_node], in_tensors=[a], out_tensors=[c], node_name=linear2;

layer=linear, in_nodes=[layer1_node, layer2_node], out_nodes=[layer3_node], in_tensors=[b,c], out_tensors=[d], node_name=linear3;

layer=linear, in_nodes=[layer3_node], out_nodes=[layer4_node], in_tensors=[d], out_tensors=[output], node_name=linear4;

Thus, the model topology can be described as: input->[linear 1, linear 2]->[linear 3]->[linear4]->[output].

From this we know: the input is given to linear layer 1 and linear layer 2; the results of linear layer 1 and linear layer 2 are given to linear layer 3; the results of linear layer 3 are given to linear layer 4; and finally output. The above linear layers can be expressed in a unified format provided by ONNX.

In a possible implementation, verifying whether the ONNX model is a valid model includes the following steps:

In the case that each layer used from TensorLayerX can be mapped to the ONNX unified format, verify that the ONNX model is a valid model.

In a possible implementation, verifying whether the ONNX model is a valid model also includes the following steps:

Validate an ONNX model as an invalid model when TensorLayerX includes at least one layer that cannot be mapped to the ONNX unified format.

In the actual application scenario, the check is checked by check_model, which is a function provided by ONNX. When the Layer used from tensorlayerx can be mapped to the ONNX unified format, the test is passed.

If the test fails, for example, there is a layer called Attention in tensorlayerx. Because Attention cannot be mapped to ONNX through multiplication and addition in ONNX like linear. Then, the network constructed by the layer with Attention cannot pass the inspection.

The method of exporting the deep learning model compatible with multiple frameworks to ONNX provided by the embodiment of the present application, because the target model is a unified programming framework model for deep learning, and the target model can be compatible with multiple frameworks using different conversion rules, thus compatible with the depth of multiple frameworks The method of exporting the learning model to ONNX can support a variety of deep learning framework backends (for example, paddle, mindspore, tensorflow, pytorch), and the computing engine can be changed by switching the backend. Therefore, users do not need to deploy and migrate models. Implement the corresponding framework code for deployment, just switch the backend. Using the method of exporting the deep learning model compatible with multiple frameworks to ONNX in the embodiment of the present application can achieve: for different deep learning frameworks using different conversion rules, the process of model deployment and migration has nothing to do with the computing backend used, thus This avoids the need to design multiple mapping methods due to differences in operator interfaces. In addition, the above method is more general. Furthermore, the above method provides an implementation way to export to ONNX as a high-level deep learning library model.

In the above embodiments, a method for exporting a multi-framework-compatible deep learning model to ONNX is provided. Correspondingly, the present application also provides a device for exporting a multi-framework-compatible deep learning model to ONNX. The device for exporting a multi-framework-compatible deep learning model to ONNX provided in the embodiment of the present application can implement the above-mentioned method for exporting a multi-framework-compatible deep learning model to ONNX, and the device for exporting a multi-framework-compatible deep learning model to ONNX can use software , hardware or a combination of hardware and software. For example, the device for exporting the multi-framework-compatible deep learning model to ONNX may include integrated or separate functional modules or units to perform corresponding steps in the above-mentioned methods.

Please refer to FIG. 6 , which shows a schematic diagram of a device for exporting a multi-framework compatible deep learning model to ONNX provided by some embodiments of the present application. Since the device embodiment is basically similar to the method embodiment, the description is relatively simple, and for relevant parts, refer to the part of the description of the method embodiment. The device embodiments described below are illustrative only.

As shown in Figure 6, the device 600 for exporting the deep learning model compatible with multiple frameworks to ONNX may include:

The obtaining module 601 is used to obtain the layer information of each layer of the target model and the breadth-first search rule. The target model is a deep learning unified programming framework model, and the target model can be compatible with multiple frameworks using different conversion rules;

The traversal module 602 is used to perform layer traversal on each layer of the target deep learning model in order according to the layer information of each layer of the target model and the breadth-first traversal rule to obtain the corresponding model topology;

The mapping module 603 is used to map each layer of the target model in sequence according to the model topology, so as to map the model nodes to the open neural network exchange ONNX;

The generating module 604 is configured to generate a model file including an ONNX model by creating an ONNX calculation graph and creating an ONNX model according to the mapping result of mapping model nodes to ONNX and multiple key functions.

In some implementations of the embodiments of this application, the acquisition module 601 is specifically used to:

Obtain the attribute information of the module node. The attribute information of the module node includes the neural network layer, the input node of the neural network layer of the current layer, the output node of the neural network layer of the current layer, the input value of the neural network layer of the current layer, the neural network layer of the current layer Output value, and the name of the current network layer node;

By calling the module node to perform a forward propagation, the layer information of each layer of the target model is obtained. The layer information of each layer of the target model includes the target neural network layer, and any layer is used as the input of the current layer neural network layer corresponding to the current layer. Node, any layer as the current layer neural network layer output node corresponding to the current layer, any layer as the current layer neural network layer input value corresponding to the current layer, any layer as the current layer neural network layer corresponding to the current layer The output value and any layer are used as the current network layer node name corresponding to the current layer.

In some implementations of the embodiments of the present application, the mapping module 603 is specifically used to:

Get model topology;

Determine the corresponding topological sorting method according to the model topology;

According to the topological sorting method, each layer is mapped sequentially to obtain the corresponding ONNX operator combination to map the model nodes to ONNX.

In some implementations of the embodiments of the present application, the multiple key functions include at least a first function for creating an ONNX calculation graph and a second function for creating an ONNX model, and the generation module 604 is used for:

Obtain the mapping result of mapping model nodes to ONNX;

According to the mapping result, the first function and the second function, create the corresponding ONNX calculation graph and the corresponding ONNX model;

Verify whether the ONNX model is a valid model, and generate a model file including the ONNX model if the ONNX model is verified to be valid.

In some implementations of the embodiments of the present application, the generation module 604 is specifically used to:

In the case that each layer used from TensorLayerX can be mapped to the ONNX unified format, verify that the ONNX model is a valid model.

In some implementations of the embodiments of the present application, the generating module 604 is specifically further configured to:

Validate an ONNX model as an invalid model when TensorLayerX includes at least one layer that cannot be mapped to the ONNX unified format.

In some implementations of the embodiments of the present application, the multiple frameworks using different conversion rules that the target model is compatible with include at least one of the following: TneosFlow, MindSpore, PaddlePaddle, and PyTorch.

In some implementations of the embodiments of the present application, the device 600 for exporting the multi-framework-compatible deep learning model to ONNX provided by the embodiment of the present application is the same as the method for exporting the multi-framework-compatible deep learning model to ONNX provided by the previous embodiments of the present application Based on the same inventive concept, they have the same beneficial effects.

The embodiment of the present application also provides an electronic device corresponding to the device for exporting the multi-framework-compatible deep learning model to ONNX provided in the foregoing embodiment. The electronic device can be an electronic device for the server, such as a server, including an independent server And distributed server clusters, etc., to implement the above method of exporting the deep learning model compatible with multiple frameworks to ONNX; the electronic device can also be an electronic device for the client, such as a mobile phone, a notebook computer, a tablet computer, a desktop computer, etc. Execute the method of exporting the above-mentioned deep learning model compatible with multiple frameworks to ONNX.

Please refer to FIG. 7 , which shows a schematic diagram of an electronic device provided by some embodiments of the present application. As shown in Figure 7, electronic equipment 70 comprises: processor 700, memory 701, bus 702 and communication interface 703, processor 700, communication interface 703 and memory 701 are connected by bus 702; When the processor 700 runs the computer program, it executes the method for exporting the multi-framework-compatible deep learning model to ONNX described above in this application.

Wherein, the memory 701 may include a high-speed random access memory (RAM: Random Access Memory), and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 703 (which may be wired or wireless), and the Internet, wide area network, local network, metropolitan area network, etc. can be used.

The bus 702 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus can be divided into address bus, data bus, control bus and so on. Wherein, the memory 701 is used to store the program, and the processor 700 executes the program after receiving the execution instruction. The method for exporting the compatible multi-framework deep learning model to ONNX disclosed in any implementation mode of the foregoing embodiments of the present application can be applied to In the processor 700, or implemented by the processor 700.

The processor 700 may be an integrated circuit chip with signal processing capability. In the implementation process, each step of the above method may be implemented by an integrated logic circuit of hardware in the processor 700 or an instruction in the form of software. The above-mentioned processor 700 can be a general-purpose processor, including a central processing unit (Central Processing Unit, referred to as CPU), a network processor (Network Processor, referred to as NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), off-the-shelf programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. Various methods, steps, and logic block diagrams disclosed in the embodiments of the present application may be implemented or executed. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like. The steps of the method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the field such as random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, register. The storage medium is located in the memory 701, and the processor 700 reads the information in the memory 701, and completes the steps of the above method in combination with its hardware.

The electronic device provided by the embodiment of the present application is based on the same inventive concept as the method for exporting the multi-framework-compatible deep learning model to ONNX provided by the embodiment of the present application, and has the same beneficial effect as the method adopted, operated or implemented.

The embodiment of the present application also provides a computer-readable medium corresponding to the method of exporting the multi-framework-compatible deep learning model to ONNX provided in the foregoing embodiment. Please refer to FIG. 8, the computer-readable storage medium shown in it is an optical disc 80, a computer program (ie, a program product) is stored thereon, and when the computer program is run by the processor, it will execute the method for exporting the aforementioned deep learning model compatible with multiple frameworks to ONNX.

It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random Access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other optical and magnetic storage media will not be repeated here.

The computer-readable storage medium provided by the above-mentioned embodiments of the present application is based on the same inventive concept as the method for exporting the deep learning model compatible with multiple frameworks to ONNX provided by the embodiments of the present application. The method for realizing the same beneficial effect.

It should be noted that the flowcharts and block diagrams in the accompanying drawings show the architecture, functions and operations of possible implementations of systems, methods and computer program products according to multiple embodiments of the present application. In this regard, each block in a flowchart or block diagram may represent a module, program segment, or part of code that includes one or more Executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved. It should also be noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented by a dedicated hardware-based system that performs the specified function or action , or may be implemented by a combination of dedicated hardware and computer instructions.

Those skilled in the art can clearly understand that for the convenience and brevity of the description, the specific working process of the above-described system, device and unit can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here.

In the several embodiments provided in this application, it should be understood that the disclosed devices and methods may be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or May be integrated into another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some communication interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.

If the functions described above are realized in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes. .

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, rather than limiting them; although the application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: It is still possible to modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the various embodiments of the present application. All of them should be covered by the scope of the claims and description of the present application.

Claims (10)

1. A method of multi-framework compatible deep learning model export to ONNX, the method comprising:

acquiring layer information of each layer of a target model and breadth-first search rules, wherein the target model is a deep learning unified programming framework model, and can be compatible with a plurality of frameworks adopting different conversion rules;

according to the layer information of each layer of the target model and breadth-first traversing rules, sequentially traversing each layer of the target deep learning model to obtain a corresponding model topological structure;

according to the model topological structure, mapping each layer of the target model in sequence to map model nodes to an open neural network exchange ONNX;

and generating a model file comprising the ONNX model by creating an ONNX calculation graph and an ONNX model according to a mapping result of mapping the model node to the ONNX and a plurality of key functions.

2. The method of claim 1, wherein the obtaining layer information for each layer of the object model comprises:

acquiring various attribute information of a module node, wherein the various attribute information of the module node comprises a neural network layer, a current layer neural network layer input node, a current layer neural network layer output node, a current layer neural network layer input value, a current layer neural network layer output value and a current network layer node name;

And carrying out forward propagation once by calling the module node to acquire layer information of each layer of the target model, wherein the layer information of each layer of the target model comprises a target neural network layer, any layer serving as a current layer neural network layer input node corresponding to the current layer, any layer serving as a current layer neural network layer output node corresponding to the current layer, any layer serving as a current layer neural network layer input value corresponding to the current layer, any layer serving as a current layer neural network layer output value corresponding to the current layer and any layer serving as a current network layer node name corresponding to the current layer.

3. The method of claim 1, wherein the mapping each layer of the target model in order to map model nodes to open neural network exchanges ONNX according to the model topology comprises:

acquiring the model topological structure;

determining a corresponding topology ordering mode according to the model topology structure;

and according to the topological ordering mode, mapping each layer sequentially to obtain a corresponding ONNX operator combination so as to map the model nodes to the ONNX.

4. The method of claim 1, wherein the plurality of key functions includes at least a first function for creating an ONNX computation graph and a second function for creating an ONNX model, wherein the generating a model file including the ONNX model by creating the ONNX computation graph and creating the ONNX model based on a mapping result of mapping model nodes to the ONNX and the plurality of key functions comprises:

Obtaining the mapping result of mapping the model node to ONNX;

creating a corresponding ONNX calculation graph and a corresponding ONNX model according to the mapping result, the first function and the second function;

and verifying whether the ONNX model is a valid model, and generating a model file comprising the ONNX model under the condition that the ONNX model is verified to be valid.

5. The method of claim 4, wherein the verifying whether the ONNX model is a valid model comprises:

in case each layer used from TensorLayerX can be mapped to the ONNX unified format, the ONNX model is verified as a valid model.

6. The method of claim 5, wherein the verifying whether the ONNX model is a valid model further comprises:

and under the condition that the TensorLayerX at least comprises one layer which cannot be mapped to the ONNX unified format, verifying the ONNX model as an invalid model.

7. The method of claim 1, wherein the step of determining the position of the substrate comprises,

the multiple frameworks which can be compatible with the target model and adopt different conversion rules comprise at least one of the following:

TneosFlow, mindSpore, paddlePaddle and pyrerch.

8. An apparatus for multi-framework compatible deep learning model export to ONNX, the apparatus comprising:

the system comprises an acquisition module, a breadth-first search module and a breadth-first search module, wherein the acquisition module is used for acquiring layer information of each layer of a target model, the target model is a deep learning unified programming framework model, and the target model can be compatible with a plurality of frameworks adopting different conversion rules;

the traversing module is used for sequentially traversing each layer of the target deep learning model according to the layer information of each layer of the target model and breadth-first traversing rules to obtain a corresponding model topological structure;

the mapping module is used for sequentially mapping each layer of the target model according to the model topological structure so as to map model nodes to an open neural network exchange ONNX;

and the generating module is used for generating a model file comprising the ONNX model by creating an ONNX calculation graph and an ONNX model according to the mapping result of mapping the model node to the ONNX and a plurality of key functions.

9. A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program for executing the method of any of the preceding claims 1 to 7.

10. An electronic device, the electronic device comprising:

a processor;

a memory for storing the processor-executable instructions;

the processor being configured to read the executable instructions from the memory and execute the executable instructions to implement the method of any one of the preceding claims 1 to 7.