CN113052328A

CN113052328A - Deep learning model production system, electronic device, and storage medium

Info

Publication number: CN113052328A
Application number: CN202110363166.XA
Authority: CN
Inventors: 林达华; 曹阳; 李兆松; 张行程; 陈恺; 杨冠姝
Original assignee: Shanghai Sensetime Technology Development Co Ltd
Current assignee: Shanghai Sensetime Technology Development Co Ltd
Priority date: 2021-04-02
Filing date: 2021-04-02
Publication date: 2021-06-29
Anticipated expiration: 2041-04-02
Also published as: WO2022205835A1; CN113052328B

Abstract

The present disclosure relates to a deep learning model production system, an electronic device, and a storage medium, the system including a user development platform, the user development platform including: a production line selection module for displaying a setting list of selected model production lines in response to a selection operation for a model production line in the production line list; the setting module is used for responding to setting operation aiming at a first setting item in the setting list, and determining a target deep learning model for realizing a target task, a training mode of the target deep learning model and a data set for training the target deep learning model; and the training module is used for responding to the training triggering operation aiming at the target deep learning model, training the target deep learning model according to the data set and the training mode, and obtaining the target deep learning model to be deployed. The method and the device can realize efficient setting operation based on the process, and realize automatic generation of the target deep learning model to be deployed.

Description

Deep learning model production system, electronic device, and storage medium

Technical Field

The present disclosure relates to the field of computer technologies, and in particular, to a deep learning model production system, an electronic device, and a storage medium.

Background

The deep learning technology has wide application in the directions of computer vision, natural language processing, voice recognition, recommendation systems and the like. When the deep learning technology is applied, a professional technician is usually relied on to customize a required deep learning model for different application scenarios, for example, data acquisition modes, network type selection, network parameter configuration and the like are customized.

Disclosure of Invention

The present disclosure provides a deep learning model production technical solution.

According to an aspect of the present disclosure, there is provided a deep learning model production system including: the production line selection module is used for responding to selection operation of a model production line in a production line list and displaying a setting list of the selected model production line, wherein the selected model production line is used for generating a target deep learning model to be deployed, the setting list comprises a first setting item of the selected model production line, the first setting item is used for setting the target deep learning model, a training mode of the target deep learning model and a data set used for training the target deep learning model, and the production line list is used for displaying production line information of the selected model production line; the setting module is used for responding to setting operation aiming at a first setting item in the setting list, and determining a target deep learning model for realizing a target task, a training mode of the target deep learning model and a data set for training the target deep learning model; and the training module is used for responding to the training triggering operation aiming at the target deep learning model, training the target deep learning model according to the data set and the training mode, and obtaining the target deep learning model to be deployed.

In one possible implementation manner, the setting operation of the first setting item includes: data set setting operation, network type setting operation and training mode setting operation; the determining, in response to a setting operation for a first setting item in the setting list, a target deep learning model for implementing the target task, a training mode of the target deep learning model, and a data set for training the target deep learning model includes: determining a target deep learning model for achieving the target task in response to a network type setting operation for the target deep learning model; determining a dataset for training the target deep learning model in response to a dataset setting operation for the dataset of the target task; and determining the training mode of the target deep learning model in response to the training mode setting operation aiming at the target deep learning model.

In one possible implementation manner, the training manner includes training equipment for performing training, a training end index, and a specified size of sample data; the training the target deep learning model according to the data set and the training mode to obtain the target deep learning model to be deployed, including: adjusting the size of the sample data in the data set according to the designated size to obtain an adjusted data set; and training the target deep learning model in the training equipment according to the adjusted data set and the training end index to obtain the target deep learning model to be deployed.

In a possible implementation manner, the setting list further includes a second setting item, where the second setting item is used to set package parameters of the target deep learning model to be deployed, and the package parameters include a package manner and/or a deployment device, and the user development platform further includes: a deployment module, configured to package the target deep learning model to be deployed in response to a setting operation for a second setting item in the setting list, to obtain a package file of the target deep learning model to be deployed, where the setting operation for the second setting item includes: setting operation on an encapsulation mode and/or deployment equipment, wherein the encapsulation file is used for deploying the target deep learning model to be deployed in the deployment equipment.

In a possible implementation manner, the setting list further includes a third setting item for importing a data set, a fourth setting item for annotating the data set, and a fifth setting item for evaluating a target deep learning model to be deployed, and the user development platform further includes: an import module, configured to obtain an original data set of the target task in response to an import operation for a third setting item in the setting list, where sample data in the original data set is data that meets a preset data acquisition standard, and the preset data acquisition standard is used to indicate acquisition of the sample data in the original data set; the marking module is used for responding to marking operation aiming at a fourth setting item in the setting list, marking the sample data in the original data set according to a preset data marking standard, and obtaining the data set used for training the target deep learning model; and the evaluation module is used for responding to the setting operation aiming at a fifth setting item in the setting list, and displaying the performance evaluation result of the target deep learning model to be deployed according to a set network evaluation index, wherein the network evaluation index is used for performing performance evaluation on the deep learning model to be deployed.

In one possible implementation, the user development platform further comprises a line purchase item for entering a line store platform in response to a click operation for the line purchase item.

In a possible implementation manner, the system further comprises a production line store platform, the system further comprises the production line store platform, information transmission is carried out between the production line store platform and the user development platform through a communication interface, and the production line store platform is used for selling model production lines.

In one possible implementation, in the case where a model production line is successfully purchased through the production line store platform, production line information of the successfully purchased model production line is added to the production line list.

In a possible implementation manner, the system further includes an expert development platform, the expert development platform performs information transmission with the production line store platform through a communication interface, and the expert development platform includes: the system comprises a production line building module, a model production line setting module and a model generation module, wherein the production line building module is used for responding to building operation aiming at a model production line to obtain a pre-built model production line, and the pre-built model production line is used for generating a target deep learning model to be deployed; and the production line publishing module is used for responding to the publishing operation aiming at the pre-built model production line, publishing the pre-built model production line to the production line shop platform so as to display and purchase the pre-built model production line on the production line shop platform.

In one possible implementation, the building operation comprises a network configuration operation for a model production line; the obtaining of the pre-built model production line in response to the building operation for the model production line comprises: responding to the network configuration operation aiming at the model production line, and obtaining at least one deep learning model, wherein the network configuration operation comprises an operation of configuring at least one of a network structure, a network algorithm and an algorithm parameter; and pre-training the at least one deep learning model to obtain a pre-trained deep learning model in the pre-built model production line, wherein the pre-trained deep learning model corresponds to the network type to be set in a setting module of the user development platform.

In one possible implementation, the building operation further includes: information editing operation, training configuration operation, standard editing operation and index configuration operation aiming at the model production line;

the method for responding to the building operation of the model production line to obtain the pre-built model production line further comprises the following steps: responding to the information editing operation aiming at the model production line, and obtaining production line information of the pre-built model production line, wherein the production line information is used for identifying the pre-built model production line in the production line shop platform and the user development platform; responding to the training configuration operation aiming at the model production line, and obtaining a training mode to be set in the pre-built model production line, wherein the training mode to be set is used for setting in a setting module of the user development platform; responding to the standard configuration operation aiming at the model production line, and obtaining a preset data acquisition standard and a preset data marking standard of the preset model production line, wherein the preset data acquisition standard is used for being displayed in an interface of a import module of the user development platform, and the preset data marking standard is used for being displayed in an interface of a marking module of the user development platform; and responding to the index configuration operation aiming at the model production line to obtain a network evaluation index to be set in the pre-built model production line, wherein the network evaluation index to be set is used for setting an evaluation module of the user development platform.

In one possible implementation, the target task includes an image processing task, and the image processing task includes: at least one of image recognition, image segmentation, image classification, and keypoint detection.

According to an aspect of the present disclosure, there is provided an electronic device including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke the memory-stored instructions to execute the above-described system.

According to an aspect of the present disclosure, there is provided a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the above-described system.

In the embodiment of the disclosure, a flow setting operation for generating a target deep learning model to be deployed can be realized on the basis of a pre-constructed model production line on a user development platform, so that the target deep learning model to be deployed can be automatically generated on the basis of the flow setting operation efficiently when a training triggering operation for the target deep learning model is responded. The problem of prior art can't satisfy the high-efficient, flow, the automatic generation demand of industry to neural network is solved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments, which proceeds with reference to the accompanying drawings.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure.

FIG. 1 shows a block diagram of a deep learning model production system according to an embodiment of the disclosure.

FIG. 2 illustrates a block diagram of a user development platform in accordance with an embodiment of the present disclosure.

FIG. 3 illustrates a schematic diagram of a line management interface according to an embodiment of the present disclosure.

FIG. 4 illustrates a schematic diagram of a list of production lines implemented in accordance with the present disclosure.

FIG. 5 illustrates a schematic diagram of a setup interface for a first setup item according to an embodiment of the disclosure.

FIG. 6 illustrates a schematic diagram of a setup interface for a second setup item, according to an embodiment of the disclosure.

FIG. 7 is a schematic diagram illustrating a setup interface for a third setup item according to an embodiment of the disclosure.

Fig. 8 shows a schematic diagram of a setting interface of a fourth setting item according to an embodiment of the present disclosure.

Fig. 9 shows a schematic diagram of a setting interface of a fifth setting item according to an embodiment of the present disclosure.

FIG. 10 shows a block diagram of a line shop platform according to an embodiment of the present disclosure.

FIG. 11 illustrates a schematic diagram of a production line presentation interface, according to an embodiment of the present disclosure.

Fig. 12 illustrates a block diagram of an expert development platform in accordance with an embodiment of the present disclosure.

FIG. 13 illustrates a schematic view of a management interface for a model production line in accordance with an embodiment of the present disclosure.

FIG. 14 illustrates a schematic diagram of a network configuration interface, according to an embodiment of the disclosure.

Fig. 15 shows a schematic diagram of an information editing interface according to an embodiment of the present disclosure.

FIG. 16 illustrates a schematic diagram of a training configuration interface, according to an embodiment of the present disclosure.

FIG. 17 illustrates a schematic diagram of a standard configuration interface in accordance with an embodiment of the present disclosure.

FIG. 18 shows a block diagram of an electronic device in accordance with an embodiment of the disclosure.

FIG. 19 shows a block diagram of an electronic device in accordance with an embodiment of the disclosure.

Detailed Description

Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. In the drawings, like reference numbers can indicate functionally identical or similar elements. While the various aspects of the embodiments are presented in drawings, the drawings are not necessarily drawn to scale unless specifically indicated.

The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration. Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.

The term "and/or" herein is merely an association describing an associated object, meaning that three relationships may exist, e.g., a and/or B, may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B, C, and may mean including any one or more elements selected from the group consisting of A, B and C.

It should be understood that the terms "first," "second," and "third," etc. in the claims, description, and drawings of the present disclosure are used for distinguishing between different objects and not for describing a particular order. The terms "comprises" and "comprising," when used in the specification and claims of this disclosure, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

Furthermore, in the following detailed description, numerous specific details are set forth in order to provide a better understanding of the present disclosure. It will be understood by those skilled in the art that the present disclosure may be practiced without some of these specific details. In some instances, methods, means, elements and circuits that are well known to those skilled in the art have not been described in detail so as not to obscure the present disclosure.

FIG. 1 shows a block diagram of a deep learning model production system according to an embodiment of the disclosure. As shown in fig. 1, the deep learning model production system includes:

the expert development platform 11 is used for building a model production line and releasing a pre-built or built model production line to the production line shop platform 12, wherein the pre-built or built model production line is used for generating a target deep learning model to be deployed;

a production line shop platform 12 for displaying and selling pre-built or built model production lines to add the purchased model production lines to the user development platform 13;

and the user development platform 13 is used for generating a target deep learning model to be deployed based on the purchased model production line.

In one possible implementation, the deep learning model production system may execute the method steps in a system at an electronic device such as a terminal device or a server, where the terminal device may be a User Equipment (UE), a mobile device, a User terminal, a cellular phone, a cordless phone, a Personal Digital Assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, and the like, and the method steps in the system may be implemented by a processor in the terminal device calling computer readable instructions stored in a memory, or the method steps in the system may be executed by the server.

In one possible implementation, the production line store platform 12 and the user development platform 13 perform information transmission through a communication interface, and the expert development platform 11 and the production line store platform 12 perform information transmission through a communication interface. That is, the expert development platform 11, the production line shop platform 12 and the user development platform 13 may realize information interaction by calling a communication interface.

It should be understood that the expert development platform 11, the production line shop platform 12, and the user development platform 13 may be independently developed applications or integrated applications, and the embodiment of the present disclosure is not limited thereto. The development of the expert development platform 11, the production line shop platform 12 and the user development platform 13 can be realized by any known technology by those skilled in the art, and the embodiment of the present disclosure is not limited thereto.

According to the embodiment of the disclosure, an expert development platform for building a model production line of a professional technician can be provided, the expert development platform is used for purchasing a shop user platform of a pre-built or built model production line, and a user development platform for generating a target deep learning model to be deployed by a common user by utilizing the pre-built or built model production line is beneficial to the generation of the deep learning model to be deployed by the common user, so that the model production line built by the professional technician can be purchased and used by the common user to generate the deep learning model to be deployed on equipment, and the common user can approach the level of the professional technician based on the efficiency and the precision of the deep learning model to be deployed generated by the pre-built or built model production line, and has higher practicability and universality.

As described above, the deep learning model production system includes a user development platform, and fig. 2 illustrates a block diagram of the user development platform according to an embodiment of the present disclosure, and as illustrated in fig. 2, the user development platform includes:

a line selection module 131 for presenting a list of settings of the selected model lines in response to a selection operation for a model line in the line list.

The selected model production line is used to generate a target deep learning model to be deployed, for example, the model production line may be a vehicle detection model production line, and is used to generate a vehicle detection model, and the like, which is not limited in this disclosure. The setting list comprises a first setting item of the selected model production line, the first setting item is used for setting a target deep learning model, a training mode of the target deep learning model and a data set used for training the target deep learning model, and the production line list is used for displaying production line information of the selected model production line.

In one possible implementation, a production line list may be provided in the production line selection module, and production line information of model production lines purchased in a production line shop by a user may be displayed in the production line display list, so that the selected model production line setting list is displayed in response to a selection operation for the model production lines. It should be understood that the selected model production line setting list may be displayed in a display interface of the terminal device.

In one possible implementation manner, a production line list, such as a production line name, a production line identifier, and the like, may be displayed in a display interface of the terminal device, so that a user can select a model production line. It should be understood that a trigger control responsive to the selection operation, for example, a trigger control responsive to a click operation or a touch operation, may be included in the production line list to determine the model production line selected by the user. The embodiments of the present disclosure are not limited to the implementation of the trigger control.

FIG. 3 illustrates a schematic diagram of a line management interface according to an embodiment of the present disclosure. FIG. 4 illustrates a schematic diagram of a list of production lines implemented in accordance with the present disclosure. The line list shown in fig. 4 may be a list expanded by clicking on the line information "AAA" of the model lines shown in fig. 3, which may facilitate a user to select any one of the model lines and to present a setting list of the selected model line in response to a selection operation for any one of the model lines. It should be understood that a trigger control responsive to a click operation on a model production line may be included in the production line list, as well as a control (e.g., a slider) for pulling up and down the slide list.

The display under "AAA" in fig. 3 may be a setting list of the selected model production line, and names, or identifiers, etc. of various setting items of the model production line may be displayed in the setting list, so as to display a setting interface of the selected setting item in response to a click operation of a user on any setting item, thereby facilitating the user to implement a setting operation for generating a target deep learning model to be deployed in the displayed setting interface. It should be understood that a trigger control responsive to a click operation on a setting item may be included in the setting list.

It should be noted that the first, second, third, and the like in this disclosure may be used to distinguish different setting items, and do not limit the positions of the setting items in the setting list. It should be understood that a person skilled in the art may set the position of the first setting item in the setting list according to actual requirements, for example, the setting item c in fig. 3 may be the first setting item.

The setting module 132 is configured to determine, in response to a setting operation for a first setting item in the setting list, a target deep learning model for implementing the target task, a training manner of the target deep learning model, and a data set for training the target deep learning model.

In one possible implementation, the target task may include an image processing task. The image processing tasks include: at least one of image recognition, image segmentation, image classification, and keypoint detection. It should be understood that the target tasks may also include speech processing tasks, such as speech recognition tasks, semantic recognition tasks, voice print recognition tasks, natural language processing tasks, and the like. The disclosed embodiments are not limited with respect to the task type of the target task.

In one possible implementation manner, the target deep learning model, the training manner and the data set can be set in the setting interface of the first setting item. FIG. 5 illustrates a schematic diagram of a setup interface for a first setup item according to an embodiment of the disclosure. As shown in fig. 5, a drop-down box for setting a data set, a drop-down box for setting a network type, a drop-down box for setting a training end index, and a drop-down box for setting a training device for performing training may be provided in the setting interface of the first setting item.

The network type can be used for indicating deep learning models under different network structures, network algorithms and algorithm parameters, the deep learning model indicated by the network type can be a pre-trained deep learning model, and a target deep learning model for realizing a target task can be determined by setting the network type. The data set in the first setting item may be a data set used for training the target deep learning model, and the data set may be a labeled data set, and it is understood that a plurality of labeled data sets may be provided for setting.

The training mode may include a training end indicator and a training device (e.g., a GPU, a CPU of an X86, etc.) for performing training. The training end index is used for indicating a condition for ending the training of the target deep learning model, for example, the number of iteration rounds reaches a set threshold, the iteration duration reaches a set threshold, and the like. The training mode of the target deep learning model can be determined by setting a training end index and training equipment.

It should be understood that a user can display a corresponding drop-down list by clicking each drop-down box in the setting interface, so that the user can set and operate the target deep learning model, the training mode and the data set according to actual requirements. And the target deep learning model, the training mode and the data set by the user are determined.

The setting interface of the first setting item may provide a relevant drop-down box control to implement the setting operation for the first setting item, which is not limited to this embodiment of the present disclosure. Of course, the setting operation for the first setting item may also be implemented by other types of controls, such as a multi-box, and the like, and the embodiment of the present disclosure is not limited thereto.

It should be noted that the setting content shown in the setting interface of the first setting item in fig. 5 is an implementation manner provided by the embodiment of the present disclosure. It should be understood that the present disclosure should not be limited thereto, and those skilled in the art can refine the settings for the target deep learning model, the training mode and the data set according to actual needs.

For example, the data set may be divided into a training set and a test set, and the duty ratio of the training set to the test set may be set; in the case where the training end index is set to stop in the iteration turn, an iteration turn threshold may also be set to end the training of the target deep learning model according to the set iteration turn threshold. The setting content in the first setting item for generating the target deep learning model to be deployed may be determined according to actual needs, and the embodiments of the present disclosure are not limited thereto.

The training module 133 is configured to respond to a training trigger operation for the target deep learning model, train the target deep learning model according to the data set and the training mode, and obtain the target deep learning model to be deployed.

In one possible implementation, the training for the target deep learning model is triggered, for example, by clicking a trigger control at "start training" in the setting interface of the first setting item shown in fig. 5, and training for the target deep learning model is started by responding to the triggering for the trigger control. It should be understood that the style, position, implementation manner, etc. of the trigger control may be determined according to actual requirements, and the embodiments of the present disclosure are not limited thereto.

As described above, the training mode may include a training end index and a training apparatus for performing training. In a possible implementation manner, training a target deep learning model according to a data set and a training manner to obtain a target deep learning model to be deployed may include: and according to the data set and the training end index, executing training of the target deep learning model in the set training equipment to obtain the target deep learning model to be deployed. The embodiments of the present disclosure are not limited to the training process of the target deep learning model.

In the embodiment of the disclosure, a flow setting operation for generating a target deep learning model to be deployed can be realized based on a pre-constructed model production line, so that when a training triggering operation for the target deep learning model is responded, the target deep learning model to be deployed can be automatically generated based on the flow setting operation efficiently.

As described above, the setting operations for the target deep learning model, the training mode, and the training set may be performed in the setting interface of the first setting item, and in one possible implementation, the setting operations of the first setting item include: a data set setting operation, a network type setting operation, and a training mode setting operation, the determining a target deep learning model for realizing a target task, a training mode of the target deep learning model, and a data set for training the target deep learning model in response to the setting operation for a first setting item in a setting list, including:

determining a target deep learning model for realizing a target task in response to a network type setting operation for the target deep learning model; determining a dataset for training a target deep learning model in response to a dataset setting operation for the dataset of the target task; and determining the training mode of the target deep learning model in response to the training mode setting operation aiming at the target deep learning model.

As described above, the network type may be used to indicate a deep learning model under different network structures, network algorithms, and algorithm parameters, the deep learning model indicated by the network type may be a pre-trained deep learning model, and a target deep learning model for implementing a target task may be determined by setting the network type.

As described above, the training mode may include a training device for performing training and a training end index. The training mode setting operation may include setting operations for the training device and the training end index.

In a possible implementation manner, in a case that the sample data is image data, the training manner may further include a specified size of the sample data. Correspondingly, the training mode setting operation may further include a setting operation of a specified size of the sample data. Wherein the specified size may include a specified image resolution. By the method, the sample data in the data set can be adjusted to the sample data under the specified size, so that the sample data in the data set can meet the size requirement of the target deep learning model on the input data in the training process, and the training effect of the target deep learning model is improved.

In a possible implementation manner, a default specified size may also be adopted to adjust sample data in the data set, and in this case, the specified size of the sample data may not be set; or in the case where the sample data is non-image data (such as voice), the designated size of the sample data is not set. It should be understood that, in such cases, the setting contents of the specified size for the sample data may not be presented in the setting interface of the first setting item.

In a possible implementation manner, network type setting operation, data set setting operation and training mode setting operation for the target deep learning model can be achieved by clicking each drop-down box control provided in the setting interface of the first setting item and in a drop-down list displayed by triggering each drop-down box control.

The network type setting operation may be, for example, clicking a network type identifier shown in a corresponding drop-down list; the data set setting operation may be, for example, clicking on a data set identifier shown in the corresponding drop-down list; the training mode setting operation may be, for example, clicking a training mode flag in the corresponding drop-down list. It should be understood that a trigger control for each setting operation can be provided in the drop-down list to determine a target deep learning model corresponding to the data set, the training mode and the network type selected by the user.

In the embodiment of the disclosure, the target deep learning model, the data set for training the target deep learning model, and the training mode of the target deep learning model can be conveniently determined through network setting operation, data set setting operation, and training mode setting operation, so that the automatic generation of the target deep learning model to be deployed is conveniently and efficiently realized.

As described above, the training mode may include a training device for performing training, a training end index, and a specified size of sample data, and in a possible implementation, the training the target deep learning model according to the data set and the training mode to obtain the target deep learning model to be deployed includes:

adjusting the size of sample data in the data set according to the specified size to obtain an adjusted data set;

and training the target deep learning model in the training equipment according to the adjusted data set and the training end index to obtain the target deep learning model to be deployed.

As described above, the sample data may include image data and the specified size may include a specified image resolution. Adjusting the size of sample data in the data set according to the specified size to obtain an adjusted data set, which may include: and adjusting the image resolution of the image data in the data set to the specified image resolution to obtain the adjusted data set. The image resolution of the image data may be adjusted by using any known image processing technology, for example, normalization processing, scaling processing, and the like, which is not limited in this disclosure.

In a possible implementation manner, the training device for performing training may be a training device set for a training manner in the first setting item; it is to be understood that a default training device may also be used, i.e., no setting may be made for the training device, and the present disclosure is not limited in its practice. The target deep learning model is trained in the training device, that is, the training of the target deep learning model is performed in the training device.

In one possible implementation, the training end index may be a training end index set for the training mode in the first setting item; it is understood that a default training end index may also be used, that is, the training end index may not be used as a device, and the present disclosure is not limited in implementation.

It is to be understood that the setting operation for the training mode in the first setting item may include a setting operation for at least one of a training apparatus for performing training, a training end index, and a specified size of sample data.

In one possible implementation, the adjusted data set, the set training end index, and the target deep learning model may be transmitted to a training device for training the target deep learning model, so as to perform training of the target deep learning model in the training device.

In a possible implementation manner, training a target deep learning model in a training device according to the adjusted data set and a training end index to obtain a target deep learning model to be deployed, which may include: inputting the sample data in the adjusted data set into a target deep learning model to obtain an output result; adjusting network parameters of a target deep learning model according to the loss of the output result; and under the condition that the adjusted target deep learning model meets the training end index, obtaining the target deep learning model to be deployed. It should be understood that any known deep learning model training system may be used to train the target deep learning model, and the disclosed embodiments are not limited thereto.

In the embodiment of the disclosure, the training of the target deep learning model can be automatically realized efficiently according to the set training set and the training mode, and the target deep learning model to be deployed, which meets the actual requirements of the user, is obtained.

It is considered that the trained deep learning model is generally packaged, i.e., packaged, to implement the deployment of the deep learning model in the relevant device. In a possible implementation manner, the setting list further includes a second setting item, and the second setting item is used to package a target deep learning model to be deployed, and the system further includes:

in response to the setting operation for a second setting item in the setting list, packaging the target deep learning model to be deployed to obtain a packaged file of the target deep learning model to be deployed, wherein the setting operation for the second setting item comprises: and setting operation on the packaging mode and/or the deployment equipment, wherein the packaging file is used for deploying the target deep learning model to be deployed in the deployment equipment.

The packaging mode can indicate an inference framework required for packaging the target deep learning model to be deployed, and the inference framework can enable the target deep learning model to be deployed to be adaptive to various deployment devices. The Inference framework may include, for example, TensorRT (a high performance deep learning model Inference (Inference) engine, introduced by England, for deploying applications for deep learning models), OpenVINO (a toolset, introduced by Intel, for deploying deep learning models).

The deployment device may indicate a processor type of the device to be deployed for the target deep learning model to be deployed, which may include, for example: an X86 processor, an Arm processor, a CUDA (computer Unified Device Architecture) processor, a GPU, and the like.

The package file may be, for example, an SDK (Software Development Kit), wherein any known packaging technology may be adopted to package the target deep learning model to be deployed, and the embodiment of the present disclosure is not limited thereto.

FIG. 6 illustrates a schematic diagram of a setup interface for a second setup item, according to an embodiment of the disclosure. As shown in fig. 6, a drop-down box for setting a packaging manner, a drop-down box for setting a deployment device, a drop-down box for setting a network version, and a radio button for setting whether deployment is quantized or not may be provided in the setting interface of the second setting item.

The network version is used for selecting different versions of target deep learning models to be deployed, and it should be understood that multiple setting operations can be performed based on the first setting item, that is, multiple training of the target deep learning models is performed to obtain multiple versions of the target deep learning models to be deployed; and whether the deployment is quantized or not is used for indicating whether the target deep learning model to be deployed is quantized or not, and the quantized target deep learning model is packaged. Any known quantization technology can be adopted to realize the quantization of the target deep learning model to be deployed, and the embodiments of the present disclosure are not limited thereto.

It should be understood that a user can display a corresponding drop-down list by clicking each drop-down box in the setting interface, so that the user can set the packaging mode and the deployment equipment according to actual requirements. The packaging mode, the deployment equipment and the like set by the user are also determined.

And a setting interface of the second setting item may provide a relevant drop-down box control to implement a setting operation for the second setting item, which is not limited in this embodiment of the disclosure. Of course, the setting operation for the second setting item may also be implemented by other types of controls, such as a multi-box, and the like, and the embodiment of the present disclosure is not limited thereto.

In one possible implementation, in response to the setting operation for the second setting item in the setting list, a triggering operation of triggering the target deep learning model to be deployed to start packaging may also be included, for example, the target deep learning model to be deployed may be triggered to be packaged by clicking a trigger control at "confirmation" in the setting interface shown in fig. 6. It should be understood that the style, position, implementation manner, etc. of the trigger control may be determined according to actual requirements, and the embodiments of the present disclosure are not limited thereto.

It should be understood that the setting interface of the second setting item shown in fig. 6 above is one implementation manner provided by the embodiment of the present disclosure. The content set in the setting interface of the second setting item can be adjusted by a person skilled in the art according to actual needs, and the embodiment of the present disclosure is not limited thereto. For example, in some cases, the target deep learning model to be deployed may be quantified by default, and the radio button for whether quantification is to be deployed in the setting interface of the second setting item may be eliminated.

In the embodiment of the disclosure, various packaging requirements of a user on a target deep learning model to be deployed can be met, and the obtained packaged file is effectively deployed in various deployment devices.

In a possible implementation manner, the setting list further includes a third setting item for importing a data set, a fourth setting item for annotating the data set, and a fifth setting item for evaluating a target deep learning model to be deployed, and the user development platform further includes:

the import module is used for responding to import operation aiming at a third setting item in the setting list and obtaining an original data set of the target task, wherein sample data in the original data set meets a preset data acquisition standard, and the preset data acquisition standard is used for indicating acquisition of the sample data in the original data set;

the marking module is used for responding to marking operation aiming at a fourth setting item in the setting list, marking sample data in the original data set according to a preset data marking standard, and obtaining a data set for training a target deep learning model;

and the evaluation module is used for responding to the setting operation aiming at the fifth setting item in the setting list, and displaying the performance evaluation result of the target deep learning model to be deployed according to the set network evaluation index, wherein the network evaluation index is used for performing performance evaluation on the deep learning model to be deployed.

In one possible implementation manner, a preset data collection standard may be displayed in the setting interface of the third setting item to guide the user to collect the original sample set meeting the preset data collection standard, for example, the data collection standard of the vehicle image may include: the shooting height is 3.5-7 m, the head and the whole body are shot, and the resolution is 1080 p; a minimum of 100 images, no more than 100 cars per image, etc.

In a possible implementation manner, the setting interface of the third setting item may include a file upload control for importing the data set, for example, the data set is imported by dragging a file, or the data set is imported by searching a local storage path of the data set, and the import manner of the data set is not limited in the embodiment of the present disclosure.

It should be understood that more than one raw data set of the target task may be imported. The imported raw data set can be transmitted to other electronic equipment and stored so as to facilitate automatic labeling of the raw data set.

FIG. 7 is a schematic diagram illustrating a setup interface for a third setup item according to an embodiment of the disclosure. As shown in fig. 7, the storage path of the original data set in the local can be determined by clicking the control at the "selection" position; and triggering to acquire the original data set according to the storage path of the original data set by clicking an import button, thereby realizing the import operation of the original data set.

As described above, the original data set of the target task may include a plurality of data sets, and in a possible implementation manner, a drop-down box for selecting the original data set may be displayed in the setting interface of the fourth setting item, so that a user may select any original data set for labeling, and a data set for training the target deep learning model is obtained. It should be understood that the annotation operation for the fourth setting item may include a selection operation for the original data set.

In one possible implementation, the preset data labeling standard may be a preset data labeling standard, for example, a labeling box with a filtering size smaller than 30 × 30. The preset data labeling standard may be displayed in the setting interface of the fourth setting item to inform the user of the preset data labeling standard, and certainly, a plurality of preset data labeling standards may also be provided for the user to select, which does not limit the embodiment of the present disclosure.

In one possible implementation, any known data labeling technique may be used to label the sample data in the original data set according to a preset data labeling standard, for example, the original technology set may be automatically labeled by a data labeling algorithm. In this case, the tagging operation for the fourth setting item may further include a trigger operation of triggering to start tagging of sample data in the original data set.

In a possible implementation manner, an annotation tool (e.g., LabelMe: a Javascript annotation tool for online image annotation) for manual annotation can be further provided in the setting interface of the fourth setting item to manually annotate the original data set. In this case, the annotation operation for the fourth setting item may further include an annotation operation of manually annotating the original data set. The labeling mode of the original data set can be set according to actual requirements, and the embodiment of the disclosure is not limited.

FIG. 8 is a schematic diagram of a setting interface of a fourth setting item according to an embodiment of the disclosure. As shown in fig. 8, a drop-down box control for selecting an original data set is shown in the setting interface of the fourth setting item, and it should be understood that the original data set shown in the drop-down box may be the selected original data set; the user can click a button at the position of the automatic labeling to realize the automatic labeling of the selected original data set; and a button at the position of manual marking can be clicked to trigger and display a marking tool for manual marking.

In one possible implementation manner, the network evaluation index is used for evaluating the performance of the trained target deep learning model. The network evaluation index may include, for example: at least one of accuracy, precision, recall, F1 Score (F1-Score).

It should be understood that the setting operation for the fifth setting item may include a setting operation for the network evaluation index to present the performance evaluation result in response to the set network evaluation index. Fig. 9 is a schematic diagram illustrating a setting interface of a fifth setting item according to an embodiment of the disclosure, and as shown in fig. 9, setting of a network evaluation index may be implemented in the form of a multi-selection box. Wherein, the performance evaluation result can be triggered and displayed by clicking the 'confirm' button in fig. 9. It should be understood that the setting operation for the fifth setting item may include a trigger operation of triggering the presentation performance evaluation result.

In a possible implementation manner, a confidence threshold option may be further included in the network evaluation index setting interface, so that a user can view performance evaluation results under different confidence thresholds conveniently. The confidence level may be a confidence level of an output result of the deep learning model, for example, a confidence level of a vehicle detection, which may characterize a possibility that the detection result is indicated as a vehicle, and the confidence threshold may be used to indicate a performance evaluation result when the target deep learning model reaches the confidence threshold.

In one possible implementation, the performance evaluation results may be presented in the form of a list, a chart (e.g., a graph), or the like. The display form of the performance evaluation result can be set according to actual requirements, and the embodiment of the disclosure is not limited.

It should be noted that the setting interfaces of the third setting item, the fourth setting item, and the fifth setting item respectively shown in fig. 7, fig. 8, and fig. 9 are an implementation manner provided by the embodiment of the present disclosure. It should be understood that the present disclosure should not be limited thereto, and those skilled in the art can design the setting interfaces of the third setting item, the fourth setting item and the fifth setting item according to the actual requirement, and the embodiments of the present disclosure are not limited thereto.

In the embodiment of the disclosure, the import and labeling of the original data set can be realized, so that the setting of the training set is facilitated when the target deep learning model is trained; and displaying the performance evaluation result of the target deep learning model to be deployed, so that a user can know whether the training effect of the target deep learning model to be deployed meets the performance requirement or not.

In one possible implementation, the user development platform further comprises a line purchase item for entering the line store platform in response to a click operation for the line purchase item. By the mode, a user can conveniently enter the production line shop platform to purchase the pre-built or built model production line at any time.

Wherein, the production line purchasing item can be realized by the form of an entrance button of a production line shop platform; the production line purchase item can be correspondingly provided with a jump address of the production line store platform, so that a user can enter the production line store platform based on the jump address when clicking the production line purchase item. The embodiments of the present disclosure are not limited with respect to the implementation of the line purchase.

In one possible implementation, the line purchasing item may be set at any position of any interface in the user development platform, for example, in a line list, a setting list of model lines, and a setting interface of each setting item, which is not limited by this disclosed embodiment.

As mentioned above, the deep learning model production system further comprises a production line shop platform, information transmission is carried out between the production line shop platform and the user development platform through a communication interface, and the production line shop platform is used for selling model production lines. FIG. 10 shows a block diagram of a line shop platform according to an embodiment of the present disclosure, as shown in FIG. 10, comprising:

and the production line display module 121 is used for displaying a pre-built or built model production line, wherein the pre-built or built model production line is a model production line built through an expert development platform.

In one possible implementation, the pre-built or built model production line may be displayed in a production line display interface of a production line store platform. FIG. 11 is a schematic diagram of a production line display interface according to an embodiment of the disclosure, as shown in FIG. 11, a "buy" button for purchasing a model production line may be provided in the production line display interface, and a user may enter a payment page by clicking the "buy" button; the user can click a detail button in the production line display interface to check the detailed information of the selected model production line; the "search" case can also be clicked on to search for the model production line to be purchased.

It should be understood that the production line display interface shown in fig. 11 is one implementation manner provided by the embodiment of the present disclosure, and it should be understood that the present disclosure should not be limited thereto, and a person skilled in the art may design the production line display interface according to actual requirements, and the embodiment of the present disclosure is not limited thereto.

The building process of the pre-built or built model production line will be explained below, and is not described herein for brevity.

A line purchasing module 122 for determining a purchased model production line in response to a purchasing operation for the displayed model production line.

In one possible implementation, the purchasing operation for the displayed model production line may include: a click operation for a "buy" button shown in the production line presentation interface shown in fig. 11. After clicking the "buy" button, the payment interface may be entered. It should be understood that a payment interface may provide a payment control for making online payments, and those skilled in the art may implement online payments for model production lines using any known technique, and the embodiments of the present disclosure are not limited to the implementation of online payments.

It should be understood that the model line for which payment has been made may be the model line for which a successful purchase was made. In one possible implementation, in the case of successfully purchasing a model production line through the production line store platform, the production line information of the successfully purchased model production line is added in the production line list of the user development platform, so that the user can generate the target deep learning model to be deployed by using the successfully purchased model production line. The implementation manner of the production line list may refer to the manner disclosed in the embodiment of the present disclosure, and is not described herein again.

In one possible implementation, the production line shop platform may further include a production line sending module for sending the successfully purchased model production line to the user development platform to display the successfully purchased model production line in a production line display list of the user development platform, wherein the production line display list is used for displaying and selecting the successfully purchased model production line.

In embodiments of the present disclosure, a user may purchase different model production lines through a production line store platform to generate target deep learning models to be deployed that achieve different target tasks.

As described above, the deep learning model production system may include an expert development platform. Fig. 12 illustrates a block diagram of an expert development platform, as shown in fig. 12, including:

the production line building module 111 is configured to respond to a building operation for a model production line to obtain a pre-built or built model production line, where the pre-built or built model production line is used to generate a target deep learning model to be deployed.

FIG. 13 illustrates a schematic view of a management interface for a model production line in accordance with an embodiment of the present disclosure. In a possible implementation manner, information of the built model production line can also be displayed in a management interface as shown in fig. 13, and as shown in fig. 13, a person in charge can be the name of a person who builds the model production line. It should be appreciated that controls (e.g., slider bars) for pulling up and down a list may be provided in the management interface to expose more model lines.

As shown in fig. 13, the management interface of the model production line may further provide a search box (a box on the left side of "search" in fig. 13) for searching the model production line and a search trigger control (a "search" in fig. 13) to quickly search out the model production line; clicking the trigger control at the detail position corresponding to any model production line in the figure 13 to display more detailed information of the model production line, clicking the trigger control at the editing position in the figure 13 to edit the model production line, clicking the trigger control at the publishing position in the figure 13 to publish the pre-built or built model production line to a production line shop platform.

In one possible implementation, the building interface of the model production line can be entered by clicking the trigger control at "create production line" in fig. 13, so as to perform building operation on the model production line. The entering mode of the building interface of the model production line can be determined according to actual requirements, and the embodiment of the disclosure is not limited. It should be understood that various controls on the interface can be provided in the construction interface, such as an edit box, a drop-down box, a multi-box, a radio box, etc., to implement the construction operation for the model production line.

In one possible implementation, the building operation for the model production line may include: network configuration operations, information editing operations, training configuration operations, standard editing operations, and index configuration operations for a model production line.

Wherein the network configuration operation is operable to configure at least one pre-trained deep learning model in the model production line; an information editing operation for editing production line information of the model production line; training configuration operation, which is used for configuring a training mode to be set in a model production line; the standard configuration operation is used for configuring a preset data acquisition standard and a preset data marking standard of the model production line; and the index configuration operation is used for configuring the network evaluation index to be set in the model production line.

It will be appreciated that after completion of the set-up operations described above for the model production line, a pre-set-up or set-up model production line may be obtained. The configuration results corresponding to each building operation may be connected in series by using a known component series connection tool to obtain a pre-built or built model production line, which is not limited in the embodiment of the present disclosure.

And a production line publishing module 112, configured to, in response to a publishing operation for the pre-built or built model production line, publish the pre-built or built model production line to the production line shop platform to display and purchase the pre-built or built model production line on the production line shop platform.

In one possible implementation, the pre-built or built model production line may be displayed in a management interface as shown in fig. 13, and the release of the pre-built or built model production line to the production line shop platform may be triggered by clicking a "release" button shown in fig. 13. It should be understood that, a person skilled in the art may set a distribution manner of the pre-built or built model production line according to actual needs, and the embodiment of the present disclosure is not limited thereto.

As described above, the expert development platform and the production line shop platform transmit information through the communication interface, and the pre-built or built model production line is released to the production line shop platform, and all configuration contents included in the pre-built or built model production line may be packaged and compressed, and the packaged and compressed file is sent to the production line shop platform. The disclosed embodiments are not limited with respect to how pre-built or built model production lines are released to production line store platforms.

The pre-built or built model production model is displayed and purchased on the production line shop platform, and reference may be made to the content in the embodiment of the present disclosure, which is not described herein again.

In the embodiment of the disclosure, a model production line can be built by using an expert development platform, so that a common user can use the model production line built by professional technicians to generate a target deep learning model to be deployed, and the production efficiency and precision of the target deep learning model to be deployed are improved.

As mentioned above, the building operation includes a network configuration operation for a model production line, and in one possible implementation, in response to the building operation for the model production line, obtaining a pre-built or built model production line includes:

responding to network configuration operation aiming at the model production line, and obtaining at least one deep learning model, wherein the network configuration operation comprises operation of configuring at least one of a network structure, a network algorithm and an algorithm parameter;

and pre-training at least one deep learning model to obtain a pre-built or pre-trained deep learning model in a built model production line, wherein the pre-trained deep learning model corresponds to the network type to be set in a setting module of a user development platform.

The network structure may include a backbone network backbone, a convergence network tack, and a branch network head of the deep learning model. The main network is used for extracting features, the fusion network is used for carrying out fusion processing on the features extracted by the main network, and the branch network is used for carrying out reasoning of different tasks aiming at the features extracted by the main network or the features subjected to fusion processing.

In a possible implementation manner, the network structures of the backbone network backbone, the convergence network tack and the branch network head may at least adopt: ResNet series, SENEt series, MobileNet series, ShuffleNet series, etc. The network algorithm may include at least: fast convolution fast R-CNN algorithm, key point convolution Keypoint R-CNN algorithm, ladder convolution Cascade R-CNN algorithm, normalization algorithm and the like. The algorithm parameters at least include: batch size, sampling mode, data enhancement mode, network parameters, etc.

Wherein, the batch size can be understood as the number of sample data of one training; the sampling mode can at least comprise: positive and negative Sample sampling mode, Hard Sample (Hard Sample) sampling mode, undersampling mode, oversampling mode and the like, wherein the data enhancement mode can comprise: data enhancement modes such as random cutting, distortion, amplification, mirror image, deformation and the like; the network parameters can include the hyper-parameters of the network, and the hyper-parameters can be determined by selecting the existing parameter configuration files due to more types of the hyper-parameters.

It should be understood that the network configuration operations for the model production line, i.e., the deep learning model, may include more than the configurations disclosed in the embodiments of the present disclosure described above; and the network configuration operation can be realized by human-computer interaction through the display interface, or can be realized by calling the relevant interface and the relevant configuration file through the background, as long as the deep learning model can be configured, and the embodiment of the disclosure is not limited.

It should be understood that different deep learning models can be obtained in different configuration modes, and the network size, the precision and the operation performance of different deep learning models can be different. For example, the network algorithms may be different for the same network structure; for the same network structure and network algorithm, the algorithm parameters may be different; the deep learning model with high precision is large in size; the small-size deep learning model has low precision and good operation performance. By the method, the deep learning models with different accuracies and different operation performances, namely the deep learning models with different network types can be configured, so that a user can select the target deep learning model according to accuracy requirements and performance requirements.

FIG. 14 illustrates a schematic diagram of a network configuration interface, according to an embodiment of the disclosure. As shown in fig. 14, a user may implement a network configuration operation for a model production line by triggering controls such as a drop-down box, an edit box, and a multi-box corresponding to each item. It should be understood that the network configuration interface shown in fig. 14 is a possible implementation manner, and those skilled in the art may set the configurable content in the network configuration interface, the layout, the style, and the like of the network configuration interface according to actual requirements, and the embodiment of the present disclosure is not limited thereto.

In a possible implementation manner, any known pre-training manner may be adopted to pre-train the at least one configured deep learning model to obtain a pre-trained deep learning model. It should be understood that the training set used in the pre-training may be related to the target task, the application scenario, etc. of the model production line, for example, the training sets corresponding to the human body detection task and the vehicle detection task are different, and the deep learning model obtained by training may also be different.

The pre-trained deep learning model corresponds to the network type to be set in the setting module of the user development platform, so that a user can determine the corresponding target deep learning model by setting the network type. The target deep learning model set on the user development platform is a pre-trained deep learning model. By the method, the deep learning model with certain precision can be obtained, so that the training efficiency of the training target deep learning model is improved, and the target deep learning model to be deployed is generated efficiently.

It should be understood that multiple processing tasks, such as vehicle detection, license plate recognition, etc., may be included in the same model production line. Then, at least one corresponding deep learning model can be configured and pre-trained respectively for a plurality of processing tasks in the same model production line.

In the embodiment of the disclosure, at least one deep learning model can be pre-established for a model production line, and the at least one deep learning model is pre-trained to obtain the deep learning model for a user to select, so that the user can obtain a target deep learning model capable of realizing a target task through simple and convenient setting operation, namely, selecting a network type, and the generation efficiency of the target deep learning model is improved.

In one possible implementation manner, in response to a building operation for a model production line, obtaining the model production line for generating a target deep learning model to be deployed further includes:

responding to information editing operation aiming at the model production line, obtaining production line information of the pre-built or built model production line, wherein the production line information is used for identifying the pre-built or built model production line in a production line shop platform and a user development platform;

responding to training configuration operation aiming at a model production line, and obtaining a training mode to be set in a pre-built or built model production line, wherein the training mode to be set is used for setting in a setting module of a user development platform;

responding to standard configuration operation aiming at the model production line, and obtaining a preset data acquisition standard and a preset data marking standard of a pre-built or built model production line, wherein the preset data acquisition standard is used for being displayed in an interface of a lead-in module of a user development platform, and the preset data marking standard is used for being displayed in an interface of a marking module of the user development platform;

and responding to index configuration operation aiming at the model production line to obtain a network evaluation index to be set in the pre-built or built model production line, wherein the network evaluation index to be set is used for setting an evaluation module of the user development platform.

Fig. 15 shows a schematic diagram of an information editing interface according to an embodiment of the present disclosure. As shown in fig. 15, the information editing interface can display prompt information for information editing, and information such as a task name, a processing object, an application scene, and data acquisition equipment can be input through the edit box, that is, information editing operation for a model production line is realized. For example, for a vehicle detection project, the project information may include "task name: vehicle detection, processing object: vehicle, application scenario: street, data acquisition equipment: a camera. It should be understood that the information entered by the user in the edit box, i.e., the resulting production line information of the model production line, may be determined by clicking on the "confirm" button shown in fig. 15.

FIG. 16 illustrates a schematic diagram of a training configuration interface, according to an embodiment of the present disclosure. As shown in fig. 16, the training end indicator may be configured through a multi-selection box, and it should be understood that the training end indicator configured here is also a training end indicator in a setting module of the user platform for setting by the user; the training equipment can also be configured through a multi-selection box, and the configured training equipment is also the training equipment which is arranged in a setting module of the user platform and is used for the user; by the method, a plurality of training end indexes and a plurality of training devices can be configured, so that a user can select any one training end index according to requirements and the training device used for training the target deep learning model.

In one possible implementation, as shown in fig. 16, an edit box for editing performance prediction information for the training device may be further provided in the training configuration interface, where the performance prediction information may be, for example, a running time length required for the training device to perform the target deep learning model training. The edited performance evaluation information of the training equipment can be displayed in a setting interface of the first setting item, so that a user can select the required training equipment to execute the training of the target deep learning model based on the displayed running time.

It should be appreciated that the training configuration operations for the model production line may include multiple selection operations for training end indicators, multiple selection operations for training equipment, and editing operations for performance prediction information. The user may determine the selected training end indicator and training equipment, and edit the performance estimation information by clicking the "confirm" button shown in fig. 16, which is not limited in this embodiment of the disclosure.

In a possible implementation manner, the preset data acquisition standard and the preset data marking standard can be respectively input through an edit box on the display interface, namely, the standard editing operation aiming at the model production line is realized; and the configuration of the preset data acquisition standard and the preset data marking standard can be realized by calling the relevant interface through the background. The configured preset data acquisition standard and the preset data marking standard can be respectively displayed in a setting interface of a third setting item for importing the data set and a setting interface of a fourth setting item for marking the data set.

Fig. 17 is a schematic diagram illustrating a standard configuration interface according to an embodiment of the disclosure, and as shown in fig. 17, the preset data acquisition standard and the preset data tagging standard may be input through an edit box, or the data acquisition standard and the data tagging standard may be configured through a drop-down box. The user may determine the edited preset data acquisition standard and the preset data labeling standard by clicking the "confirm" button shown in fig. 17, which is not limited in the embodiment of the present disclosure.

In a possible implementation manner, the index configuration operation on the network evaluation index may be implemented in the form of a multi-box in the index configuration interface, where the index configuration interface may refer to the setting interface of the fifth setting item shown in fig. 9. It should be understood that the network evaluation index selected in the index configuration interface, that is, the network evaluation index shown in the setting interface of the fifth setting item, and the network evaluation index to be set in the setting interface of the fifth setting item may be a part or all of the network evaluation index shown in the index configuration interface, and the embodiment of the present disclosure is not limited thereto.

It should be noted that the network configuration, the information editing, the training configuration, the standard configuration, and the index configuration may be displayed in the same display interface or in different display interfaces, that is, the network configuration interface, the information editing interface, the training configuration interface, the standard configuration interface, and the index configuration interface may be the same interface or different interfaces, which is not limited to the embodiment of the present disclosure. The building interface of the model production line comprises: the system comprises a network configuration interface, an information editing interface, a training configuration interface, a standard editing interface and an index configuration interface.

In one possible implementation mode, after configuration such as deep learning models, production line information, training modes, preset data acquisition standards, preset data marking standards, network evaluation indexes and the like, all configuration results can be connected in series through an existing component series connection tool, and therefore a model production line is built.

In the embodiment of the disclosure, professional technicians can configure the contents of each setting item in the model production line to build the model production line suitable for different scene requirements and different task requirements, so that a common user can efficiently obtain a target deep learning model to be deployed, which meets the actual requirements, through simple and convenient flow setting operation based on the built model production line.

In the embodiment of the present disclosure, at least one model production line and a recommended configuration included therein can be established, where the recommended configuration may include at least a deep learning model to pre-training, a data acquisition standard, a data labeling standard, a deep learning model structure, a hyper-parameter, an evaluation index, a training configuration, an inference configuration, and the like. Therefore, the user can conveniently perform the streamlined setting operation based on the built model production line and the recommended configuration contained in the model production line, so as to efficiently generate the target deep learning model to be deployed.

It can be known that, in the generation process of the deep learning model, professional technicians are required to have professional knowledge and experience, such as professional knowledge for a data acquisition mode, deep learning model selection, hyper-parameters, a pre-training model, a data enhancement mode, iteration times, logic of an iteration mode, and the like. Such knowledge and experience vary with the application scenario and processing task of the deep learning model. Therefore, in a practical scenario, the deep learning model generation process usually needs a professional technician to perform customization. In addition, the migration and adaptability of the deep learning model are weak, for example, a vehicle identification network with very good training precision in one city is poor, and in another city, the performance is poor due to the change of vehicle type distribution.

According to the embodiment of the disclosure, a configuration environment for a model production line of a professional technician and a setting environment for a general user to generate a required deep learning model using the model production line can be provided. Thus, the model production lines of the deep learning model can be defined and generated by the professional technicians, and each model production line can support the generation of the deep learning model of at least one application scene. The model production line built by the professional technical personnel can be selected and used by a common user, the generation process of the deep learning model is completed by using the setting environment of the model production line, and the deep learning model which can be deployed on equipment is obtained. By the method, the efficiency and the precision of the deep learning model generated by the common user based on the model production line can be close to the level of professional technicians.

In the related technology, the generation process of the deep learning model cannot meet the precision requirement of deep learning model training in various scenes; and the migration capability of the deep learning model in the deep learning field is not strong enough, so that a user needs to acquire data again to train and verify the deep learning model in different application scenes. And the deep learning model generated by the same generation process in the related technology cannot automatically adapt to the generation requirement of the deep learning model under various scenes. According to the embodiment of the disclosure, the model production line is customized under different application scenes by defining the application scenes of the model production line, and the automatic deep learning model generation under various application scenes can be supported.

In the related art, the deep learning model is generated in a single process, so that the accuracy of the provided trained deep learning model in a similar scene is low. According to the embodiment of the disclosure, different setting operations are performed on the same model production line, so that the generated deep learning model is suitable for similar application scenes, and a higher-precision deep learning model is obtained through an optimized setting process and training configuration. For example, in a model production line in a vehicle detection scene, different training sets can be adopted to train the same target deep learning model, and different target deep learning models to be deployed can be efficiently obtained, so that the method can be suitable for similar vehicle detection scenes of different cities and different streets.

According to the embodiment of the disclosure, model production lines of different application scenes can be established by professional technicians, and a common user can provide a generation flow and related configuration of the configuration by using professional technology, so that the deep learning model generation requirements of various different application scenes and similar application scenes can be met, and a deep learning model which is close to the professional technician level, high in precision and performance and suitable for the scene requirements can be obtained.

It is understood that the above-mentioned system embodiments mentioned in the present disclosure can be combined with each other to form a combined embodiment without departing from the principle logic, which is limited by the space, and the detailed description of the present disclosure is omitted. Those skilled in the art will appreciate that in the above described system of the specific embodiment, the specific order of execution of the steps should be determined by their function and possibly their inherent logic.

Embodiments of the present disclosure also provide a computer-readable storage medium having stored thereon computer program instructions, which when executed by a processor, implement the above-described system. The computer readable storage medium may be a non-volatile computer readable storage medium.

An embodiment of the present disclosure further provides an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke the memory-stored instructions to perform the method steps in the system described above.

The disclosed embodiments also provide a computer program product comprising computer readable code that when run on a device, a processor in the device executes instructions for implementing a deep learning model production system as provided in any of the above embodiments.

The disclosed embodiments also provide another computer program product for storing computer readable instructions, which when executed, cause a computer to perform the operations of the deep learning model production system provided in any of the above embodiments.

The electronic device may be provided as a terminal, server, or other form of device.

Fig. 18 shows a block diagram of an electronic device 800 according to an embodiment of the disclosure. For example, the electronic device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, or the like terminal.

Referring to fig. 18, electronic device 800 may include one or more of the following components: processing component 802, memory 804, power component 806, multimedia component 808, audio component 810, input/output (I/O) interface 812, sensor component 814, and communication component 816.

The processing component 802 generally controls overall operation of the electronic device 800, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing components 802 may include one or more processors 820 to execute instructions to perform all or a portion of the steps of the system described above. Further, the processing component 802 can include one or more modules that facilitate interaction between the processing component 802 and other components. For example, the processing component 802 can include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.

The memory 804 is configured to store various types of data to support operations at the electronic device 800. Examples of such data include instructions for any application or system operating on the electronic device 800, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 804 may be implemented by any type or combination of volatile or non-volatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.

The power supply component 806 provides power to the various components of the electronic device 800. The power components 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 800.

The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and a user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front facing camera and/or a rear facing camera. The front camera and/or the rear camera may receive external multimedia data when the electronic device 800 is in an operation mode, such as a shooting mode or a video mode. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

The audio component 810 is configured to output and/or input audio signals. For example, the audio component 810 includes a Microphone (MIC) configured to receive external audio signals when the electronic device 800 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may further be stored in the memory 804 or transmitted via the communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

The I/O interface 812 provides an interface between the processing component 802 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.

The sensor assembly 814 includes one or more sensors for providing various aspects of state assessment for the electronic device 800. For example, the sensor assembly 814 may detect an open/closed state of the electronic device 800, the relative positioning of components, such as a display and keypad of the electronic device 800, the sensor assembly 814 may also detect a change in the position of the electronic device 800 or a component of the electronic device 800, the presence or absence of user contact with the electronic device 800, orientation or acceleration/deceleration of the electronic device 800, and a change in the temperature of the electronic device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include a light sensor, such as a Complementary Metal Oxide Semiconductor (CMOS) or Charge Coupled Device (CCD) image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 may access a wireless network based on a communication standard, such as a wireless network (WiFi), a second generation mobile communication technology (2G) or a third generation mobile communication technology (3G), or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

In an exemplary embodiment, the electronic device 800 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic components for performing the method steps in the above-described systems.

In an exemplary embodiment, a non-transitory computer readable storage medium, such as the memory 804, is also provided that includes computer program instructions executable by the processor 820 of the electronic device 800 to perform the method steps in the system described above.

Fig. 19 shows a block diagram of an electronic device 1900 in accordance with an embodiment of the disclosure. For example, the electronic device 1900 may be provided as a server. Referring to FIG. 19, electronic device 1900 includes a processing component 1922 further including one or more processors and memory resources, represented by memory 1932, for storing instructions, e.g., applications, executable by processing component 1922. The application programs stored in memory 1932 may include one or more modules that each correspond to a set of instructions. Further, the processing component 1922 is configured to execute instructions to perform method steps in the system described above.

The electronic device 1900 may also include a power component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input/output (I/O) interface 1958. The electronic device 1900 may operate an operating system based on storage in memory 1932, such as the Microsoft Windows operating System (Win)dows Server^TM) Apple Inc. of the present application based on the graphic user interface operating System (Mac OS X)^TM) Multi-user, multi-process computer operating system (Unix)^TM) Free and open native code Unix-like operating System (Linux)^TM) Open native code Unix-like operating System (FreeBSD)^TM) Or the like.

In an exemplary embodiment, a non-transitory computer readable storage medium, such as the memory 1932, is also provided that includes computer program instructions executable by the processing component 1922 of the electronic device 1900 to perform the method steps of the system described above.

The present disclosure may be a system and/or a computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions embodied thereon for causing a processor to implement various aspects of the present disclosure.

The computer readable storage medium may be a tangible device that can hold and store the instructions for use by the instruction execution device. The computer readable storage medium may be, for example, but is not limited to, an electronic memory device, a magnetic memory device, an optical memory device, an electromagnetic memory device, a semiconductor memory device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a Static Random Access Memory (SRAM), a portable compact disc read-only memory (CD-ROM), a Digital Versatile Disc (DVD), a memory stick, a floppy disk, a mechanical coding device, such as punch cards or in-groove projection structures having instructions stored thereon, and any suitable combination of the foregoing. Computer-readable storage media as used herein is not to be construed as transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., optical pulses through a fiber optic cable), or electrical signals transmitted through electrical wires.

The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to a respective computing/processing device, or to an external computer or external storage device via a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. The network adapter card or network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in the respective computing/processing device.

The computer program instructions for carrying out operations of the present disclosure may be assembler instructions, Instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider). In some embodiments, the electronic circuitry that can execute the computer-readable program instructions implements aspects of the present disclosure by utilizing the state information of the computer-readable program instructions to personalize the electronic circuitry, such as a programmable logic circuit, a Field Programmable Gate Array (FPGA), or a Programmable Logic Array (PLA).

Various aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of systems and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.

These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer-readable medium storing the instructions comprises an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The computer program product may be embodied in hardware, software or a combination thereof. In an alternative embodiment, the computer program product is embodied in a computer storage medium, and in another alternative embodiment, the computer program product is embodied in a Software product, such as a Software Development Kit (SDK), or the like.

Having described embodiments of the present disclosure, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen in order to best explain the principles of the embodiments, the practical application, or improvements made to the technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A deep learning model production system comprising a user development platform, the user development platform comprising:

the production line selection module is used for responding to selection operation of a model production line in a production line list and displaying a setting list of the selected model production line, wherein the selected model production line is used for generating a target deep learning model to be deployed, the setting list comprises a first setting item of the selected model production line, the first setting item is used for setting the target deep learning model, a training mode of the target deep learning model and a data set used for training the target deep learning model, and the production line list is used for displaying production line information of the selected model production line;

the setting module is used for responding to setting operation aiming at a first setting item in the setting list, and determining a target deep learning model for realizing a target task, a training mode of the target deep learning model and a data set for training the target deep learning model;

and the training module is used for responding to the training triggering operation aiming at the target deep learning model, training the target deep learning model according to the data set and the training mode, and obtaining the target deep learning model to be deployed.

2. The system according to claim 1, wherein the setting operation of the first setting item includes: data set setting operation, network type setting operation and training mode setting operation;

the determining, in response to a setting operation for a first setting item in the setting list, a target deep learning model for implementing the target task, a training mode of the target deep learning model, and a data set for training the target deep learning model includes:

determining a target deep learning model for achieving the target task in response to a network type setting operation for the target deep learning model;

determining a dataset for training the target deep learning model in response to a dataset setting operation for the dataset of the target task;

and determining the training mode of the target deep learning model in response to the training mode setting operation aiming at the target deep learning model.

3. The system according to claim 1 or 2, wherein the training mode includes a training device for performing training, a training end index, and a specified size of sample data;

the training the target deep learning model according to the data set and the training mode to obtain the target deep learning model to be deployed, including:

adjusting the size of the sample data in the data set according to the designated size to obtain an adjusted data set;

4. The system according to any one of claims 1 to 3, wherein the setting list further includes a second setting item, the second setting item is used to set package parameters of the target deep learning model to be deployed, the package parameters include a package mode and/or a deployment device, and the user development platform further includes:

a deployment module, configured to package the target deep learning model to be deployed in response to a setting operation for a second setting item in the setting list, to obtain a package file of the target deep learning model to be deployed,

wherein the setting operation of the second setting item includes: setting operation on an encapsulation mode and/or deployment equipment, wherein the encapsulation file is used for deploying the target deep learning model to be deployed in the deployment equipment.

5. The system according to any one of claims 1 to 4, wherein the setting list further includes a third setting item for importing a data set, a fourth setting item for annotating the data set, and a fifth setting item for evaluating a target deep learning model to be deployed, and the user development platform further includes:

an import module, configured to obtain an original data set of the target task in response to an import operation for a third setting item in the setting list, where sample data in the original data set is data that meets a preset data acquisition standard, and the preset data acquisition standard is used to indicate acquisition of the sample data in the original data set;

the marking module is used for responding to marking operation aiming at a fourth setting item in the setting list, marking the sample data in the original data set according to a preset data marking standard, and obtaining the data set used for training the target deep learning model;

and the evaluation module is used for responding to the setting operation aiming at a fifth setting item in the setting list, and displaying the performance evaluation result of the target deep learning model to be deployed according to a set network evaluation index, wherein the network evaluation index is used for performing performance evaluation on the deep learning model to be deployed.

6. The system of claim 1, wherein the user development platform further comprises a line purchase item for entering a line store platform in response to a click operation for the line purchase item.

7. The system of claim 6, further comprising the line shop platform, wherein the information is transmitted between the line shop platform and the user development platform through a communication interface, and wherein the line shop platform is used for selling model production lines.

8. The system according to claim 6 or 7, wherein in the case of successful purchase of a model production line by the production line shop platform, production line information of the successfully purchased model production line is added in the production line list.

9. The system of claim 1, further comprising an expert development platform, wherein the expert development platform communicates information with the production line store platform via a communication interface, the expert development platform comprising:

the system comprises a production line building module, a model production line setting module and a model generation module, wherein the production line building module is used for responding to building operation aiming at a model production line to obtain a pre-built model production line, and the pre-built model production line is used for generating a target deep learning model to be deployed;

and the production line publishing module is used for responding to the publishing operation aiming at the pre-built model production line, publishing the pre-built model production line to the production line shop platform so as to display and purchase the pre-built model production line on the production line shop platform.

10. The system of claim 7, wherein the construction operation comprises a network configuration operation for a model production line;

the obtaining of the pre-built model production line in response to the building operation for the model production line comprises:

responding to the network configuration operation aiming at the model production line, and obtaining at least one deep learning model, wherein the network configuration operation comprises an operation of configuring at least one of a network structure, a network algorithm and an algorithm parameter;

and pre-training the at least one deep learning model to obtain a pre-trained deep learning model in the pre-built model production line, wherein the pre-trained deep learning model corresponds to the network type to be set in a setting module of the user development platform.

11. The system of claim 7 or 8, wherein the building operation further comprises: information editing operation, training configuration operation, standard editing operation and index configuration operation aiming at the model production line;

the method for responding to the building operation of the model production line to obtain the pre-built model production line further comprises the following steps:

responding to the information editing operation aiming at the model production line, and obtaining production line information of the pre-built model production line, wherein the production line information is used for identifying the pre-built model production line in the production line shop platform and the user development platform;

responding to the training configuration operation aiming at the model production line, and obtaining a training mode to be set in the pre-built model production line, wherein the training mode to be set is used for setting in a setting module of the user development platform;

responding to the standard configuration operation aiming at the model production line, and obtaining a preset data acquisition standard and a preset data marking standard of the preset model production line, wherein the preset data acquisition standard is used for being displayed in an interface of a import module of the user development platform, and the preset data marking standard is used for being displayed in an interface of a marking module of the user development platform;

and responding to the index configuration operation aiming at the model production line to obtain a network evaluation index to be set in the pre-built model production line, wherein the network evaluation index to be set is used for setting an evaluation module of the user development platform.

12. The system of claim 1, wherein the target task comprises an image processing task comprising: at least one of image recognition, image segmentation, image classification, and keypoint detection.

13. An electronic device, comprising:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to invoke the memory-stored instructions to execute the system of any one of claims 1 to 12.

14. A computer readable storage medium having computer program instructions stored thereon which, when executed by a processor, implement the system of any one of claims 1 to 12.