CN112862105A

CN112862105A - Data publishing system, method and device

Info

Publication number: CN112862105A
Application number: CN201911191747.9A
Authority: CN
Inventors: 马良
Original assignee: Hangzhou Hikvision Digital Technology Co Ltd
Current assignee: Hangzhou Zhongwang Technology Co ltd
Priority date: 2019-11-28
Filing date: 2019-11-28
Publication date: 2021-05-28
Anticipated expiration: 2039-11-28
Also published as: CN112862105B

Abstract

Embodiments of the present application provide a data publishing system, method, and device. The data publishing system includes a first data terminal, a service platform, and a second data terminal. The service platform receives an evaluation request sent by the first data terminal, and utilizes the data for the evaluation request. The pre-established deep learning model set and verification data set in the business field in the database, evaluate the data to be evaluated, obtain the evaluation result of the data to be evaluated, and feed back the evaluation result to the first data terminal. After receiving the evaluation result, the first data terminal, According to the evaluation result, determine whether to release the data to be evaluated, and send the determination result to the service platform. If the determination result sent by the first data terminal received by the service platform is yes, the data to be evaluated is released, so that the second data terminal can choose to download the data to be evaluated. Evaluate data. Through this solution, the accelerated implementation of the deep learning model is realized.

Description

Data publishing system, method and device

Technical Field

The present application relates to the field of computer technologies, and in particular, to a data distribution system, method, and apparatus.

Background

The deep learning technology is a new technology in machine learning, analyzes data by simulating a mechanism of a human brain, and is an artificial intelligence technology for analyzing and learning by establishing and simulating the human brain. By establishing the deep learning model, the data to be processed is input into the deep learning model, and the intelligent processing of the data to be processed is rapidly realized through the end-to-end calculation process of the deep learning model, so that the deep learning technology is widely applied to the fields of city safety monitoring, traffic supervision and the like through the end-to-end rapid processing process.

The application of the deep learning technology is closely related to the accuracy of a deep learning model, the deep learning model is obtained through training, in the training process of the deep learning model, massive training samples and an initial deep learning model are obtained, the massive training samples are sequentially input into the initial deep learning model, model parameters are continuously adjusted according to the difference between the output result of the model and a nominal value, and the output result of the deep learning model is closer to the nominal value more and more through continuously adjusting the model parameters until the deep learning model training is finished when convergence occurs.

However, the initial deep learning model generally selects a general model, and it is difficult to ensure good performance in a specific business field, and a large amount of training samples also include some training samples with low values, and a long training process is required to make the deep learning model converge, so that the landing speed of the deep learning model is slow when a user uses the deep learning model.

Disclosure of Invention

The embodiment of the application aims to provide a data publishing system, method and device so as to realize accelerated landing of a deep learning model. The specific technical scheme is as follows:

in a first aspect, an embodiment of the present application provides a data publishing system, where the system includes a first data end, a service platform, and a second data end:

the first data terminal is used for sending an evaluation request to the service platform when the data to be evaluated has a requirement for evaluation, wherein the evaluation request comprises the data to be evaluated and a service field to which the data to be evaluated belongs, and the data to be evaluated comprises a deep learning model to be evaluated and/or a sample data set to be evaluated;

the service platform is used for receiving an evaluation request sent by the first data terminal; evaluating the data to be evaluated by utilizing a deep learning model set and a verification data set which are pre-established aiming at the business field to obtain an evaluation result of the data to be evaluated; feeding back an evaluation result to the first data terminal;

the first data terminal is also used for judging whether to issue the data to be evaluated or not according to the evaluation result after receiving the evaluation result fed back by the service platform, and sending the judgment result to the service platform;

the service platform is also used for receiving the judgment result sent by the first data terminal; if the judgment result is yes, the data to be evaluated is issued;

and the second data terminal is used for selecting and downloading the data to be evaluated, which is issued by the service platform.

In a second aspect, an embodiment of the present application provides a data publishing method, which is applied to a service platform, and the method includes:

receiving an evaluation request sent by a first data terminal, wherein the evaluation request comprises data to be evaluated and a service field to which the data to be evaluated belongs, and the data to be evaluated comprises a deep learning model to be evaluated and/or a sample data set to be evaluated;

evaluating the data to be evaluated by utilizing a deep learning model set and a verification data set which are pre-established aiming at the business field to obtain an evaluation result of the data to be evaluated;

feeding back an evaluation result to the first data end so that the first data end judges whether to issue the data to be evaluated according to the evaluation result and sends a judgment result to the service platform;

receiving a judgment result sent by a first data end;

if the judgment result is yes, the data to be evaluated is issued, so that the second data terminal can download the data to be evaluated selectively.

Optionally, the step of evaluating the data to be evaluated by using a deep learning model set and a verification data set pre-established for the business field for the deep learning model to be evaluated to obtain an evaluation result of the data to be evaluated includes:

determining a deep learning model set and a verification data set which are pre-established aiming at the business field according to the field information of the business field;

respectively inputting the data in the verification data set into the deep learning model to be evaluated and each deep learning model in the deep learning model set to obtain an output result of the deep learning model to be evaluated and an output result of each deep learning model;

and comparing the output result of the deep learning model to be evaluated with the output result of each deep learning model by taking the output result of each deep learning model as a reference to obtain the difference between the output result of the deep learning model to be evaluated and the output result of each deep learning model, and determining the evaluation result of the deep learning model to be evaluated according to the difference.

Optionally, the step of evaluating the data to be evaluated by using a deep learning model set and a verification data set pre-established for the business field for the sample data set to be evaluated to obtain an evaluation result of the data to be evaluated includes:

inputting each sample data in a sample data set to be evaluated into each deep learning model in a deep learning model set established in advance to obtain a first output result of each deep learning model;

inputting the data in the verification data set into each deep learning model in the pre-established deep learning model set to obtain a second output result of each deep learning model;

and comparing the first output result with the second output result by taking the second output result as a reference to obtain the difference between the first output result and the second output result, and determining the evaluation result of the sample data set to be evaluated according to the difference.

Optionally, after the step of issuing the data to be evaluated and the recommendation information of the data to be evaluated, the method further includes:

if selection information for selecting the deep learning model to be evaluated, which is sent by a second data end, is received, the deep learning model to be evaluated is sent to the second data end, or the deep learning model after the deep learning model to be evaluated is trained by using a verification data set is sent to the second data end;

if the selection information for selecting the deep learning model to be evaluated and the data to be processed, which are sent by the second data terminal, are received, the data to be processed are input into the deep learning model after the deep learning model to be evaluated is trained by using the verification data set, a processing result is obtained, and the processing result is sent to the second data terminal;

if receiving selection information for selecting the sample data set to be evaluated, which is sent by a second data terminal, sending the sample data set to be evaluated to the second data terminal;

and if the selection information for selecting the deep learning model to be evaluated and the sample data set to be evaluated, which is sent by the second data terminal, is received, the deep learning model to be evaluated is trained by using the sample data set to be evaluated, and the trained deep learning model and the sample data set to be evaluated are sent to the second data terminal.

Optionally, the step of sending the deep learning model to be evaluated to the second data end includes:

encrypting the deep learning model to be evaluated, and sending the encrypted deep learning model to be evaluated and encrypted identification information to the second data terminal;

the step of sending the deep learning model after training the deep learning model to be evaluated by using the verification data set to the second data terminal comprises the following steps:

encrypting the deep learning model after the deep learning model to be evaluated is trained by using the verification data set, and sending the encrypted deep learning model after the deep learning model to be evaluated and encrypted identification information to the second data terminal;

the step of sending the processing result to the second data terminal includes:

encrypting the processing result and sending the encrypted processing result and the encrypted identification information to the second data terminal;

the step of sending the sample data set to be evaluated to the second data terminal comprises the following steps:

encrypting the sample data set to be evaluated, and sending the encrypted sample data set to be evaluated and the encrypted identification information to the second data terminal;

the step of sending the trained deep learning model and the sample data set to be evaluated to the second data terminal includes:

and encrypting the trained deep learning model and the sample data set to be evaluated, and sending the encrypted trained deep learning model, the sample data set to be evaluated and the encrypted identification information to the second data terminal.

Optionally, after the step of receiving the evaluation request sent by the first data end, the method further includes:

if the received sample data set to be evaluated comprises sample data without a corresponding annotation file, sending annotation prompt information to the first data end, so that the first data end selects an annotation type to send to the service platform after receiving the annotation prompt information;

receiving a mark type sent by a first data end;

selecting a preset deep learning model from a deep learning model set pre-established aiming at the business field according to the field information of the business field;

according to the labeling type, labeling the sample data by using a preset deep learning model to obtain an initialization labeling result;

feeding back an initialization marking result to the first data end so that the first data end sets marking task information and sends the marking task information to the service platform according to the initialization marking result;

receiving the labeling task information sent by the first data end, and publishing the sample data and the labeling task information so that the labeling end judges whether to label the sample data or not based on the labeling task information;

and if a marking file for marking the sample data by the marking end is received, recording the corresponding marking file to the sample data set to be evaluated.

In a third aspect, an embodiment of the present application provides a data publishing device, which is applied to a service platform, and the device includes: the evaluation service module and the data release service module;

the evaluation service module is used for receiving an evaluation request sent by a first data terminal, wherein the evaluation request comprises data to be evaluated and a service field to which the data to be evaluated belongs, and the data to be evaluated comprises a deep learning model to be evaluated and/or a sample data set to be evaluated; evaluating the data to be evaluated by utilizing a deep learning model set and a verification data set which are pre-established aiming at the business field to obtain an evaluation result of the data to be evaluated; feeding back an evaluation result to the first data end so that the first data end judges whether to issue the data to be evaluated according to the evaluation result and sends a judgment result to the service platform;

the data release service module is used for receiving the judgment result sent by the first data terminal; if the judgment result is yes, the data to be evaluated is issued, so that the second data terminal can download the data to be evaluated selectively.

Optionally, the evaluation service module is specifically configured to, for the deep learning model to be evaluated:

Optionally, the evaluation service module is specifically configured to, for a sample data set to be evaluated:

Optionally, the apparatus further comprises: an intelligent service module;

the intelligent service module is used for sending the deep learning model to be evaluated to the second data terminal or sending the deep learning model after the deep learning model to be evaluated is trained by using the verification data set to the second data terminal if the selection information for selecting the deep learning model to be evaluated, which is sent by the second data terminal, is received; and if the selection information for selecting the deep learning model to be evaluated and the data to be processed, which are sent by the second data terminal, are received, the data to be processed are input into the deep learning model after the deep learning model to be evaluated is trained by using the verification data set, a processing result is obtained, and the processing result is sent to the second data terminal.

Optionally, the apparatus further comprises: a crowdsourcing annotation service module;

the crowdsourcing annotation service module is used for sending annotation prompt information to the first data end if the received sample data set to be evaluated comprises sample data without corresponding annotation files, so that the first data end selects an annotation type to send to the service platform after receiving the annotation prompt information; receiving a mark type sent by a first data end; selecting a preset deep learning model from a deep learning model set pre-established aiming at the business field according to the field information of the business field; according to the labeling type, labeling the sample data by using a preset deep learning model to obtain an initialization labeling result; feeding back an initialization marking result to the first data end so that the first data end sets marking task information and sends the marking task information to the service platform according to the initialization marking result; receiving the labeling task information sent by the first data end, and publishing the sample data and the labeling task information so that the labeling end judges whether to label the sample data or not based on the labeling task information; and if a marking file for marking the sample data by the marking end is received, recording the corresponding marking file to the sample data set to be evaluated.

In a fourth aspect, embodiments of the present application provide a service platform comprising a processor and a machine-readable storage medium storing machine-executable instructions executable by the processor, the processor being caused by the machine-executable instructions to: the method provided by the second aspect of the embodiments of the present application is implemented.

In a fifth aspect, embodiments of the present application provide a machine-readable storage medium storing machine-executable instructions, which when invoked and executed by a processor, implement the method provided by the second aspect of the embodiments of the present application.

The data release system comprises a first data end, a service platform and a second data end, wherein the service platform receives an evaluation request sent by the first data end, evaluates data to be evaluated by utilizing a deep learning model set and a verification data set which are pre-established aiming at a service field in the evaluation request to obtain an evaluation result of the data to be evaluated, feeds back the evaluation result to the first data end, judges whether the data to be evaluated is released or not according to the evaluation result after the first data end receives the evaluation result, and sends a judgment result to the service platform, if the judgment result sent by the first data end received by the service platform is positive, the data to be evaluated is released to be used for the second data end to select to download the data to be evaluated.

The service platform evaluates the deep learning model to be evaluated and/or the sample data set to be evaluated in the service field of the evaluation request, whether the deep learning model to be evaluated and/or the sample data set to be evaluated are issued or not can be judged based on the evaluation result, after the deep learning model to be evaluated and/or the sample data set to be evaluated are issued, the second data end can select the deep learning model to be evaluated and/or the sample data set to be evaluated according to the actual requirements of the second data end, the evaluation result of the deep learning model to be evaluated and/or the sample data set to be evaluated is considered when the deep learning model to be evaluated and/or the sample data set to be evaluated is issued, the deep learning model to be evaluated and/or the sample data set to be evaluated can be selected according to the actual requirements of the second data end when the second data end is selected, and the deep learning model to be evaluated and/or the sample data, when the deep learning model is used, the second data end does not need to undergo long-time iterative training, and therefore accelerated landing of the deep learning model is achieved.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

Fig. 1 is a schematic structural diagram of a data distribution system according to an embodiment of the present application;

fig. 2 is a schematic flowchart of a data distribution method according to an embodiment of the present application;

fig. 3 is a schematic diagram of an evaluation process of a deep learning model to be evaluated according to an embodiment of the present application;

fig. 4 is a schematic view of an evaluation flow for a sample data set to be evaluated according to an embodiment of the present application;

FIG. 5 is a schematic flow chart diagram illustrating a data distribution method according to another embodiment of the present application;

FIG. 6 is a schematic flow chart illustrating a data distribution method according to yet another embodiment of the present application;

fig. 7 is a schematic structural diagram of a data distribution apparatus according to an embodiment of the present application;

fig. 8 is a schematic structural diagram of a service platform according to an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

In order to realize accelerated landing of deep learning models in different business fields, it is not enough to adjust the structure of the deep learning model in a targeted manner by only collecting data inside. In order to promote the landing of the deep learning model, a shared service platform needs to be established, a data end can acquire a sample data set and the deep learning model of each business field through the service platform, and the service platform can be a server device or a background service system.

The service platform and the two data terminals form a data distribution system, as shown in fig. 1, the data distribution system includes a first data terminal 110, a service platform 120, and a second data terminal 130.

The first data terminal 110 is configured to send an evaluation request to the service platform 120 when there is a need for evaluating data to be evaluated, where the evaluation request includes the data to be evaluated and a service domain to which the data to be evaluated belongs, and the data to be evaluated includes a deep learning model to be evaluated and/or a sample data set to be evaluated;

the service platform 120 is configured to receive an evaluation request sent by the first data terminal 110; evaluating the data to be evaluated by utilizing a deep learning model set and a verification data set which are pre-established aiming at the business field to obtain an evaluation result of the data to be evaluated; feeding back the evaluation result to the first data terminal 110;

the first data end 110 is further configured to, after receiving the evaluation result fed back by the service platform 120, determine whether to issue the data to be evaluated according to the evaluation result, and send the determination result to the service platform 120;

the service platform 120 is further configured to receive a determination result sent by the first data end 110; if the judgment result is yes, the data to be evaluated is issued;

and the second data terminal 130 is configured to select to download the data to be evaluated, which is issued by the service platform 120.

Based on the data publishing system shown in fig. 1, an embodiment of the present application provides a data publishing method, which is applied to a service platform of the data publishing system shown in fig. 1, and as shown in fig. 2, the method may include the following steps.

S201, an evaluation request sent by a first data terminal is received, wherein the evaluation request comprises data to be evaluated and a service field to which the data to be evaluated belongs, and the data to be evaluated comprises a deep learning model to be evaluated and/or a sample data set to be evaluated.

The first data end is a general term of an object initiating an evaluation request, and may be a development end (including a developer and software and hardware used by the developer) of model development, or a user end (including a user and software and hardware used by the user) of model development. The evaluation request at least comprises a deep learning model to be evaluated and/or a sample data set to be evaluated, wherein the deep learning model to be evaluated and/or the sample data set to be evaluated need to be evaluated, in addition, when the first data terminal initiates the evaluation request, the business field (such as commodity identification of retail business) to which the data to be evaluated belongs can be specified, and evaluation strategies of the data to be evaluated are different under different business fields.

S202, evaluating the data to be evaluated by utilizing a deep learning model set and a verification data set which are pre-established aiming at the business field to obtain an evaluation result of the data to be evaluated.

The service platform has a self-built verification data set and a deep learning model set which are suitable for different business fields, and the main principle of evaluating the deep learning model to be evaluated and the sample data set to be evaluated is as follows: when the deep learning model to be evaluated is evaluated, the same verification data set is adopted to input the deep learning model to be evaluated and each deep learning model in the pre-established deep learning model set, the output result is compared, and the higher the accuracy of the output result of the deep learning model to be evaluated is, the better the performance is; when the sample data set to be evaluated is evaluated, the lower the accuracy of the deep learning model in the pre-established deep learning model set is, the sample data in the sample data set to be evaluated has specificity, and the higher the value of training the sample data set to be evaluated is.

S203, feeding back the evaluation result to the first data end so that the first data end judges whether to issue the data to be evaluated according to the evaluation result, and sending the judgment result to the service platform.

After the evaluation result of the data to be evaluated is obtained, the evaluation result is fed back to the first data end, the first data end receives the evaluation result, the value of the data to be evaluated can be known, and the higher the value of the data to be evaluated is, the higher the income which can be obtained when the data to be evaluated is issued is, therefore, the first data end can judge whether the data to be evaluated is issued or not according to the evaluation result and send the judgment result to the service platform, and the higher the evaluation result is, the higher the possibility that the data to be evaluated with the better evaluation result is selected to be issued is.

And S204, receiving the judgment result sent by the first data terminal.

And S205, if the judgment result is yes, issuing the data to be evaluated for the second data terminal to select to download the data to be evaluated.

The service platform receives the judgment result sent by the first data end, and can know whether to release the data to be evaluated, and the second data end can see the released data to be evaluated on the service platform and select whether to download the data to be evaluated. The first data end may also give recommendation information such as selling price, authorization term, development time and the like for issuing the data to be evaluated, or the service platform may give recommendation information such as selling price, authorization term, development time and the like for the data to be evaluated according to the evaluation result, and the recommendation information represents the use value of the data to be evaluated. When the data to be evaluated is released, the recommendation information is released at the same time, so that the second data end can obtain the use values of different data at the interface provided by the service platform, and the second data end can select the required deep learning model and/or the sample data set according to the recommendation information.

The service platform provides a transaction interface for the second data terminal, the second data terminal selects the deep learning model and/or the sample data set which need to be used through the transaction interface, and after the deep learning model and/or the sample data set are selected, the service platform submits the corresponding reward purchase deep learning model and/or the sample data set. The second data end is a general term of an object running the deep learning model, the deep learning model and/or the sample data set need to be purchased from the service platform, and the second data end may be the same as or different from the first data end.

By applying the embodiment of the application, the service platform receives an evaluation request sent by the first data end, evaluates the data to be evaluated by utilizing a deep learning model set and a verification data set which are pre-established aiming at the service field in the evaluation request to obtain an evaluation result of the data to be evaluated, feeds back the evaluation result to the first data end, judges whether to issue the data to be evaluated according to the evaluation result after the first data end receives the evaluation result, and sends a judgment result to the service platform, if the judgment result sent by the first data end received by the service platform is positive, the data to be evaluated is issued to be used for the second data end to select to download the data to be evaluated. The service platform evaluates the deep learning model to be evaluated and/or the sample data set to be evaluated in the service field of the evaluation request, whether the deep learning model to be evaluated and/or the sample data set to be evaluated are issued or not can be judged based on the evaluation result, after the deep learning model to be evaluated and/or the sample data set to be evaluated are issued, the second data end can select the deep learning model to be evaluated and/or the sample data set to be evaluated according to the actual requirements of the second data end, the evaluation result of the deep learning model to be evaluated and/or the sample data set to be evaluated is considered when the deep learning model to be evaluated and/or the sample data set to be evaluated is issued, the deep learning model to be evaluated and/or the sample data set to be evaluated can be selected according to the actual requirements of the second data end when the second data end is selected, and the deep learning model to be evaluated and/or the sample data, when the deep learning model is used, the second data end does not need to undergo long-time iterative training, and therefore accelerated landing of the deep learning model is achieved.

The service platform acquires the sample data set and the deep learning model of each business field, and the deep learning model and the sample data set which are excellent in performance can be rapidly pushed to the market. After the service platform is established, the processing of the sample data set can be issued to the first data end, so that the sample data set is processed by using the resources of the first data end, external small companies or small groups can be well promoted to participate in the artificial intelligence service in a specific business field by using the first data end, the resources (such as a processor, computing power and network bandwidth) of the first data end are converted into the processing result of the sample data set and are fed back to the service platform for evaluation, and the artificial intelligence service iteration is outsourced to the external small companies or small groups by taking the service platform as the center, so that the promotion of the whole business landing is facilitated.

The data to be evaluated comprises two types, namely a deep learning model to be evaluated and a sample data set to be evaluated, and the data evaluation method provided by the embodiment of the application is introduced below respectively aiming at the two types of data to be evaluated.

As shown in fig. 3, for the deep learning model to be evaluated, S102 is specifically implemented by the following steps:

s301, according to the domain information of the business domain, determining a deep learning model set and a verification data set which are pre-established for the business domain.

The service platform generates a verification data set in the business field according to the received field information of the business field, and can acquire a deep learning model set which is self-built in advance aiming at the business field.

S302, inputting the data in the verification data set into the deep learning model to be evaluated and each deep learning model in the deep learning model set respectively to obtain an output result of the deep learning model to be evaluated and an output result of each deep learning model.

And S303, comparing the output result of the deep learning model to be evaluated with the output result of each deep learning model by taking the output result of each deep learning model as a reference to obtain the difference between the output result of the deep learning model to be evaluated and the output result of each deep learning model, and determining the evaluation result of the deep learning model to be evaluated according to the difference.

And operating the deep learning model to be evaluated and each deep learning model in the deep learning model set, and comparing the performance of the deep learning model to be evaluated on the verification data set by taking the output result of each deep learning model as a reference. And providing an evaluation result about the value of the deep learning model to be evaluated according to the deep learning model to be evaluated and the performance of each deep learning model in the deep learning model set. After the evaluation result is obtained, the service platform can submit the evaluation result to a relevant technician for rechecking, and finally, the rechecked evaluation result is fed back to the first data terminal.

As shown in fig. 4, S202 is specifically implemented by the following steps for a sample data set to be evaluated:

s401, according to the domain information of the business domain, determining a deep learning model set and a verification data set which are pre-established for the business domain.

S402, inputting each sample data in the sample data set to be evaluated into each deep learning model in the pre-established deep learning model set to obtain a first output result of each deep learning model.

And S403, inputting the data in the verification data set into each deep learning model in the pre-established deep learning model set to obtain a second output result of each deep learning model.

S404, comparing the first output result with the second output result by taking the second output result as a reference to obtain the difference between the first output result and the second output result, and determining the evaluation result of the sample data set to be evaluated according to the difference.

And running each deep learning model in the deep learning model set on the sample data set to be evaluated, running each deep learning model in the deep learning model set on the verification data set, and giving an evaluation result according to the performance of each deep learning model on the sample data set and the performance of each deep learning model on the verification data set. After the evaluation result is obtained, the service platform can submit the evaluation result to a relevant technician for rechecking, and finally, the rechecked evaluation result is fed back to the first data terminal. The evaluation result of the sample data set to be evaluated can be expressed as the action value of the sample data set on model training.

Based on the embodiment shown in fig. 2, the embodiment of the present application further provides a data publishing method, as shown in fig. 5, the method includes the following steps.

S501, an evaluation request sent by a first data terminal is received, wherein the evaluation request comprises data to be evaluated and a service field to which the data to be evaluated belongs, and the data to be evaluated comprises a deep learning model to be evaluated and/or a sample data set to be evaluated.

S502, evaluating the data to be evaluated by utilizing a deep learning model set and a verification data set which are pre-established aiming at the business field to obtain an evaluation result of the data to be evaluated.

S503, feeding back the evaluation result to the first data end, so that the first data end judges whether to issue the data to be evaluated according to the evaluation result, and sending the judgment result to the service platform.

S504, the judgment result sent by the first data terminal is received.

And S505, if the judgment result is yes, issuing the data to be evaluated.

And S506, if the selection information for selecting the deep learning model to be evaluated, which is sent by the second data terminal, is received, the deep learning model to be evaluated is sent to the second data terminal, or the deep learning model after the deep learning model to be evaluated is trained by using the verification data set is sent to the second data terminal.

The service platform is also provided with intelligent services, for example, in the field of image recognition, the service platform can provide an algorithm library based on deep learning for target detection, image classification, behavior recognition and the like; in the field of voice recognition, a service platform can provide intelligent voice-to-text conversion service. Based on this, if the second data end selects the deep learning model to be evaluated for downloading, the service platform may directly send the deep learning model selected by the second data end to the second data end, or may train the deep learning model with a local verification data set, and send the trained deep learning model to the second data end, specifically, the timing for the service platform to train the model may be after the selection information sent by the second data end is received, or before the selection information sent by the second data end is received, where no specific limitation is made here. If the service platform sends the deep learning model to be evaluated, the service platform does not need to train the model, so that the resources of the service platform can be saved; and if the service platform sends the trained deep learning model, the second data end receives the trained deep learning model and can directly take the deep learning model for intelligent analysis, so that the resources of the second data end can be saved.

And S507, if the selection information for selecting the deep learning model to be evaluated and the data to be processed, which are sent by the second data terminal, are received, the data to be processed are input into the deep learning model after the deep learning model to be evaluated is trained by using the verification data set, a processing result is obtained, and the processing result is sent to the second data terminal.

If the second data terminal selects the deep learning model to be evaluated for downloading and provides the data to be processed, the service platform can input the data to be processed into the deep learning model trained on the deep learning model to be evaluated by using the verification data set to obtain a processing result and send the processing result to the second data terminal. The service platform provides a complete intelligent service for the second data end, the second data end only needs to input data to be processed, the processing result can be directly obtained from the service platform, and the second data end further improves the intelligence.

And S508, if the selection information for selecting the sample data set to be evaluated, which is sent by the second data terminal, is received, sending the sample data to be evaluated to the second data terminal.

If the second data terminal selects the sample data set to be evaluated for downloading, the service platform can directly send the sample data set selected by the second data terminal to the second data terminal.

And S509, if selection information for selecting the deep learning model to be evaluated and the sample data set to be evaluated, which is sent by the second data terminal, is received, training the deep learning model to be evaluated by using the sample data set to be evaluated, and sending the trained deep learning model and the sample data set to be evaluated to the second data terminal.

If the second data terminal selects to download the deep learning model to be evaluated and the sample data to be evaluated, the service platform can train the selected deep learning model by using the sample data set selected by the second data terminal, and send the trained deep learning model and the selected sample data set to the second data terminal, and the second data terminal receives the deep learning model which is trained and can be directly taken to perform intelligent analysis, so that the resources of the second data terminal can be saved.

Optionally, in the embodiment shown in fig. 5, the step of sending the deep learning model to be evaluated to the second data end may specifically be: and encrypting the deep learning model to be evaluated, and sending the encrypted deep learning model to be evaluated and the encrypted identification information to the second data terminal.

The step of sending the deep learning model after training the deep learning model to be evaluated by using the verification data set to the second data end may specifically be: and encrypting the deep learning model after the deep learning model to be evaluated is trained by using the verification data set, and sending the encrypted deep learning model after the deep learning model to be evaluated and the encrypted identification information to the second data terminal.

The step of sending the processing result to the second data end may specifically be: and encrypting the processing result and sending the encrypted processing result and the encrypted identification information to the second data terminal.

The step of sending the sample data set to be evaluated to the second data terminal may specifically be: and encrypting the sample data set to be evaluated, and sending the encrypted sample data set to be evaluated and the encrypted identification information to the second data terminal.

The step of sending the trained deep learning model and the sample data set to be evaluated to the second data terminal may specifically be: and encrypting the trained deep learning model and the sample data set to be evaluated, and sending the encrypted trained deep learning model, the sample data set to be evaluated and the encrypted identification information to the second data terminal.

When the service platform sends data such as a deep learning model to be evaluated, a trained deep learning model, a sample data set to be evaluated and the like to the second data terminal, in order to ensure the security of the data and prevent the data from being leaked and stolen, the data can be encrypted firstly, the encrypted data is sent to the second data terminal, and the encrypted identification information is synchronously sent to the second data terminal while the encrypted data is sent, wherein the encrypted identification information mainly comprises a verification tool (such as a dongle), a secret key and the like. In a specific implementation manner, the service platform may send a download address of the encrypted identification information, the data, and the like to the second data terminal, and the second data terminal downloads the data based on the download address, and inputs a key through the verification tool, decrypts the deep learning model, the sample data set, and the like, and then binds the sample data set to a specific training server for deployment, or binds the deep learning model to a specific terminal for deployment.

Based on the embodiment shown in fig. 2, the embodiment of the present application further provides a data publishing method, as shown in fig. 6, including the following steps.

S601, receiving an evaluation request sent by a first data terminal, wherein the evaluation request comprises data to be evaluated and a service field to which the data to be evaluated belongs, and the data to be evaluated comprises a deep learning model to be evaluated and/or a sample data set to be evaluated.

S602, if the received sample data set to be evaluated comprises sample data without a corresponding annotation file, sending annotation prompt information to the first data end, so that the first data end selects an annotation type to send to the service platform after receiving the annotation prompt information.

When a sample data set to be evaluated is evaluated, generally, sample data in the input sample data set to be evaluated must be labeled sample data, and if the sample data set to be evaluated includes sample data without a corresponding label file, crowdsourcing and labeling of the sample data is required. At this moment, the service platform sends labeling prompt information to the first data end to prompt that the first data end has sample data to be labeled, the first data end selects a labeling type after receiving the labeling prompt information, specifically, the first data end sets a labeling type according to the actual service field and the service requirement, and the labeling type is image classification labeling, target detection area labeling and the like. The service platform can also provide a specific marking tool, and the core of the marking tool is a deep learning model and a matched fine adjustment tool.

S603, receiving the mark type sent by the first data end.

S604, according to the domain information of the business domain, a preset deep learning model is selected from a deep learning model set which is pre-established aiming at the business domain.

And S605, marking the sample data by using a preset deep learning model according to the marking type to obtain an initialization marking result.

The first data terminal can select whether to adopt the existing deep learning model of the service platform to initialize the marking tool. For example, pedestrian detection is required, and the existing pedestrian detection model of the service platform is loaded by the marking tool, so that the initial marking result can be generated for the marked picture by the marking tool. And if the first data end selects not to adopt the existing deep learning model, the marking tool cannot generate an initialization marking result.

And S606, feeding back the initialization marking result to the first data end, so that the first data end sets marking task information according to the initialization marking result and sends the marking task information to the service platform.

And S607, receiving the labeling task information sent by the first data end, and publishing the sample data and the labeling task information, so that the labeling end judges whether to label the sample data based on the labeling task information.

After the initialization marking result is obtained, the initialization marking result is output to the first data end, and the first data end can set marking task information according to the initialization marking result, for example, whether the initialization marking result is satisfied is judged, generally speaking, the more accurate the initialization marking result is, the more satisfied the initialization marking result is. And if the mark is not satisfied, setting the mark difficulty to be high, if the mark is satisfied, setting the mark difficulty to be low, and setting corresponding mark remuneration according to the difference of the mark difficulty, wherein in general, the mark difficulty is higher, and the mark remuneration is more. After the annotation task information is obtained, the annotation task information and the sample data can be issued at the same time, and the annotation end can judge whether to accept the annotation task according to the annotation task information.

And S608, if a label file for labeling the sample data by the label end is received, recording the label file correspondingly to the sample data set to be evaluated.

And after the marking end receives the marking task, the service platform generates an online marking working environment, so that the marking end directly carries out online marking on the service platform, and meanwhile, the first data end initiating the marking task receives marking remuneration. After the labeling task is completed, the service platform records the labeling result in the sample data set, and the sample data set is perfected, so that all sample data in the sample data set have labeling files. Certainly, the service platform can also feed back the labeling result to the first data end, and after the first data end confirms that the labeling result meets the requirement, the service platform pays the remuneration to the staff at the labeling end, and records corresponding to the labeling file are collected to the sample data to be evaluated.

And S609, evaluating the data to be evaluated by utilizing a deep learning model set and a verification data set which are pre-established aiming at the business field to obtain an evaluation result of the data to be evaluated.

S610, feeding back an evaluation result to the first data end, so that the first data end judges whether to issue the data to be evaluated according to the evaluation result, and sending the judgment result to the service platform.

S611, receiving the determination result sent by the first data end.

And S612, if the judgment result is yes, issuing the data to be evaluated and recommendation information of the data to be evaluated, wherein the recommendation information represents the use value of the data to be evaluated.

In the embodiment of the application, the service platform further provides crowdsourcing annotation service, the annotation task can be issued, the annotation task information can be marked according to the corresponding setting of the initial annotation result of the service platform, the annotation end can visually see the annotation task information and decide whether to perform annotation work, the annotation work for sample data brings a basis, and the efficiency of sample data annotation can be improved.

Corresponding to the foregoing method embodiment, an embodiment of the present application provides a data publishing device, which is applied to a service platform, and as shown in fig. 7, the device includes: an evaluation service module 710 and a data distribution service module 720;

the evaluation service module 710 is configured to receive an evaluation request sent by a first data end, where the evaluation request includes data to be evaluated and a service field to which the data to be evaluated belongs, and the data to be evaluated includes a deep learning model to be evaluated and/or a sample data set to be evaluated; evaluating the data to be evaluated by utilizing a deep learning model set and a verification data set which are pre-established aiming at the business field to obtain an evaluation result of the data to be evaluated; feeding back an evaluation result to the first data end so that the first data end judges whether to issue the data to be evaluated according to the evaluation result and sends a judgment result to the service platform;

the data publishing service module 720 is configured to receive a determination result sent by the first data end; if the judgment result is yes, the data to be evaluated is issued, so that the second data terminal can download the data to be evaluated selectively.

Optionally, for the deep learning model to be evaluated, the evaluation service module 710 may be specifically configured to:

Optionally, for the sample data set to be evaluated, the evaluation service module 710 may be specifically configured to:

Optionally, the apparatus may further include: an intelligent service module;

Optionally, the apparatus may further include: a crowdsourcing annotation service module;

Embodiments of the present application further provide a service platform, as shown in fig. 8, where the service platform includes a processor 801 and a machine-readable storage medium 802, where the machine-readable storage medium 802 stores machine-executable instructions that can be executed by the processor 801, and the processor is caused by the machine-executable instructions to: the data publishing method provided by the embodiment of the application is realized.

In the embodiment of the present application, the processor 801 is caused by machine executable instructions to realize that by reading the machine executable instructions stored in the machine readable storage medium 802: the method comprises the steps that a service platform receives an evaluation request sent by a first data end, a deep learning model set and a verification data set which are pre-established aiming at a business field in the evaluation request are utilized to evaluate data to be evaluated, an evaluation result of the data to be evaluated is obtained, the evaluation result is fed back to the first data end, after the first data end receives the evaluation result, whether the data to be evaluated is issued or not is judged according to the evaluation result, a judgment result is sent to the service platform, and if the judgment result sent by the first data end received by the service platform is positive, the data to be evaluated is issued so that a second data end can selectively download the data to be evaluated. The service platform evaluates the deep learning model to be evaluated and/or the sample data set to be evaluated in the service field of the evaluation request, whether the deep learning model to be evaluated and/or the sample data set to be evaluated are issued or not can be judged based on the evaluation result, after the deep learning model to be evaluated and/or the sample data set to be evaluated are issued, the second data end can select the deep learning model to be evaluated and/or the sample data set to be evaluated according to the actual requirements of the second data end, the evaluation result of the deep learning model to be evaluated and/or the sample data set to be evaluated is considered when the deep learning model to be evaluated and/or the sample data set to be evaluated is issued, the deep learning model to be evaluated and/or the sample data set to be evaluated can be selected according to the actual requirements of the second data end when the second data end is selected, and the deep learning model to be evaluated and/or the sample data, when the deep learning model is used, the second data end does not need to undergo long-time iterative training, and therefore accelerated landing of the deep learning model is achieved.

The machine-readable storage medium may include a RAM (Random Access Memory) and a NVM (Non-volatile Memory), such as at least one disk Memory. Alternatively, the machine-readable storage medium may be at least one memory device located remotely from the processor.

The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component.

In addition, a machine-readable storage medium is provided, where the machine-readable storage medium stores machine-executable instructions, and when the machine-readable storage medium is called and executed by a processor, the data publishing method provided in the embodiments of the present application is implemented.

In this embodiment, the machine executable instructions stored in the machine readable storage medium are capable of, when executed: the method comprises the steps that a service platform receives an evaluation request sent by a first data end, a deep learning model set and a verification data set which are pre-established aiming at a business field in the evaluation request are utilized to evaluate data to be evaluated, an evaluation result of the data to be evaluated is obtained, the evaluation result is fed back to the first data end, after the first data end receives the evaluation result, whether the data to be evaluated is issued or not is judged according to the evaluation result, a judgment result is sent to the service platform, and if the judgment result sent by the first data end received by the service platform is positive, the data to be evaluated is issued so that a second data end can selectively download the data to be evaluated. The service platform evaluates the deep learning model to be evaluated and/or the sample data set to be evaluated in the service field of the evaluation request, whether the deep learning model to be evaluated and/or the sample data set to be evaluated are issued or not can be judged based on the evaluation result, after the deep learning model to be evaluated and/or the sample data set to be evaluated are issued, the second data end can select the deep learning model to be evaluated and/or the sample data set to be evaluated according to the actual requirements of the second data end, the evaluation result of the deep learning model to be evaluated and/or the sample data set to be evaluated is considered when the deep learning model to be evaluated and/or the sample data set to be evaluated is issued, the deep learning model to be evaluated and/or the sample data set to be evaluated can be selected according to the actual requirements of the second data end when the second data end is selected, and the deep learning model to be evaluated and/or the sample data, when the deep learning model is used, the second data end does not need to undergo long-time iterative training, and therefore accelerated landing of the deep learning model is achieved.

For the embodiments of the data distribution system, the service platform and the machine-readable storage medium, the contents of the related methods are basically similar to the foregoing embodiments of the methods, so that the description is relatively simple, and for the relevant points, reference may be made to the partial description of the embodiments of the methods.

It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the embodiments of the data distribution system, the apparatus, the service platform and the machine-readable storage medium, since they are substantially similar to the embodiments of the method, the description is simple, and the relevant points can be referred to the partial description of the embodiments of the method.

The above description is only for the preferred embodiment of the present application, and is not intended to limit the scope of the present application. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. a data publishing system, it is characterised in that the system comprises a first data terminal, a service platform and a second data terminal:

The first data terminal is configured to send an evaluation request to the service platform when there is a need to evaluate the data to be evaluated, where the evaluation request includes the data to be evaluated and the business field to which the data to be evaluated belongs, The data to be evaluated includes a deep learning model to be evaluated and/or a sample data set to be evaluated;

The service platform is configured to receive the evaluation request sent by the first data terminal; use the deep learning model set and verification data set pre-established for the business field to evaluate the data to be evaluated, and obtain the result. the evaluation result of the data to be evaluated; feedback the evaluation result to the first data terminal;

The first data terminal is further configured to, after receiving the evaluation result fed back by the service platform, determine whether to release the data to be evaluated according to the evaluation result, and send the determination result to the service platform;

The service platform is further configured to receive the judgment result sent by the first data terminal; if the judgment result is yes, publish the to-be-evaluated data;

The second data terminal is used to select and download the data to be evaluated published by the service platform.

2. a data publishing method, characterized in that, applied to a service platform, the method comprising:

receiving an evaluation request sent by the first data terminal, the evaluation request includes the data to be evaluated and the business domain to which the data to be evaluated belongs, and the data to be evaluated includes the deep learning model to be evaluated and/or the sample data set to be evaluated;

Using the deep learning model set and verification data set pre-established for the business field, evaluate the data to be evaluated, and obtain the evaluation result of the data to be evaluated;

Feeding back the evaluation result to the first data terminal, so that the first data terminal judges whether to publish the data to be evaluated according to the evaluation result, and sends the judgment result to the service platform;

receiving the judgment result sent by the first data terminal;

If the judgment result is yes, the data to be evaluated is released, so that the second data terminal can choose to download the data to be evaluated.

3. The method according to claim 2, characterized in that, for the deep learning model to be evaluated, the deep learning model set and verification data set pre-established for the business field are used to analyze the data to be evaluated. Carry out an evaluation to obtain an evaluation result of the data to be evaluated, including:

According to the domain information of the business domain, determine a pre-established deep learning model set and a verification data set for the business domain;

Input the data in the verification data set into the deep learning model to be evaluated and each deep learning model in the deep learning model set, and obtain the output result of the deep learning model to be evaluated and the output of each deep learning model result;

Based on the output results of the deep learning models, the output results of the deep learning models to be evaluated are compared with the output results of the deep learning models to obtain the output results of the deep learning models to be evaluated and the output results of the deep learning models to be evaluated. Differences in the output results of the deep learning models, and according to the differences, determine the evaluation results of the deep learning models to be evaluated.

4 . The method according to claim 2 , wherein, for the sample data set to be evaluated, the deep learning model set and verification data set pre-established for the business field are used to analyze the data to be evaluated. 5 . Carry out an evaluation to obtain an evaluation result of the data to be evaluated, including:

Input each sample data in the sample data set to be evaluated into each deep learning model in the pre-established deep learning model set, and obtain the first output result of each deep learning model;

Inputting the data in the verification data set into each deep learning model in the pre-established deep learning model set to obtain the second output result of each deep learning model;

Using the second output result as a benchmark, compare the first output result with the second output result to obtain the difference between the first output result and the second output result, and according to the difference, Determine the evaluation result of the sample data set to be evaluated.

5. The method according to claim 2, wherein after the publishing the data to be evaluated, the method further comprises:

If the selection information for selecting the deep learning model to be evaluated sent by the second data terminal is received, send the deep learning model to be evaluated to the second data terminal, or send the deep learning model to the second data terminal The deep learning model after training the deep learning model to be evaluated using the verification data set;

If the selection information for selecting the deep learning model to be evaluated and the data to be processed sent by the second data terminal are received, the data to be processed is input to use the verification data set to train the deep learning model to be evaluated obtain the processing result, and send the processing result to the second data terminal;

If receiving the selection information for selecting the sample data set to be evaluated sent by the second data terminal, sending the sample data set to be evaluated to the second data terminal;

If the selection information for selecting the deep learning model to be evaluated and the sample data set to be evaluated sent by the second data terminal is received, use the sample data set to be evaluated to train the deep learning model to be evaluated , and send the trained deep learning model and the to-be-evaluated sample data set to the second data terminal.

6. The method according to claim 5, wherein the sending the deep learning model to be evaluated to the second data terminal comprises:

The sending to the second data terminal the deep learning model trained on the deep learning model to be evaluated by using the verification data set includes:

Encrypting the deep learning model trained by the deep learning model to be evaluated using the verification data set, and sending the encrypted deep learning model and encrypted identification information to the second data terminal;

The sending the processing result to the second data terminal includes:

Encrypting the processing result, and sending the encrypted processing result and encrypted identification information to the second data terminal;

The sending the to-be-evaluated sample data set to the second data terminal includes:

Encrypting the to-be-evaluated sample data set, and sending the encrypted to-be-evaluated sample data set and encrypted identification information to the second data terminal;

The sending of the trained deep learning model and the to-be-evaluated sample data set to the second data terminal includes:

Encrypting the trained deep learning model and the sample data set to be evaluated, and sending the encrypted deep learning model after training, the sample data set to be evaluated and encrypted identification information to the second data terminal.

7. The method according to claim 2, wherein after receiving the evaluation request sent by the first data terminal, the method further comprises:

If the received sample data set to be evaluated includes sample data without corresponding annotation files, send annotation prompt information to the first data terminal, so that the first data terminal receives the annotation prompt information after receiving the annotation prompt information. Select the type of annotation and send it to the service platform;

receiving the annotation type sent by the first data terminal;

According to the domain information of the business domain, select a preset deep learning model from a set of deep learning models pre-established for the business domain;

According to the labeling type, use the preset deep learning model to label the sample data to obtain an initialization labeling result;

Feeding back the initialization labeling result to the first data terminal, so that the first data end sets the labeling task information and sends it to the service platform according to the initializing labeling result;

receiving the labeling task information sent by the first data terminal, and publishing the sample data and the labeling task information, so that the labeling end judges whether to label the sample data based on the labeling task information;

If an annotation file in which the sample data is annotated by the annotation terminal is received, the corresponding record of the annotation file is recorded in the sample data set to be evaluated.

8. A data publishing device, characterized in that, when applied to a service platform, the device comprises: an evaluation service module and a data publishing service module;

The evaluation service module is used to receive an evaluation request sent by the first data terminal, the evaluation request includes the data to be evaluated and the business domain to which the data to be evaluated belongs, and the data to be evaluated includes the deep learning model to be evaluated and/or Or the sample data set to be evaluated; use the deep learning model set and verification data set pre-established for the business field to evaluate the data to be evaluated to obtain the evaluation result of the data to be evaluated; to the first data The terminal feeds back the evaluation result, so that the first data terminal judges whether to release the data to be evaluated according to the evaluation result, and sends the judgment result to the service platform;

The data publishing service module is configured to receive the judgment result sent by the first data terminal; if the judgment result is yes, publish the to-be-evaluated data for the second data terminal to choose to download the to-be-evaluated data. Evaluate data.

9. The device according to claim 8, wherein, for the deep learning model to be evaluated, the evaluation service module is specifically used for:

10. The apparatus according to claim 8, wherein, for the sample data set to be evaluated, the evaluation service module is specifically used for:

11. The apparatus according to claim 8, wherein the apparatus further comprises: an intelligent service module;

The intelligent service module is configured to send the deep learning model to be evaluated to the second data terminal if the selection information for selecting the deep learning model to be evaluated sent by the second data terminal is received, or, Send the deep learning model after using the verification data set to train the deep learning model to be evaluated to the second data terminal; if receiving the selection of the deep learning model to be evaluated sent by the second data terminal information and data to be processed, input the data to be processed into the deep learning model trained on the deep learning model to be evaluated using the verification data set, obtain the processing result, and send the data to the second data terminal. process result.

12. The apparatus according to claim 8, wherein the apparatus further comprises: a crowdsourced labeling service module;

The crowdsourcing labeling service module is configured to send labeling prompt information to the first data terminal if the received sample data set to be evaluated includes sample data without corresponding labeling files, so that the first data After receiving the annotation prompt information, the terminal selects the annotation type and sends it to the service platform; receives the annotation type sent by the first data terminal; Selecting a preset deep learning model from a set of established deep learning models; according to the labeling type, using the preset deep learning model to label the sample data to obtain an initialization labeling result; and feeding back to the first data terminal The initialized labeling result, so that the first data terminal sets the labeling task information to send to the service platform according to the initialized labeling result; receives the labeling task information sent by the first data end, and publishes it The sample data and the labeling task information, so that the labeling end determines whether to label the sample data based on the labeling task information; if receiving the labeling file that the labeling end labels the sample data, then Recording corresponding to the marked file into the sample data set to be evaluated.