CN109829375A - A machine learning method, device, equipment and system - Google Patents

Abstract

The present application discloses a machine learning method, device, device and system. The system includes: a processing module for cyclically executing a target operation until a test result associated with data in a test data set satisfies a preset condition; the processing module includes: a labeling module , training module and calculation module; storage module, used to store the obtained data set and test data set; labeling module, used to label the data if the data in the data set is not labelled; training module, used to label the data After feature extraction is performed on the data that is greater than or equal to the first preset threshold, the extracted features are input into the pre-training model to obtain a trained training model; the computing module is used to input the data in the test data set into the trained model. Train the model, obtain the test results associated with the data in the test data set, and evaluate the test results to evaluate whether the test results meet the preset conditions. By adopting the present application, the model training speed can be improved.

Description

A machine learning method, device, equipment and system

technical field

The present application relates to the technical field of machine learning, and in particular, to a machine learning method, apparatus, device and system.

Background technique

With the growth of data scale, computing power, and storage capacity, artificial intelligence has developed rapidly and has been widely used in various fields to mine the value of data. Behind the rapid development is a large amount of human input. The industry circulating "artificial intelligence has as much intelligence as there are artificial intelligence", which refers to the need to invest a lot of manpower before combining artificial intelligence technology with specific businesses. First, large-scale After labeling, algorithm engineers spend a lot of time writing code for training and parameter adjustment, and finally outputting a model. A complete machine learning process includes dataset labeling, dataset processing, feature processing, model training, model evaluation, and model deployment. In order to lower the threshold of machine learning and improve the efficiency of model training, a machine learning platform can be built on top of a machine learning framework such as tensorflow, providing a friendly interaction method and supporting process operations. way to generate a trained directed acyclic graph.

However, in the actual production environment, a model that can be commercialized (mainly unsupervised learning) often requires multiple rounds of training iterations, and it is necessary to collect and label the data set for retraining to improve the accuracy of the model, which is a closed-loop process. Each iteration starts from importing unlabeled raw data into the labeling system, then exporting it from the labeling system, and then importing it into the machine learning system. The operation is complicated, and the large-scale data (TB level) is transmitted to and from different systems, and the transmission speed is It is slow and prone to network interruptions. Multiple copies of the same data are stored in different systems, which is redundant and wastes storage space. The current annotation is manual, which is time-consuming, labor-intensive, and inefficient.

SUMMARY OF THE INVENTION

The present application provides a machine learning method, device, equipment and system, which can realize shared storage between the labeling module and the training module. On the one hand, by sharing the data, data redundancy is reduced, and the utilization rate of storage space is greatly improved. On the one hand, it reduces the need for a large number of data sets to be transmitted between different systems (in the prior art, the labeling systems and training systems are far apart in space), which reduces the delay of data transmission and improves the efficiency of model training. In addition, the machine automatically labels the data in the unlabeled dataset, which improves the labeling speed of the data in the dataset.

In a first aspect, the present application provides a machine learning system, the system comprising:

a processing module, used for cyclically executing the target operation until the test result associated with the data in the test data set satisfies the preset condition; the processing module includes: a labeling module, a training module and a computing module;

The storage module is used to store the obtained data set and test data set;

The labeling module is configured to label the data in the data set if the data in the obtained data set is not labelled;

The training module is used to screen the marked data that is greater than or equal to the first preset threshold, and then perform feature extraction on the screened data, and then input the extracted features into the pre-training model to obtain a trained model. training a model; the feature is used to train the pre-training model;

The computing module is configured to, after inputting the data in the obtained test data set into the trained training model, and obtaining the test result associated with the data in the test data set, perform the data associated with the data in the test data set. The test result is evaluated to evaluate whether the test result associated with the data in the test data set satisfies the preset condition.

In combination with the method of the first aspect, in some optional embodiments,

After judging whether the data in the data set is marked,

The training module is further configured to perform feature extraction on the data in the data set if all the data in the data set are marked, and input the extracted features into the pre-training model to obtain a trained training model; the features for training the pre-training model;

The computing module is further configured to, after inputting the obtained data in the test data set into the trained training model, and obtaining the test result of the data association in the test data set, correlate the data in the test data set The test results are evaluated to evaluate whether the test results associated with the data in the test data set meet the preset conditions.

In combination with the method of the first aspect, in some optional embodiments,

The labeling module is configured to label the data in the data set by means of machine labeling if the data in the data set is not labelled.

In combination with the method of the first aspect, in some optional embodiments,

The labeling module is further configured to visualize the data after labeling the data in the data set by means of machine labeling.

In combination with the method of the first aspect, in some optional embodiments,

The training module is used for screening the data whose marked confidence is greater than or equal to the first preset threshold.

In combination with the method of the first aspect, in some optional embodiments,

The computing module is used to evaluate the test results associated with the data in the test data set, so as to evaluate whether the difference between the test results associated with the data in the test data set and the preset reference results associated with the test data satisfies less than the second preset threshold.

In a second aspect, the application provides a machine learning method, the device comprising:

The target operation is performed cyclically until the test result associated with the data in the test data set satisfies the preset condition; the target operation includes:

get the dataset;

Determine whether the data in the data set is marked, and if the data in the data set is not marked, mark the data in the data set;

Screen the marked data that meets the first preset threshold, and perform feature extraction on the screened data; input the extracted features into the pre-training model to obtain a trained training model; the features are used to Pre-trained model for training;

Input the data in the acquired test data set into the trained training model, and obtain the test result associated with the data in the test data set; the data in the test data set is the marked data;

Evaluate the test results associated with the data in the test data set to evaluate whether the test results associated with the data in the test data set satisfy a preset condition.

In a third aspect, the application provides a machine learning device, the device comprising:

a processing unit, configured to cyclically execute a target operation until the test result associated with the data in the test data set satisfies a preset condition; the target operation;

The processing unit includes:

The first acquisition unit, judgment unit, labeling unit, screening unit, extraction unit, training unit,

a second acquisition unit and an evaluation unit;

the first obtaining unit, used to obtain a data set;

The judging unit is used to judge whether the data in the data set is marked;

The labeling unit is configured to label the data in the data set if the data in the data set is not labelled;

The screening unit is configured to screen the marked data that meets the first preset threshold;

The extraction unit is used to perform feature extraction on the filtered data;

The training unit is used to input the extracted features into the pre-training model to obtain a trained training model; the features are used to train the pre-training model;

The second obtaining unit is configured to input the obtained data in the test data set into the trained training model, and obtain the test result associated with the data in the test data set; the data in the test data set is the marked data ;

The evaluation unit is configured to evaluate the test results associated with the data in the test data set, so as to evaluate whether the test results associated with the data in the test data set satisfy a preset condition.

In a fourth aspect, the present application provides a device including an input device, an output device, a processor, and a memory, wherein the processor, the input device, the output device, and the memory are connected to each other, wherein the memory is used to store and support the execution of the device. In the application code of the above method, the processor is configured to execute the machine learning method provided in the above second aspect.

In a fifth aspect, the present application provides a computer-readable storage medium for storing one or more computer programs, wherein the one or more computer programs include instructions, and when the above-mentioned computer programs are run on a computer, the above-mentioned instructions use for executing the machine learning method provided in the second aspect.

In a sixth aspect, the present application provides a computer program, the computer program includes machine learning instructions, and when the computer program is executed on a computer, the above-mentioned machine learning instructions are used to execute the machine learning method provided in the second aspect.

The present application provides a machine learning method, apparatus, device and system. Wherein, the system includes: a processing module, which is used for cyclically executing the target operation until the test result associated with the data in the test data set satisfies the preset condition; the processing module includes: a labeling module, a training module and a computing module; and a storage module, which is used for storing and obtaining The obtained data set and the test data set; the labeling module is used to label the data in the data set if the data in the obtained data set is not labelled; the training module is used to label the data that meets the first preset threshold after labeling. The data is screened, and after feature extraction is performed on the screened data, the extracted features are input into the pre-training model to obtain a trained training model; the features are used to train the pre-training model; the computing module is used for Input the obtained data in the test dataset into the trained training model, and after obtaining the test results associated with the data in the test dataset, evaluate the test results associated with the data in the test dataset to evaluate the test results associated with the data in the test dataset Whether the preset conditions are met. By adopting the present application, on the one hand, the data stored in the labeling module and the training module are shared (such as shared pictures and other data in a dataset that needs to occupy a large storage space). By sharing this data, data redundancy is reduced and greatly improved The utilization rate of storage space, on the other hand, reduces the need for a large amount of data in the data set to be transmitted between different systems (in the prior art, the labeling system and the training system which are far apart in space and are independent of each other), when the retrieval of data transmission In addition, the machine automatically labels the data in the unlabeled data set, which improves the labeling speed of the data in the data set.

Description of drawings

In order to explain the technical solutions of the embodiments of the present application more clearly, the following briefly introduces the accompanying drawings used in the description of the embodiments. For those of ordinary skill, other drawings can also be obtained from these drawings without any creative effort.

1 is a schematic diagram of the architecture of a machine learning system provided by the application;

2 is a schematic flowchart of a machine learning method provided by the present application;

3 is a schematic flowchart of another machine learning method provided by the present application;

4 is a schematic block diagram of an apparatus provided by an embodiment of the present application;

FIG. 5 is a schematic block diagram of a device provided by the present application.

Detailed ways

The technical solutions in the present application will be clearly and completely described below with reference to the accompanying drawings in the present application. Obviously, the described embodiments are part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

It is to be understood that, when used in this specification and the appended claims, the terms "comprising" and "comprising" indicate the presence of the described features, integers, steps, operations, elements and/or components, but do not exclude one or The presence or addition of a number of other features, integers, steps, operations, elements, components, and/or sets thereof.

It should also be understood that the terminology used in the specification of the application herein is for the purpose of describing particular embodiments only and is not intended to limit the application. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural unless the context clearly dictates otherwise.

It should also be further understood that, as used in this specification and the appended claims, the term "and/or" refers to and including any and all possible combinations of one or more of the associated listed items .

As used in this specification and the appended claims, the term "if" may be contextually interpreted as "when" or "once" or "in response to determining" or "in response to detecting" . Similarly, the phrases "if it is determined" or "if the [described condition or event] is detected" may be interpreted, depending on the context, to mean "once it is determined" or "in response to the determination" or "once the [described condition or event] is detected. ]" or "in response to detection of the [described condition or event]".

In specific implementations, the devices described in this application include, but are not limited to, other portable devices such as mobile phones, laptops, or tablet computers with touch-sensitive surfaces (eg, touchscreen displays and/or touchpads). It should also be understood that in some embodiments, the device is not a portable communication device, but rather a desktop computer with a touch-sensitive surface (eg, a touch screen display and/or a touch pad).

In the discussion that follows, a device including a display and a touch-sensitive surface is described. It should be understood, however, that the device may include one or more other physical user interface devices such as a physical keyboard, mouse, and/or joystick.

The device supports a variety of applications, such as one or more of the following: drawing applications, presentation applications, word processing applications, website creation applications, disk burning applications, spreadsheet applications, gaming applications, telephony applications programs, video conferencing applications, email applications, instant messaging applications, exercise support applications, photo management applications, digital camera applications, digital video camera applications, web browsing applications, digital music player applications and / or a digital video player application.

Various applications that may execute on the device may use at least one common physical user interface device, such as a touch-sensitive surface. One or more functions of the touch-sensitive surface and corresponding information displayed on the device may be adjusted and/or changed between applications and/or within respective applications. In this way, the common physical architecture of the device (eg, touch-sensitive surfaces) can support various applications with user interfaces that are intuitive and transparent to the user.

In order to better understand the present application, the following describes the architecture diagram of the machine learning system to which the present application applies. Please refer to FIG. 1. FIG. 1 is an architecture diagram of a machine learning system provided by the present application. As shown in FIG. 1 , the system may include, but is not limited to, a web product interface design (Website User Interface, WebUi), a workflow module, a training module, a labeling module, a storage module, and a computing module. The workflow module may include: a state transition interface (Restful Api), an operation engine (Operator Engine), a directed acyclic graph engine (DAG Engine), a database, an executor, and the like.

It should be noted that, in the machine learning system, the acquisition of the data set used by the training module for training the pre-training model may include but not limited to the following steps:

Step 1: Based on the state transfer interface of the http request in the workflow module, a workflow is formed by dragging and dropping components on the Internet (WorldWide Web, Web) end such as a browser. It should be noted that the workflow includes the acquisition of the data set, wherein the acquisition of the data in the data set is a task in the workflow.

Step 2: Analyze the workflow through the directed acyclic graph engine in the workflow module, and store the parsed workflow in the database.

Step 3: When the operation engine in the workflow module receives the scheduling request sent by the scheduling engine sent to the acyclic graph engine, the operation engine reads tasks in the workflow including data set acquisition from the database.

Step 4: Execute tasks such as acquiring the data set read by the operation engine through the executor in the workflow module.

Step 5: The acquired data set can be stored in the storage module.

After the data set is acquired, the target operation can be performed cyclically through modules such as the labeling module, the training module, and the computing module. The labeling module is used to label the data in the data set if the data in the obtained data set is not labelled.

The training module is used to filter the marked data that is greater than or equal to the first preset threshold, and then perform feature extraction on the filtered data, and then input the extracted features into the pre-training model to obtain a trained training model. ; Features are used to train the pre-trained model;

The calculation module is used to evaluate the test results associated with the data in the test data set after inputting the obtained data in the test data set into the trained training model and obtaining the test results associated with the data in the test data set, so as to evaluate the test data Whether the test results associated with the centralized data meet the preset conditions.

It should be noted that after judging whether the data in the dataset is marked,

The training module is also used to extract features from the data in the dataset if all the data in the dataset are marked, and input the extracted features into the pre-training model to obtain a trained training model; the features are used to train the pre-training model .

The labeling module is specifically used to label the data in the data set by machine labeling if the data in the data set is not labelled.

The labeling module is further configured to visualize the data after labeling the data in the dataset by means of machine labeling.

The training module can specifically be used to filter the data whose marked confidence is greater than or equal to the first preset threshold.

In the embodiment of the present application, the system includes: a processing module for cyclically executing the target operation until the test result associated with the data in the test data set satisfies a preset condition; the processing module includes: a labeling module, a training module and a computing module; a storage module, It is used to store the obtained data set and test data set; the labeling module is used to label the data in the data set if the data in the obtained data set is not labelled; the training module is used to label the data that satisfies the first The data with the preset threshold is screened, and after feature extraction is performed on the screened data, the extracted features are input into the pre-training model to obtain a trained training model; the features are used to train the pre-training model; the computing module , used to input the obtained data in the test data set into the trained training model, and obtain the test results associated with the data in the test data set, and then evaluate the test results associated with the data in the test data set to evaluate the data in the test data set. Whether the associated test result meets the preset condition. By adopting the present application, on the one hand, the data stored in the labeling module and the training module are shared (such as shared pictures and other data in a dataset that needs to occupy a large storage space). By sharing this data, data redundancy is reduced and greatly improved The utilization rate of storage space, on the other hand, reduces the need for a large amount of data in the dataset to be transmitted between different systems (in the prior art, the labeling system and the training system that are far apart from each other in space), reducing the time required for data transmission. In addition, the machine automatically labels the data in the unlabeled data set, which improves the labeling speed of the data in the data set.

Referring to FIG. 2, it is a schematic flowchart of a machine learning method provided by the present application. As shown in FIG. 2, the method may at least include the following steps:

S201. Acquire a data set containing data.

The data in the data set in the embodiments of the present application may include, but are not limited to, types such as pictures, videos, voices, or texts.

Additionally, the data can be classified as raw data or labeled data. Among them, the label is the attribute of the data, such as men and women in the gender classification, and the label is the human experience knowledge, which needs to be provided artificially. It should be noted that a trained training model can be obtained through data learning, and the trained training model can be expressed as a function mapping, that is to say, there is a certain correspondence between the input and output of the training model, and the empirical data This correspondence can be learned by inputting (labeled data) into the training model. When faced with new test data, the trained training model will output the test results associated with the new test data.

With reference to Figure 1, in the machine learning system, the acquisition of data sets may include but not limited to the following steps:

Step 1: Based on the state transfer interface of the http request in the workflow module, a workflow is formed by dragging and dropping components on the Internet (WorldWide Web, Web) end such as a browser. It should be noted that the workflow includes the acquisition of the data set, wherein the acquisition of the data in the data set is a task in the workflow.

Step 2: Analyze the workflow through the directed acyclic graph engine in the workflow module, and store the parsed workflow in the database.

Step 3: When the operation engine in the workflow module receives the scheduling request sent by the scheduling engine sent to the acyclic graph engine, the operation engine reads the tasks in the workflow from the database (such as the acquisition of data in the dataset) .

Step 4: Execute tasks such as acquiring the data set read by the operation engine through the executor in the workflow module.

Step 5: The acquired data set can be stored in the storage module.

S202. Determine whether the data in the data set is marked.

In the embodiment of the present application, the data in the data set is labeled, in other words, the data in the data set is labeled. For example, in image processing, such as age recognition of people, labeling is to use a frame to mark a person's face on a picture and mark his age; in identification with or without a mask, it is to frame the picture. Show your face and mark whether you are wearing a mask. These tags are ultimately stored in the form of numerical codes.

It should be noted that after judging whether the data in the dataset is marked, it also includes:

If all the data in the dataset are labeled, perform feature extraction on the data in the dataset;

Input the extracted feature into the pre-training model to obtain a trained training model; the feature is used to train the pre-training model;

Input the data in the test data set into the trained training model, and obtain the test results associated with the data in the test data set;

The test result is evaluated to evaluate whether the test result associated with the data in the test dataset satisfies the preset condition.

Specifically, the test results associated with the data in the test data set are evaluated to evaluate whether the test results associated with the data in the test data set meet the preset conditions, which may specifically include the following processes:

Evaluate the test results associated with the data in the test data set to evaluate whether the difference between the test results associated with the data in the test data set and the preset reference results associated with the test data satisfies less than a second preset threshold. The preset reference result associated with the test data is the preset reference result associated with the test data.

For example, taking the second preset threshold of 5% as an example, for example, the data in the test data set is a person's face, and his age is marked as 30 years old. If the output test result is: according to the person's face, it can be determined that the person's age is 30 years old, and the confidence level is 50%. When the preset reference result is: the face of the person, it can be determined that the age of the person is 30 years old, and the confidence level reaches 95%, the test result associated with the data in the test data set and the preset reference result associated with the test data are the same. The difference between them does not meet the second preset threshold.

S203, if the data in the dataset is not marked, mark the data in the dataset.

In the embodiment of the present application, specifically, marking (labeling) the data in the data set may include but not be limited to the following two ways.

Method 1: If the data in the data set is not labeled, the data in the data set is labeled by the machine.

Method 2: If the data in the dataset is not labeled, label the data in the dataset by manual labeling.

S204: Screen the marked data that meets the first preset threshold.

Specifically, screening the marked data that meets the first preset threshold may specifically include the following process:

The data whose annotated confidence levels meet the first preset threshold are screened.

It should be noted that data with high confidence is filtered out by filtering the data whose marked confidence meets the first preset threshold.

It should be noted that the first preset threshold may be 90%.

Taking age recognition as an example, in age recognition, a batch of face data is input, and after the above-mentioned face data can be marked by machine, the batch of face data will have age labels. For example, when a person's face data is labeled, when the label of the person's face is 20 years old and the confidence level is 71%, the machine can think that there is a 71% probability that the person is 20 years old. In age recognition, in order to filter out data with high confidence, when the first preset threshold is 90%, face data with a confidence level of more than 90% can be filtered out, and face data with a confidence level of less than 90% are invalid. , because the face data below the confidence level is used to train the pre-training model, and a training model with poor training effect will be obtained.

S205. Perform feature extraction on the filtered data.

In this embodiment of the present application, feature extraction may be performed on the filtered data through a neural network. The neural network may include, but is not limited to, a Long Short-Term Memory (LSTM), a Recurrent Neural Network (RNN), a deep convolutional network, and a deep residual network.

For example, feature extraction can be performed on the filtered data with a confidence level of more than 90% through a recurrent neural network.

S206. Input the extracted features into the pre-training model to obtain a trained training model; input the data in the pre-stored test data set into the trained training model to obtain a test result associated with the data in the test data set.

In the embodiment of the present application, the feature is used to train the pre-training model; the data in the test data set is the marked data.

It should be noted that, taking face data as an example, the test result of the data association in the test data set can be as follows: according to A's face data, it can be determined that A's age is 30 years old, and the confidence level is 90%.

S207: Evaluate the test results associated with the data in the test data set to evaluate whether the test results associated with the data in the test data set meet the preset conditions.

In the embodiment of the present application, the test results of the data association in the test data set are evaluated to evaluate whether the test results of the data association in the test data set meet the preset conditions, and if the test results of the data association in the test data set do not meet the preset conditions, Steps S201 to S207 are executed cyclically until the test result associated with the data in the test data set satisfies the preset condition.

Specifically, evaluating the test results associated with the data in the test data set to evaluate whether the test results associated with the data in the test data set meet the preset conditions may include but are not limited to the following two situations.

Case 1: Evaluate the test results associated with the data in the test data set to assess whether the difference between the test results associated with the data in the test data set and the preset reference results associated with the test data satisfies less than a second preset threshold.

When the data in the test set is a single test data, whether the difference between the test result associated with the test data and the preset reference result associated with the test data is less than a second preset threshold.

In the case of age recognition, taking the test data as face data and the second preset threshold of 4% as an example, according to the person's face, it can be determined that the person's age is 50 years old, and the confidence level is 70%. When the preset reference result is: the face of the person, it can be determined that the age of the person reaches 50 years old, and the confidence level reaches 96%, the test result associated with the data in the test data set and the preset reference result associated with the test data are the same. The difference between them does not meet the second preset threshold.

Situation 2: The test results associated with the data in the test data set are respectively evaluated, so as to separately evaluate whether the difference between the test results associated with the data in the test data set and the preset reference results associated with the data in the test data set is all less than the second Preset threshold.

In the case of age recognition, taking the test data as multiple face data and the second preset threshold value of 7% as an example,

For example, the data in the test data set is a person's face, and his age is marked as 30 years old. If the output test result is: based on the person's face, it can be determined that the person's age is 30 years old, and the confidence level is 90%. When A's preset reference result is: A's age reaches 30 years old, and the confidence level reaches 94%; at this time, the difference between the test result associated with A and A's preset reference result is less than the second preset reference result. threshold.

For example, the data in the test data set is a person's face, and his age is marked as 60 years old. If the output test result is: based on the person's face, it can be determined that the person's age is 60 years old, and the confidence level is 89%. When the preset reference result of B is: B's age reaches 60 years old, and the confidence level reaches 94%; at this time, the difference between the test result associated with B and the preset reference result of B satisfies that it is smaller than the second preset reference result threshold.

By synthesizing A and B, it can be judged that the difference between the test result of A and the ideal result of A and the difference between the test result of B and the ideal result of B are all less than the second preset threshold.

Another machine learning method is introduced below.

FIG. 3 exemplarily shows a schematic flow chart of a machine learning method.

As shown in Figure 3, the following target operations are performed in a loop until the derived derivation function satisfies the requirements (until the accuracy of the output of the trained model is high enough). The target operation includes: labeling the data in the data set (Dataset-1) through a data labeling function (Data LabelingFunction) to obtain the labelled data; Feature extraction to obtain corresponding features; then, by inputting the above-mentioned corresponding features into the pre-training model (Pretraining Model), the pre-training model is trained to obtain a trained training model (Training Function); then, the test data are The Test Dataset is input into the training model to obtain the Evaluation Function. Finally, on the one hand, the obtained evaluation function is deployed to obtain the Deploy Function; on the other hand, the obtained evaluation function is combined with the Dataset. -1 to get the inference function.

To sum up, in the embodiment of the present application, the target operation is performed cyclically until the test result associated with the data in the test data set satisfies the preset condition; the target operation includes: acquiring a data set containing data; judging whether the data in the data set is marked, if If the data in the dataset is not marked, mark the data in the dataset; screen the marked data that meets the first preset threshold, and perform feature extraction on the filtered data; input the extracted features into the pre-training model, Obtain the trained training model; the features are used to train the pre-training model; input the data in the pre-stored test data set into the trained training model to obtain the test results associated with the data in the test data set; the data in the test data set is marked The test results associated with the data in the test data set are evaluated to evaluate whether the test results associated with the data in the test data set meet the preset conditions. Using the embodiments of the present application, on the one hand, by sharing the data, data redundancy is reduced, and the utilization rate of storage space is greatly improved; Spatially distant and independent labeling systems and training systems), the data transmission delay is reduced, and the speed of model training is improved. The speed of labeling the data in the dataset.

It should be noted that FIG. 3 is only used to explain the embodiment of the present application, and should not limit the present application.

It is understandable that for related definitions and descriptions not provided in the embodiment of the method in FIG. 2 , reference may be made to the embodiment in FIG. 1 , and details are not repeated here.

Referring to FIG. 4 , it is a machine learning device provided by the present application. As shown in FIG. 4 , the machine learning device 40 includes: a first obtaining unit 401 , a judging unit 402 , a labeling unit 403 , a screening unit 404 , an extracting unit 405 , a training unit 406 , a second obtaining unit 407 and an evaluating unit 408 . in:

The first obtaining unit 401 is used to obtain a data set.

The judging unit 402 is used for judging whether the data in the data set is marked.

The labeling unit 403 is configured to label the data in the data set if the data in the data set is not labelled.

The screening unit 404 is configured to screen the marked data satisfying the first preset threshold.

The extraction unit 405 is configured to perform feature extraction on the filtered data.

The training unit 406 is configured to input the extracted features into the pre-training model to obtain a trained training model; the features are used to train the pre-training model.

The second obtaining unit 407 is configured to input the obtained data in the test data set into the trained training model, and obtain the test result associated with the data in the test data set; the data in the test data set is the marked data.

The evaluating unit 408 is configured to evaluate the test results associated with the data in the test data set, so as to evaluate whether the test results associated with the data in the test data set satisfy the preset condition.

The labeling unit 403 can be specifically used for:

If the data in the data set is not labeled, the data in the data set is labeled by means of machine labeling.

The screening unit 404 is specifically used for:

The data whose annotated confidence levels meet the first preset threshold are screened.

The evaluation unit 408 is specifically used for:

Evaluate the test results associated with the data in the test data set to evaluate whether the difference between the test results associated with the data in the test data set and the preset reference results associated with the test data satisfies less than a second preset threshold.

or,

Evaluate the test results associated with the data in the test data set, respectively, to evaluate whether the difference between the test results associated with the data in the test data set and the preset reference results associated with the data in the test data set both satisfies less than a second preset threshold .

To sum up, in this embodiment of the present application, the device 40 cyclically executes the target operation through the processing unit until the test result associated with the data in the test data set satisfies the preset condition; the target operation; the processing unit includes:

The first acquisition unit 401, the judgment unit 402, the labeling unit 403, the screening unit 404, the extraction unit 405, the training unit 406, the second acquisition unit 407 and the evaluation unit 408; the first acquisition unit 401 is used to acquire the data set; the judgment unit 402, for judging whether the data in the data set is marked; the marking unit 403, for marking the data in the data set if the data in the data set is not marked; the screening unit 404, for marking the marked data satisfying the first preset threshold The extraction unit 405 is used for feature extraction from the screened data; the training unit 406 is used to input the extracted features into the pre-training model to obtain a trained training model; the features are used for pre-training The model is trained; the second acquisition unit 407 is used to input the acquired data in the test data set into the trained training model, and obtain the test result associated with the data in the test data set; the data in the test data set is the marked data; the evaluation unit 408: Evaluate the test result of the data association in the test data set, so as to evaluate whether the test result of the data association in the test data set satisfies the preset condition. By adopting the embodiments of the present application, it is realized that the labeling unit and the training unit share the stored data. On the one hand, by sharing the data, data redundancy is reduced, and the utilization rate of the storage space is greatly improved. On the other hand, a large number of The data in the dataset needs to be transmitted between different systems (in the prior art, the labeling systems and training systems that are far apart from each other in space) reduce the delay of data transmission and improve the speed of model training. Automatic labeling of data in unlabeled datasets improves the labeling speed of data in datasets.

It should be understood that device 40 is merely an example provided by the embodiments of the present application, and that device 40 may have more or fewer components than those shown, may combine two or more components, or may have a combination of components Different configurations are implemented.

It is understandable that, for the specific implementation of the functional blocks included in the apparatus 40 in FIG. 4 , reference may be made to the embodiments described in the foregoing FIG. 1 and FIG. 2 , and details are not repeated here.

FIG. 5 is a schematic structural diagram of a machine learning device provided by the present application. In the embodiments of the present application, the devices may include mobile phones, tablet computers, personal digital assistants (Personal Digital Assistant, PDA), mobile Internet devices (Mobile Internet Device, MID), smart wearable devices (such as smart watches, smart wristbands), etc. This kind of equipment is not limited in this embodiment of the present application. As shown in FIG. 5 , the device 50 may include: a baseband chip 501 , a memory 502 (one or more computer-readable storage media), and a peripheral system 503 . These components may communicate on one or more communication buses 504 .

The baseband chip 501 may include: one or more processors (CPUs) 505 .

The processor 505 can be specifically used for:

The target operation is performed cyclically until the test result associated with the data in the test data set satisfies the preset condition; the target operation includes:

Get the dataset.

Determine whether the data in the dataset is labeled, and if the data in the dataset is not labeled, label the data in the dataset.

Screen the marked data that meets the first preset threshold, and perform feature extraction on the screened data; input the extracted features into the pre-training model to obtain a trained training model; the features are used for the pre-training model. train.

Input the obtained data in the test data set into the trained training model, and obtain the test result associated with the data in the test data set; the data in the test data set is the marked data.

Evaluate the test result of the data association in the test data set, so as to evaluate whether the test result of the data association in the test data set satisfies the preset condition.

Memory 502 is coupled to processor 505 and may be used to store various software programs and/or sets of instructions. In particular implementations, memory 502 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. The memory 502 may store an operating system (hereinafter referred to as a system), such as an embedded operating system such as ANDROID, IOS, WINDOWS, or LINUX. Memory 502 may also store network communication programs that may be used to communicate with one or more additional devices, one or more device devices, one or more network devices. The memory 502 can also store a user interface program, which can vividly display the content of the application program through a graphical operation interface, and receive user control operations on the application program through input controls such as menus, dialog boxes, and keys. .

Understandably, the memory 502 may be used to store implementation codes for implementing the machine learning method.

Memory 502 may also store one or more application programs. As shown in Figure 5, these applications may include: social applications (eg Facebook), image management applications (eg photo albums), map applications (eg Google Maps), browsers (eg Safari, Google Chrome), etc. .

The peripheral system 503 is mainly used to realize the interaction function between the user of the device 50 and the external environment, and mainly includes the input and output devices of the device 50 . In a specific implementation, the peripheral system 503 may include: a display screen controller 507 , a camera controller 508 and an audio controller 509 . Wherein, each controller can be coupled with its corresponding peripheral devices (such as the display screen 510 , the camera 511 and the audio circuit 52 ). In some embodiments, the display screen may be configured with a display screen 1 of a self-capacitive floating touch panel, or may be a display screen configured with an infrared floating touch panel. In some embodiments, the camera 511 may be a 3D camera. It should be noted that the peripheral system 503 may also include other I/O peripherals.

To sum up, in the embodiment of the present application, the device 50 performs the target operation cyclically until the test result associated with the data in the test data set satisfies the preset condition; the target operation includes: the device 50 obtains the data set through the processor 505; the device 50 obtains the data set through the processing The device 505 determines whether the data in the data set is marked. If the data in the data set is not marked, the device 50 uses the processor 505 to mark the data in the data set; the device 50 uses the processor 505 to mark the marked data that meets the first preset threshold. For screening, the device 50 performs feature extraction on the screened data through the processor 505; the device 50 inputs the extracted features into the pre-training model through the processor 505 to obtain a trained training model; the features are used for the pre-training model. training; input the acquired data in the test data set into the trained training model, and the device 50 obtains the test results associated with the data in the test data set through the processor 505; the data in the test data set is the marked data; the device 50 passes the processor 505 Evaluate the test result of the data association in the test data set, so as to evaluate whether the test result of the data association in the test data set satisfies the preset condition. Using the embodiments of the present application, it is possible to quickly label and train the data in the data set. On the one hand, by sharing the data, data redundancy is reduced, and the utilization rate of storage space is greatly improved; on the other hand, a large number of The data in the dataset needs to be transmitted between different systems (in the prior art, the labeling systems and training systems that are far apart from each other in space) reduce the delay of data transmission and improve the speed of model training. Automatic labeling of data in unlabeled datasets improves the labeling speed of data in datasets.

It should be understood that device 50 is merely an example provided by the embodiments of the present application, and that device 50 may have more or fewer components than those shown, may combine two or more components, or may have a combination of components Different configurations are implemented.

It is understandable that, for the specific implementation of the functional modules included in the device 50 in FIG. 5 , reference may be made to the embodiments in FIG. 1 and FIG. 2 , and details are not repeated here.

The present application provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and the computer program is implemented when executed by a processor.

The computer-readable storage medium may be an internal storage unit of the device described in any of the foregoing embodiments, such as a hard disk or a memory of the device. The computer-readable storage medium may also be an external storage device of the device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a Secure Digital (SD) card, a flash memory card (Flash card) equipped on the device. Card), etc. Further, the computer-readable storage medium may also include both an internal storage unit of the device and an external storage device. The computer-readable storage medium is used to store computer programs and other programs and data required by the device. The computer-readable storage medium can also be used to temporarily store data that has been or will be output.

The present application also provides a computer program product, the computer program product comprising a non-transitory computer-readable storage medium storing a computer program, the computer program being operable to cause a computer to execute any of the methods described in the above method embodiments some or all of the steps. The computer program product may be a software installation package, the computer including the electronic device.

Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. Interchangeability, the above description has generally described the components and steps of each example in terms of function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each particular application, but such implementations should not be considered beyond the scope of this application.

Those skilled in the art can clearly understand that, for the convenience and brevity of description, for the specific working process of the above-described devices and units, reference may be made to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

In the several embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the components and steps of each example are described. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each particular application, but such implementations should not be considered beyond the scope of the present invention.

The device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into Another system, or some features can be ignored, or not implemented. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, or may be electrical, mechanical or other forms of connection.

The units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solutions of the embodiments of the present application.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.

The integrated unit, if implemented in the form of a software functional unit and sold or used as an independent product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the present application are essentially or part of contributions to the prior art, or all or part of the technical solutions can be embodied in the form of software products, and the computer software products are stored in a storage medium , including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program codes .

The above are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of various equivalents within the technical scope disclosed in the present application. Modifications or substitutions shall be covered by the protection scope of this application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (10)

1. a machine learning system, is characterized in that, comprises: a processing module, used for cyclically executing the target operation until the test result associated with the data in the test data set satisfies the preset condition; the processing module includes: a labeling module, a training module and a computing module; The storage module is used to store the obtained data set and test data set; The labeling module is configured to label the data in the data set if the data in the obtained data set is not labelled; The training module is used to screen the marked data that is greater than or equal to the first preset threshold, and then perform feature extraction on the screened data, and then input the extracted features into the pre-training model to obtain a trained model. training a model; the feature is used to train the pre-training model; The computing module is configured to, after inputting the data in the obtained test data set into the trained training model, and obtaining the test result associated with the data in the test data set, perform the data associated with the data in the test data set. The test result is evaluated to evaluate whether the test result associated with the data in the test data set satisfies the preset condition. 2. The system of claim 1, wherein after judging whether the data in the data set is marked, The training module is further configured to perform feature extraction on the data in the data set if all the data in the data set are marked, and input the extracted features into the pre-training model to obtain a trained training model; the features for training the pre-training model; The computing module is further configured to, after inputting the obtained data in the test data set into the trained training model, and obtaining the test result of the data association in the test data set, correlate the data in the test data set The test results are evaluated to evaluate whether the test results associated with the data in the test data set meet the preset conditions. 3. The system of claim 1, wherein: The labeling module is configured to label the data in the data set by means of machine labeling if the data in the data set is not labelled. 4. The system of claim 3, wherein: The labeling module is further configured to visualize the data after labeling the data in the data set by means of machine labeling. 5. The system of claim 1, wherein: The training module is used for screening the data whose marked confidence is greater than or equal to the first preset threshold. 6. The system of claim 1, wherein: The computing module is used to evaluate the test results associated with the data in the test data set, so as to evaluate whether the difference between the test results associated with the data in the test data set and the preset reference results associated with the test data satisfies less than the second preset threshold. 7. A machine learning method, comprising: The target operation is performed cyclically until the test result associated with the data in the test data set satisfies the preset condition; the target operation includes: get the dataset; Determine whether the data in the data set is marked, and if the data in the data set is not marked, mark the data in the data set; Screen the marked data that meets the first preset threshold, and perform feature extraction on the screened data; input the extracted features into the pre-training model to obtain a trained training model; the features are used to Pre-trained model for training; Input the data in the acquired test data set into the trained training model, and obtain the test result associated with the data in the test data set; the data in the test data set is the marked data; Evaluate the test results associated with the data in the test data set to evaluate whether the test results associated with the data in the test data set satisfy a preset condition. 8. A machine learning device, comprising: a processing unit, configured to cyclically execute a target operation until the test result associated with the data in the test data set satisfies a preset condition; the target operation; The processing unit includes: a first acquisition unit, a judgment unit, an annotation unit, a screening unit, an extraction unit, a training unit, a second acquisition unit, and an evaluation unit; the first obtaining unit, used to obtain a data set; The judging unit is used to judge whether the data in the data set is marked; The labeling unit is configured to label the data in the data set if the data in the data set is not labelled; The screening unit is configured to screen the marked data that meets the first preset threshold; The extraction unit is used to perform feature extraction on the filtered data; The training unit is used to input the extracted features into the pre-training model to obtain a trained training model; the features are used to train the pre-training model; The second obtaining unit is configured to input the obtained data in the test data set into the trained training model, and obtain the test result associated with the data in the test data set; the data in the test data set is the marked data ; The evaluation unit is configured to evaluate the test results associated with the data in the test data set, so as to evaluate whether the test results associated with the data in the test data set satisfy a preset condition. 9. A machine learning device, comprising: an input device, an output device, a memory, and a processor coupled to the memory, wherein the input device, the output device, the processor, and the memory are connected to each other, wherein the The memory is used to store application program code, and the processor is configured to invoke the program code to execute the machine learning method of claim 7 . 10. A computer-readable storage medium, characterized in that the computer storage medium stores a computer program, the computer program comprising program instructions, the program instructions, when executed by a processor, cause the processor to execute as claimed The machine learning method described in claim 7.