CN104657744A - Multi-classifier training method and classifying method based on non-deterministic active learning - Google Patents

Abstract

The invention discloses a multi-classifier training method and classification method based on non-deterministic active learning. The method is: 1) select or initialize a multi-classifier; for each sample in the unmarked sample set, use the multi-classifier to calculate the overall information amount Info of the sample; the overall information amount is: model change information amount and The sum of model tuning information; 2) cluster the unlabeled sample set to obtain J subclasses; 3) select some unlabeled samples with the smallest overall information Info value from each subclass; Select K samples from among them for labeling and add them to the labeled sample set L; 4) Use the updated labeled set L as training data to retrain the multi-classifier; 5) Iteratively execute steps 1) to 4) to set the number of times; The resulting multi-classifier is then used to classify the unlabeled set. The invention realizes the comprehensive evaluation of the amount of sample information, thereby obtaining a highly efficient and intelligent multi-classifier.

Description

A multi-classifier training method and classification method based on non-deterministic active learning

technical field

The invention relates to a multi-classifier training method and classification method based on non-deterministic active learning, belonging to the technical field of software engineering.

Background technique

Data classification has always been a research hotspot, such as patent ZL 201010166225.6 "an adaptive cascade classifier training method based on online learning", patent ZL 200910076428.3 "a cross-domain text sentiment classifier training method and classification method" , Patent ZL 200810094208.9 "Document Classifier Generation Method and System".

In the classification problem of massive data, "active learning" (reference: McCallum and K. Nigam, "EmployingEM in pool-based active learning for text classification," in Proc. of the 15th International Conference on Machine Learning, 1998, pp. 350–358.) is a machine learning method that efficiently utilizes expert labeling. Its main idea is: the machine actively and targetedly selects the most informative samples and sends them to experts for labeling (query to experts), so that Obtain the largest classification performance improvement under the limited amount of sample labels, such as referring to authorized patents: ZL 201210050383 "Multi-class image classification method based on active learning and semi-supervised learning"; ZL200810082814.9 "Used to make the boost classifier suitable for method for new samples".

The advantages of active learning are particularly obvious in the application scenarios where the cost of sample labeling is high and the number is limited, while the number of unlabeled samples is large and easy to obtain. Selective sampling strategies are a key part of active learning. Existing selective sampling strategies roughly include the following types—(1) Uncertainty-based: Submit the samples whose current model is most uncertain about how to classify them to experts for labeling (References: D.Lewis and W.Gale, "A sequential algorithm for training text classifiers," In Proc.of the ACM SIGIR Conference on Research and Development in Information Retrieval, 1994, pp.3–12.); (2) Combination-based decision-making: starting from different models, using voting mode, and submit the most divergent samples to expert annotation (references: H.S.Seung, M.Opper, and H.Sompolinsky, “Query by committee,” In Proc. of the ACM Workshop on Computational Learning Theory, 1992, pp.287–294 ); (3) Based on the minimization of expected error: starting from the decision theory, estimate the expected error of the model after the unlabeled sample is marked, and finally select the sample that can obtain the minimum expected error and submit it to the expert for labeling (references: Y.Guo and R . Greiner, “Optimistic active learning using mutual information,” In Proc. of International Joint Conference on Artificial Intelligence, 2007, pp.823–829.).

The notation of the document of the present invention is as follows: the sample is represented by the feature vector ^x ; Represented by U and L; the classification model uses the posterior probability said, among them Denotes the parameters of the N classification model corresponding to the labeled set L.

In traditional active learning methods, the number of categories N can be known in advance through empirical analysis or prior knowledge, so it can be regarded as a constant. This type of method is called "Determinate Active Learning" (D-AL for short). According to the number of categories (N=2 or N>2), the corresponding classification models can be divided into two types—binary and multi-class. The binary classification model divides samples into one of two categories, which is a basic classification model that has been widely studied and applied; the multi-classification model divides samples into one of multiple categories, which is a generalized form of the binary classification model. There are two ways to build a multi-classification model:

(1) A straightforward approach is to transform the multi-classification model into multiple binary classification models. In the training phase, for each category or every pair of two categories, a corresponding binary classification model is constructed. In the prediction stage, multiple trained binary classification models are combined into a total classification model by voting or fusion. For example, for each category c∈C ^N , using the label y _c ={0,1} to indicate whether the sample x belongs to this category, logistic regression can be used for the construction of a binary classification model:

$P({\mathrm{the\; y}}_c=1|x;{\mathrm{\θ}}_{L_c}^2)={\mathrm{\θ}}_{{\mathrm{\θ}}_{L_c}^2}\left(x\right)=\frac{1}{1+\exp (-{\left({\mathrm{\θ}}_{L_c}^2\right)}^{T}x)}.$ Formula 1)

The final prediction is labeled as the category with the largest posterior probability:

${\mathrm{the\; y}}^{*}=\arg \underset{c\∈C^N}{\max }P({\mathrm{the\; y}}_c=1|x;{\mathrm{\θ}}_{L_c}^2).$ Formula (2)

(2) Another processing method considers all categories comprehensively and models them in a unified manner. For example, given a sample x, softmax regression uses an N-dimensional vector to estimate the probability of labeling y taking each value from 1 to N, thereby simultaneously modeling N categories in a unified process:

$\left[\begin{array}{c}P(\mathrm{the\; y}=1|x;{\mathrm{\θ}}_L^N)\\ P(\mathrm{the\; y}=2|x;{\mathrm{\θ}}_L^N)\\ ...\\ P(\mathrm{the\; y}=N|x;{\mathrm{\θ}}_L^N)\end{array}\right]=h_{{\mathrm{\θ}}_L^N}\left(x\right)=\frac{1}{{\mathrm{\Σ}}_{i=1}^N\exp \left({\left({\mathrm{\θ}}_L^N\right)}_i^{T}x\right)}\left[\begin{array}{c}\exp \left({\left({\mathrm{\θ}}_L^N\right)}_1^{T}x\right)\\ \exp \left({\left({\mathrm{\θ}}_L^N\right)}_2^{T}x\right)\\ ...\\ \exp \left({\left({\mathrm{\θ}}_L^N\right)}_N^{T}x\right)\end{array}\right].$ Formula (3)

Since the probability distribution of samples belonging to various types is modeled uniformly, the results are directly comparable, and the largest element in the vector corresponds to the final prediction label, so this processing method is more suitable for multi-classification model construction.

In order to optimize the classification model, the traditional multi-classification method based on deterministic active learning selects the most informative samples and submits them to experts for labeling, so as to realize the model update. The selection method of the most informative sample is: respectively calculate the expected error of each sample in the unlabeled set after labeling, and select the sample that minimizes the expected error as the most informative sample. The formula is as follows:

$x_{\mathrm{D.}-\mathrm{AL}}^{*}=\underset{x\∈u}{\arg \min }\underset{\widetilde{\mathrm{the\; y}}\∈C^N}{\mathrm{\Σ}}P\left(\widetilde{\mathrm{the\; y}}\right|x;{\mathrm{\θ}}_L^N)f(x,\widetilde{\mathrm{the\; y}};{\mathrm{\θ}}_L^N).$ Formula (4)

in,

$\begin{array}{c}f(x,\widetilde{\mathrm{the\; y}};{\mathrm{\θ}}_L^N)=\underset{x_u\∈u-x}{\mathrm{\Σ}}h\left({\mathrm{the\; y}}_u\right|x_u;{\mathrm{\θ}}_{L+(x,\widetilde{\mathrm{the\; y}})}^N)\\ =\underset{x_u\∈u-x}{\mathrm{\Σ}}(-\underset{{\widetilde{\mathrm{the\; y}}}_u\∈C^N}{\mathrm{\Σ}}P\left({\widetilde{\mathrm{the\; y}}}_u\right|x_u;{\mathrm{\θ}}_{L+(x,\widetilde{\mathrm{the\; y}})}^N)\&Center\; Dot;\log P\left({\widetilde{\mathrm{the\; y}}}_u\right|x_u;{\mathrm{\θ}}_{L+(x,\widetilde{\mathrm{the\; y}})}^N))\end{array}.$ Formula (5)

The formula expresses that given the existing model parameters and the newly annotated sample In the case of , the information entropy of other unlabeled samples x _u ∈ Ux Sum; Indicates the sample The new labeled set after being labeled.

The most informative sample selected according to the formula Annotated by experts in the form of human-computer interaction, after the annotation is completed, the sample is removed from the unlabeled set and added to the labeled set.

For the multi-classification method based on deterministic active learning, the number of categories N is known in advance, so the form of the N classification model can be directly determined accordingly. The remaining task is to select and label the most informative samples through deterministic active learning, so that in the existing Continuously optimize model parameters under the model framework However, in many practical applications, the number of categories cannot be accurately known in advance; even in some applications, the number of categories will change over time. In the above cases, the number of categories N itself is the variable that needs to be solved, active learning not only needs to optimize the existing model parameters At the same time, the number of categories N (that is, the form of the classification model) needs to be updated according to the sample distribution.

For the sake of clarity, in the document of the present invention, the model parameters under the existing N classification model The optimization of is called "model tuning", and the model reconstruction caused by updating the number of categories N is called "model change". The traditional multi-classification method based on deterministic active learning only focuses on model tuning and ignores model changes, so it is only suitable for application scenarios where the number of categories is known; while the number of categories is uncertain, the multi-classification method based on deterministic active learning The method is limited to the evaluation of the amount of sample information under the existing N classification model, but it cannot accurately describe and fit the real distribution of the sample data, so that it cannot effectively improve the classification performance.

Contents of the invention

The purpose of the present invention is to provide a multi-classifier training method and classification method based on non-deterministic active learning. On the one hand, it evaluates the ability of the sample to optimize model parameters under the existing model framework, and on the other hand, it introduces new Classes can be used to evaluate the possibility of triggering model reconstruction. By comprehensively considering the two factors of model optimization and model change, the comprehensive evaluation of sample information can be realized, so as to obtain an efficient and intelligent massive data classification model.

1. The provided multi-classification method based on non-deterministic active learning measures the amount of information of samples from two aspects of model change and model tuning, on the one hand, evaluates the ability of samples to optimize model parameters under the existing model framework, On the other hand, the possibility of triggering model reconstruction by labeling samples as new categories is evaluated. By combining two factors, a comprehensive evaluation of the amount of sample information is achieved. According to this evaluation standard, the sample with the largest amount of information is selected for labeling, which can ensure limited The optimization of the classification effect under the amount of sample labeling, so as to obtain an efficient and intelligent massive data classification model.

2. The provided sample information calculation method not only considers the probability of samples being labeled as existing categories, but also considers the probability of samples being labeled as new categories, forming a unified and comprehensive calculation method for sample information.

3. The provided clustering-based sample batch selection method, on the basis of clustering the unmarked samples, selects the most informative sample set in batches, which avoids information redundancy while ensuring the amount of sample information.

Technical scheme of the present invention is:

A multi-classifier training method based on non-deterministic active learning, the steps of which are:

1) Select or initialize a multi-classifier; for each sample in the unlabeled sample set, use the multi-classifier to calculate the overall information amount Info of the sample; the overall information amount is: model change information amount and model tuning information the sum of the quantities;

2) Clustering the unlabeled sample set to obtain J subclasses;

3) Select a number of unlabeled samples with the smallest overall information Info value from each subcategory; then select K samples from the selected unlabeled samples for labeling and add them to the labeled sample set L;

4) Retrain the multi-classifier with the updated labeled sample set L as training data.

A multi-classifier classification method based on non-deterministic active learning, the steps of which are:

1) Select or initialize a multi-classifier; for each sample in the unlabeled sample set, use the multi-classifier to calculate the overall information amount Info of the sample; the overall information amount is: model change information amount and model tuning information the sum of the quantities;

2) Clustering the unlabeled sample set to obtain J subclasses;

3) Select a number of unlabeled samples with the smallest overall information Info value from each subcategory; then select K samples from the selected samples for labeling and add them to the labeled sample set L;

4) Retrain the multi-classifier with the updated labeled set L as training data;

5) Iteratively execute steps 1) to 4) for a set number of times; then use the finally obtained multi-classifier to classify the unlabeled set.

Further, the information amount of the model change is: select a sample a from the unlabeled sample set and set the labeled category of the sample as a new category; then use the multi-classifier to calculate the unlabeled sample a after removing the sample a For the information entropy of the new category in the sample set, use the information entropy as the model change information amount of the sample a; the calculation method of the model tuning information amount is: select a sample a from the unlabeled sample set and add the sample a The labeled category of is set as a category in the multi-classifier; then the updated multi-classifier is used to calculate the weighted sum of the information entropy of each existing category in the unlabeled sample set that removes the sample a, as the sample The amount of model tuning information for a.

Further, the method for calculating the amount of model change information is as follows: first construct an N+1 multi-classifier based on the labeled sample set L with N categories of training data; then for the unlabeled sample set after removing the sample a For each sample x, the probability that it does not belong to any of the existing N categories is defined as the probability that the sample x belongs to the N+1th category; Mark the information entropy of the new category in the sample set as the model change information amount of the sample a.

Further, the method for calculating the amount of model change information is as follows: first construct a binary classifier based on the labeled sample set L with N categories of training data, wherein the existing N categories are merged into one category A, and the current There are other categories other than N categories classified into another category B; then, for each sample x in the unlabeled sample set after removing the sample a, the probability that it does not belong to any of the existing N categories is defined as The probability that the sample x belongs to category B; then use the multi-classifier to calculate the information entropy of the unlabeled sample set about the new category after removing the sample a, as the model change information amount of the sample a.

Further, the method for calculating the amount of model change information is as follows: first construct a one-element classifier based on the labeled sample set L with N categories of training data; then for each sample in the unlabeled sample set after removing the sample a x, the probability that it does not belong to any of the existing N categories is defined as the probability that the sample x is an outlier; then use the multi-classifier to calculate the unlabeled sample set after removing the sample a about the new category The information entropy of is used as the model change information amount of the sample a.

Main contents of the present invention include:

For classification problems with an uncertain number of categories, the value of the number of categories N is the number of different labels of samples in the current labeled set. With the expansion of the labeled set, the number of categories N will be adjusted accordingly. Figure 1 is an example of the construction process of a classification model with an uncertain number of categories: in Figure 1(1), the initial labeled set only contains two labeled samples A and B, which belong to category 1 and category 2 respectively, so the corresponding classification model It is a binary classification model; in Figure 1(2), sample C is marked as category 1 and added to the labeled set, since no new label is added, the classification model is still a binary classification model; in Figure 1(3), sample D is Labeled as category 3 and added to the labeled set, due to the emergence of new labels (category 3), the classification model was changed to a three-category model.

Figure 1 also shows that the two factors of model tuning and model change are equally important for classification model optimization and cannot be neglected. In Figure 1(1), if only starting from the existing binary classification model, sample C has more information than sample D (because sample C is closer to the classification surface, so it has greater uncertainty); however, in fact , sample D is more meaningful for the optimization of the classification model (because the annotation of sample D not only helps to update the model parameters, but also introduces new annotation information, and then rebuilds the model into a three-category model that better fits the real distribution of the data).

1. Measurement of sample information

The multi-classification method based on non-deterministic active learning provided by the present invention organically integrates the amount of sample information into a unified framework, and realizes the comprehensive and effective measurement of the amount of sample information.

This method starts from information theory and is based on the following analysis: (1) When a sample is labeled as a new category and added to the labeled set, the sample introduces new information that has not been modeled before into the existing model, thereby increasing the existing model. For the uncertainty of the global estimate of an unlabeled sample, from the perspective of information theory, this sample will increase the overall information entropy of the unlabeled set; (2) when a sample is labeled as a known category and added to the labeled set, the The sample provides new constraints for the existing model to better fit the data distribution. From the perspective of information theory, the sample tends to reduce the overall information entropy of the unlabeled set.

Based on the above analysis, the multi-classification method based on non-deterministic active learning measures the amount of sample information from two aspects: model change and model tuning:

(1) The "model change information amount" of a sample is defined as: assuming that the sample is labeled as a new category, use the multi-classifier to calculate the information entropy of the unlabeled set that removes the sample about the new category; the formula is as follows:

${\mathrm{Info}}_{\mathrm{up}\mathrm{grad}e}(x;{\mathrm{\φ}}_L,{\mathrm{\θ}}_L^N)=P(\mathrm{the\; y}\∉C^N|x;{\mathrm{\φ}}_L)f(x,N+1;{\mathrm{\θ}}_L^{N+1}).$ Formula (6)

Info _upgrade is positively correlated with the amount of sample information. formula, is the probability that the sample x is marked as a new category (that is, does not belong to any of the existing N categories) under the existing model, and φ _L is the parameter of the probability model.

(2) The "model tuning information amount" of a sample is defined as: Assuming that the sample is marked as one of the existing N categories, the multi-classifier is used to calculate the unlabeled set of the sample except for each existing category The weighted sum of information entropy; the formula is as follows:

$\begin{array}{c}{\mathrm{Info}}_{\mathrm{update}}(x;{\mathrm{\φ}}_L,{\mathrm{\θ}}_L^N)=P(\mathrm{the\; y}\∈C^N|x;{\mathrm{\φ}}_L)\underset{\widetilde{\mathrm{the\; y}}\∈C^N}{\mathrm{\Σ}}P\left(\widetilde{\mathrm{the\; y}}\right|x;{\mathrm{\θ}}_L^N)f(x,\widetilde{\mathrm{the\; y}};{\mathrm{\θ}}_L^N)\\ =(1-P(\mathrm{the\; y}\∉C^N|x;{\mathrm{\φ}}_L))\underset{\widetilde{\mathrm{the\; y}}\∈C^N}{\mathrm{\Σ}}P\left(\widetilde{\mathrm{the\; y}}\right|x;{\mathrm{\θ}}_L^N)f(x,\widetilde{\mathrm{the\; y}};{\mathrm{\θ}}_L^N)\end{array}.$ Formula (7)

Info _update is negatively correlated with the amount of sample information.

Based on the multi-classification method of non-deterministic active learning, the amount of model change information and model tuning information of samples are organically combined into a comprehensive measure; according to their respective characteristics, focus on monitoring samples with significantly high Info _upgrade values and significantly low Info _update values ; is formulated as follows (but not limited to this representation):

$x_{\mathrm{IMC}-\mathrm{AL}}^{*}=\underset{x\∈u}{\arg \min }\mathrm{Info}(x;{\mathrm{\φ}}_L,{\mathrm{\θ}}_L^N).$ Formula (8)

in,

$\begin{array}{c}\mathrm{Info}(x;{\mathrm{\φ}}_L,{\mathrm{\θ}}_L^N)=\log [{\mathrm{Info}}_{\mathrm{update}}(x;{\mathrm{\φ}}_L,{\mathrm{\θ}}_L^N)=\underset{x\∈u}{\min }{\mathrm{Info}}_{\mathrm{update}}(x;{\mathrm{\φ}}_L,{\mathrm{\θ}}_L^N)+\mathrm{\σ}]\\ +\mathrm{\λ}\log [-({\mathrm{Info}}_{\mathrm{up}\mathrm{grad}e}(x;{\mathrm{\φ}}_L,{\mathrm{\θ}}_L^N)-\underset{x\∈u}{\max }{\mathrm{Info}}_{\mathrm{up}\mathrm{grad}e}(x;{\mathrm{\φ}}_L,{\mathrm{\θ}}_L^N))+\mathrm{\σ}]\end{array}.$ Formula (9)

Info is the total information amount of the sample. In the formula, λ is a parameter to adjust the relative weight between Info _upgrade and Info _update , and σ is a very small constant (such as e ^-10 ) artificially added in order to avoid -∞ in the calculation result, that is, according to formula (8) Select several samples with the largest amount of information for labeling.

2. Calculation of probability of new category of samples

In the formula, the probability that sample x does not belong to any of the existing N classes There are many ways to calculate . Given the existing labeled sets L and N classification models and their model parameters The methods for calculating the above probability include but are not limited to the following three methods:

(1) Construct an N+1 classification model based on the labeled set L, so that the probability that the sample x does not belong to any of the existing N categories is defined as the probability that the sample x belongs to the N+1th category. Formulated as follows:

$P(\mathrm{the\; y}\∉C^N|x;{\mathrm{\φ}}_L)=P(\mathrm{the\; y}=N+1|x;{\mathrm{\θ}}_L^{N+1}).$ Formula (10)

(2) Construct a binary classification model based on the labeled set L, merge the existing N categories into one category "+1", and classify other categories other than the existing N categories as "-1", so that the sample x The probability of not belonging to any of the existing N classes is defined as the probability that sample x belongs to class "-1". Formulated as follows:

$P(\mathrm{the\; y}\∉C^N|x;{\mathrm{\φ}}_L)=P(\mathrm{the\; y}=-1|x;{\mathrm{\θ}}_L^2).$ Formula (11)

(3) Construct a one-element classification model based on the labeled set L. This classification model aims to train a classifier from a labeled set containing only one category to identify similar samples or detect outlier samples. Common methods include single-class support vector machines ( OC-SVM). This method merges the existing N categories into one category, and converts the output value of OC-SVM into a probability form through the sigmoid function, so that the probability that the sample x does not belong to any of the existing N categories is defined as the sample x is separated from Probability of cluster points. Formulated as follows:

$P(\mathrm{the\; y}\∉C^N|x;{\mathrm{\φ}}_L)=\frac{1}{1+\exp (-{\mathrm{Our\; put}}_{\mathrm{OC}-\mathrm{SVM}}(\mathrm{the\; y}=\mathrm{outlier}|x;L))}.$ Formula (12).

3. Batch selection mechanism of samples based on clustering

In practical applications, in order to ensure the execution efficiency of the method, the most informative samples selected each time are not one but a batch (such as K samples). If only the K samples with the smallest Info values are selected according to the formula, redundant information will inevitably be introduced, resulting in a decrease in classification efficiency.

The present invention provides an improved sample batch selection method—a cluster-based sample batch selection method: (1) cluster the samples in the unmarked set into J (J≥K) classes, and the clustering methods include but are not limited to K- means, K-medoids, spectral clustering, etc.; (2) in each category, select the most informative samples according to the formula, and obtain a sample set with the number of samples J; (3) in the above sample sets, select the most informative samples according to the formula K samples with information (minimum Info value). J is a number greater than or equal to K and is used to deal with information redundancy.

After selecting the K most informative samples according to the method of the present invention: (1) manually label the selected K samples; (2) remove the marked K samples from the unmarked set and add to the marked set; ( 3) Based on the new labeled set, train a new classification model according to formula (3) to obtain classification results.

Compared with prior art, positive effect of the present invention is:

The multi-classification method based on non-deterministic active learning provided by the present invention measures the amount of information of samples from two aspects of model modification and model optimization, on the one hand, evaluates the ability of samples to optimize model parameters under the existing model framework, On the other hand, the possibility of introducing a new category into the sample to trigger model reconstruction is evaluated. By combining two factors, the comprehensive and comprehensive evaluation of the sample information is realized, so as to obtain the most optimized massive data for the efficient use of limited labeled samples. Classification results provide an intelligent solution.

Description of drawings

Figure 1 is an example of the construction process of a classification model with an uncertain number of categories; where,

(1) The initial labeled set only contains two labeled samples, A and B,

(2) Sample C is labeled as category 1 and added to the labeled set,

(3) Sample D is marked as category 3 and added to the marked set;

Fig. 2 is a flow chart of the multi-classification method based on non-deterministic active learning provided by the present invention.

Detailed ways

The principles and features of the present invention are described below in conjunction with the accompanying drawings, and the examples given are only used to explain the present invention, and are not intended to limit the scope of the present invention.

Example based non-deterministic active learning method for multi-classification

The multi-classification method based on non-deterministic active learning provided by the invention realizes the gradual optimization of the classification model through a cyclic iterative process.

Assuming that each round of loop iteration needs to label K samples, the following process is executed inside each round of loop iteration:

After the method is executed, if the number of loop iterations is M, the total number of samples marked by experts through human-computer interaction is K×M.

Taking image classification as an example, the image sample is represented by a feature vector x composed of color histogram, wavelet texture, etc.; the initial set of marked images includes 5 categories of cars, ships, airplanes, tigers, and elephants, which are represented by numbers 1 to 5 , the image annotation is represented by y={1,2,...,5}; the unlabeled image constitutes the unlabeled set U, and the labeled image constitutes the labeled set L; the classification model uses the posterior probability P(y|x; θ _L ) said.

In order to improve the performance of the classification model, it is necessary to select some unlabeled images for labeling, and update the existing model with new labeling information. Assuming that each model update requires new labeling K=5 image samples, iteratively execute the following process:

1) Calculate the overall information amount Info of the unlabeled image sample (that is, the sum of the model change information amount and the model tuning information amount of the sample);

2) Cluster unlabeled images into J=10 subcategories; select an image sample with the smallest Info value from each subcategory, and obtain 10 image samples in total; among the selected 10 image samples, select the image sample with the smallest Info value 5 image samples;

3) Label the selected 5 image samples and add them to the labeled set;

4) Retraining the image classification model with the new labeled set L as training data;

5) Use the updated classification model to classify the unlabeled set, and then obtain improved image classification results.

The multi-classification method based on the non-deterministic active learning provided by the present invention can obtain the optimal classification effect with a limited number of sample labels under the condition that the number of categories is uncertain.

Claims (10)

1. A multi-classifier training method based on non-deterministic active learning comprises the following steps:

1) selecting or initializing a multi-classifier; for each sample in the unlabeled sample set, calculating the total information amount Info of the sample by using the multi-classifier; the total information amount is: the sum of the model change information quantity and the model tuning information quantity;

2) clustering the unlabeled sample set to obtain J subclasses;

3) selecting a plurality of unlabeled samples with the minimum total information quantity Info value from each subclass; selecting K samples from the selected unlabeled samples, labeling the K samples, and adding the K samples into the labeled sample set L;

4) and using the updated labeled sample set L as training data to retrain the multi-classifier.

2. The method of claim 1, wherein the amount of model change information is: selecting a sample a from the unmarked sample set and setting the marked type of the sample as a new type; then, calculating the information entropy of the unlabeled sample set about the new class after the sample a is removed by using the multi-classifier, and taking the information entropy as the model change information quantity of the sample a; the calculation method of the model tuning information quantity comprises the following steps: selecting a sample a from the unlabeled sample set and setting the labeled class of the sample as one class of the multiple classifiers; and then, calculating the information entropy weighted sum of the unlabeled sample set without the sample a about each existing class by using the updated multi-classifier, and using the information entropy weighted sum as the model tuning information quantity of the sample a.

3. The method of claim 2, wherein the amount of model change information is calculated by: firstly, constructing an N +1 multi-classifier according to a labeled sample set L with N classes of training data; then, for each sample x in the unlabeled sample set after the sample a is removed, defining the probability that the sample x does not belong to any one of the existing N classes as the probability that the sample x belongs to the N +1 th class; and then, calculating the information entropy of the unlabeled sample set with the sample a removed with respect to the new class by using the multi-classifier, and taking the information entropy as the model change information quantity of the sample a.

4. The method of claim 2, wherein the amount of model change information is calculated by: firstly, constructing a two-classifier according to a labeled sample set L with N classes of training data, wherein the existing N classes are merged into a class A, and other classes except the existing N classes are classified into another class B; then, for each sample x in the unlabeled sample set after the sample a is removed, defining the probability that the sample x does not belong to any one of the existing N classes as the probability that the sample x belongs to the class B; and then, calculating the information entropy of the unlabeled sample set with the sample a removed with respect to the new class by using the multi-classifier, and taking the information entropy as the model change information quantity of the sample a.

5. The method of claim 2, wherein the amount of model change information is calculated by: firstly, constructing a unitary classifier according to a labeled sample set L with N types of training data; then, for each sample x in the unlabeled sample set after the sample a is removed, defining the probability that the sample x does not belong to any one of the existing N classes as the probability that the sample x is an outlier; and then, calculating the information entropy of the unlabeled sample set with the sample a removed with respect to the new class by using the multi-classifier, and taking the information entropy as the model change information quantity of the sample a.

6. A multi-classifier classification method based on non-deterministic active learning comprises the following steps:

1) selecting or initializing a multi-classifier; for each sample in the unlabeled sample set, calculating the total information amount Info of the sample by using the multi-classifier; the total information amount is: the sum of the model change information quantity and the model tuning information quantity;

2) clustering the unlabeled sample set to obtain J subclasses;

3) selecting a plurality of unlabeled samples with the minimum total information quantity Info value from each subclass; selecting K samples from the selected samples, labeling the K samples, and adding the K samples into a labeled sample set L;

4) retraining the multi-classifier by using the updated labeled set L as training data;

5) iteratively executing the steps 1) to 4) for a set number of times; and then classifying the unlabeled set by using the finally obtained multi-classifier.

7. The method of claim 6, wherein the amount of model change information is: selecting a sample a from the unmarked sample set and setting the marked type of the sample as a new type; then, calculating the information entropy of the unlabeled sample set about the new class after the sample a is removed by using the multi-classifier, and taking the information entropy as the model change information quantity of the sample a; the calculation method of the model tuning information quantity comprises the following steps: selecting a sample a from the unlabeled sample set and setting the labeled class of the sample as one class of the multiple classifiers; and then, calculating the information entropy weighted sum of the unlabeled sample set without the sample a about each existing class by using the updated multi-classifier, and using the information entropy weighted sum as the model tuning information quantity of the sample a.

8. The method of claim 7, wherein the amount of model change information is calculated by: firstly, constructing an N +1 multi-classifier according to a labeled sample set L with N classes of training data; then, for each sample x in the unlabeled sample set after the sample a is removed, defining the probability that the sample x does not belong to any one of the existing N classes as the probability that the sample x belongs to the N +1 th class; and then, calculating the information entropy of the unlabeled sample set with the sample a removed with respect to the new class by using the multi-classifier, and taking the information entropy as the model change information quantity of the sample a.

9. The method of claim 7, wherein the amount of model change information is calculated by: firstly, constructing a two-classifier according to a labeled sample set L with N classes of training data, wherein the existing N classes are merged into a class A, and other classes except the existing N classes are classified into another class B; then, for each sample x in the unlabeled sample set after the sample a is removed, defining the probability that the sample x does not belong to any one of the existing N classes as the probability that the sample x belongs to the class B; and then, calculating the information entropy of the unlabeled sample set with the sample a removed with respect to the new class by using the multi-classifier, and taking the information entropy as the model change information quantity of the sample a.

10. The method of claim 7, wherein the amount of model change information is calculated by: firstly, constructing a unitary classifier according to a labeled sample set L with N types of training data; then, for each sample x in the unlabeled sample set after the sample a is removed, defining the probability that the sample x does not belong to any one of the existing N classes as the probability that the sample x is an outlier; and then, calculating the information entropy of the unlabeled sample set with the sample a removed with respect to the new class by using the multi-classifier, and taking the information entropy as the model change information quantity of the sample a.