CN113298184A - Sample extraction and expansion method and storage medium for small sample image recognition - Google Patents

Abstract

A sample extraction and expansion method and a storage medium for small sample image recognition belong to the technical field of image processing. In order to solve the problem of generating wrong samples that may be caused in the way of generating new samples in the process of image recognition with small samples. The invention first proposes a sample extraction method based on feature reconstruction to solve the problem of missing features of small sample data sets, and realizes the extraction of a typical small sample data set from large sample data sets from the perspective of data features. In this method, the centroid of the large sample data is used as the standard of extraction measurement, so that the typical small sample data set extracted has more comprehensive features and the effect is more stable. The invention also proposes a sample expansion method based on deformation information, which utilizes the deformation information between the same and different clusters in the optimal division to realize the expansion of the extracted typical small sample data set into a new large sample data set. It is mainly used for sample extraction and expansion of small sample image recognition.

Description

Sample extraction, expansion method and storage medium for small sample image recognition

technical field

The present invention relates to an image sample extraction and expansion method and a storage medium. It belongs to the technical field of image processing.

Background technique

Neural networks often require large amounts of data to complete effective training. In the low data area, the training effect and generalization ability of the network will be poor.

In order to ensure the recognition effect of the network, the small-sample learning method based on data expansion is basically used at present. Most of the data expansion is to use the idea of generative confrontation to generate data or directly use the difference information between similar data sets to achieve data enhancement. However, this method does not consider the completeness of the features of the small sample data set before expansion, which will lead to the lack of important features in the expanded data. There is also no question of whether the use of difference information is reasonable. If irrelevant difference information is used on small sample data to generate new samples, wrong samples will be generated. Reusing the newly generated wrong samples for training will affect the training effect of the network, not only can not improve the recognition effect, but may even cause the recognition effect to deteriorate.

SUMMARY OF THE INVENTION

The present invention aims to solve the problem of generating wrong samples that may be caused in the way of generating new samples in the process of small sample image recognition, thereby affecting the training effect.

A sample extraction method for small sample image recognition, comprising the following steps:

S1. Calculate the center support point C _k of each category of the image large sample data set;

S2. Divide the same sample data into a dynamic number of clusters from the perspective of sample characteristics, and first calculate the centroid of each cluster according to each division situation;

According to the centroid of each cluster, the new center point of the category under this kind of division is calculated; and the sum of the error of the same cluster and the centroid error is calculated as the total error of the category in this kind of division; the case with the smallest error is selected as the The optimal way of dividing the category;

S3 , extracting samples uniformly and quantitatively from each optimally divided cluster according to its sample distribution, as a small sample data set for feature reconstruction.

Further, the process of calculating the center support point C _k of each category of the image large sample data set includes the following steps:

For the image large sample data set, calculate the average vector of all feature vectors in each category in the large sample data set, and use it as the central support point C _k of the category:

为嵌入函数。S _k represents the sample set of the K-th category, |S _k | represents the number of samples in the K-th category of the sample set; _xi represents the sample data belonging to the S _k sample set, and _yi is the label corresponding to the sample _xi ;

is an embedded function.

Further, step S2 includes the following steps:

2.1. Cluster the eigenvectors of the samples in each category in the large sample data set into a dynamic number of clusters, with m clusters;

2.2. Calculate the centroid C′ _{k_m_n} of each new cluster, and denote all samples under the cluster as X∈S _{k_m_} ; C′ _{k_m_n} indicates the centroid of the nth cluster after dividing the data of category k into m clusters; S _{k_m_n} represents the set of all samples in the nth cluster after dividing the samples in category k into m clusters;

2.3. Calculate the new centroid C′ _{k_m} of the category in this case of division through the centroid C′ _{k_m_n} of each new cluster;

Among them, C′ _{k_m} represents the new center support point of the category formed by the centroid of each cluster after the samples of the Kth category are divided into m clusters;

2.4. Calculate the sum of the same-cluster error L _sce and the centroid error L _ce as the total error L _s of the category in this division; select the case with the smallest total error as the optimal score for the category of data.

Further, the total error L _s =L _sve +L _ce ; wherein

The co-cluster error is the sum of the distances between all samples in each cluster and the centroid of the cluster:

The centroid error is the distance between the new centroid of the category constructed by the centroids of each cluster and the centroid of the category in the large sample:

L _ce =C' _{k_m} -C _k .

A storage medium stores at least one instruction, the at least one instruction is loaded and executed by a processor to implement a sample extraction method for small sample image recognition.

The sample expansion method for small-sample image recognition uses a sample extraction method for small-sample image recognition to determine a small-sample data set, and expands based on the small-sample data set. The sample expansion includes a training phase and a generation phase; training stage:

A typical small-sample data set is divided into the form of sample pairs ( _x'm , _x'n ) according to the optimal division of each class in the sample extraction method for small-sample image recognition, where _x'm and x ′ _n belong to the sample data of different clusters in the same category; that is, x′ _m ∈ Q _{k_i_s} , x′ _n ∈ Q _{k_i_t} , s≠t, where Q _{k_i_s} and Q _{k_i_t} respectively represent the optimal division of the data of category k into i clusters After, the sample data of the s-th cluster and the t-th cluster;

The process of sample expansion is completed by using a sample expansion model, and the sample expansion model is a network structure including an inference model, a generative model, a discriminant model and a classifier;

The inference model encodes the sample pairs (x′ _m , x′ _n ) of the same different clusters, and learns the deformation information Z=E(x′ _m , x′ _n between the samples x′ _m and x′ _n of the same different clusters );

表示为

The generative model uses the deformation information Z in the latent space and the input samples x′ _n to generate

Expressed as

通过对抗性训练使得网络能够更加准确的重构出该样本；The discriminative model is trained to distinguish between real sample pairs (x _c , x′ _m ) and reconstructed sample pairs

Through adversarial training, the network can reconstruct the sample more accurately;

进行分类；The classifier is used to reconstruct the sample

sort;

Build stage:

Take the sample pair (x′ _m , x′ _n ) of the same type and different clusters as the input of the model, and then take the deformation information Z and the sample x′ _u ∈ Q _{k_i_t} as the input of the generative model, and finally generate a new sample for this category; More new samples for the class are generated by constantly changing the sample pair ( _x'm , _x'n ) input to the inference model or changing the sample _x'u input to the generative model.

Further, in the process of changing the sample pair (x' _m , x' _n ) input by the inference model, it is necessary to ensure that the sample pair (x' _m , x' _n ) is the sample pair trained in the training process.

Further, the process of changing the input samples x' _u of the generative model must ensure that x' _u and the samples x' _n input by the generative model are of the same type and the same cluster.

Further, the total loss of the sample augmentation model in the training phase is as follows:

L=L _mse +L _D +L _cls

Among them, L _mse is the generation loss, L _D is the discriminative loss, and L _cls is the classification loss.

A storage medium storing at least one instruction, the at least one instruction is loaded and executed by a processor to implement a sample expansion method for small sample image recognition.

Beneficial effects:

The invention first proposes a sample extraction method based on feature reconstruction to solve the problem of missing features of small sample data sets, and realizes the extraction of a typical small sample data set from large sample data sets from the perspective of data features. In this method, the centroid of the large sample data is used as the standard of extraction measurement, so that the extracted typical small sample data set has more comprehensive characteristics and the effect is more stable; at the same time, it also provides a typical small sample data with relatively complete characteristics for data expansion. set. The invention further proposes a sample expansion method based on deformation information, which realizes the expansion of the extracted typical small sample data set into a new large sample data set by using the deformation information between the data of the same kind and different clusters in the optimal division. This method extracts and uses deformation information more reasonably, making the generated data more accurate. Since the invention ensures that the generated image data is more accurate, the recognition effect of the network can be effectively improved when the image recognition network is further trained.

Description of drawings

Figure 1 is an overview of sample extraction and expansion methods;

Figure 2 is an adversarial data augmentation model based on feature reconstruction and deformation information;

Figure 3 is a graph showing the relationship between the amount of data and the accuracy of the model;

Figure 4 shows the centroid of each cluster in the optimal partition of the sample with the label of 5 in the MNIST dataset as an example;

Figure 5 shows new samples generated on the MNIST numeric dataset (label 3) and the EMNIST character dataset (label 46) using the network structure in the SECIDI method.

Detailed ways

Specific implementation one:

This embodiment is a sample extraction method for small sample image recognition, which is essentially a sampling method based on feature reconstruction. The main idea is to optimally divide large sample data through an unsupervised fuzzy clustering method. Then reconstruct the typical small sample dataset. Specifically, the following steps are included:

First, the center support point C _k of each category of the image large sample data set is calculated.

Secondly, from the perspective of sample characteristics, the same sample data is divided into a dynamic number of clusters, and the centroids of each cluster are first calculated according to each division.

Then, according to the centroid of each cluster, the new center point of the category under this kind of division is calculated. And by calculating the sum of the same cluster error and centroid error as the total error of the category in this kind of division. The case with the smallest error is selected as the optimal division method for this category.

Finally, the typical small sample data is reconstructed from the optimally divided clusters for each class.

The specific reconstruction process includes the following steps:

1. Calculation of category mean (centroid):

For the image large sample data set, calculate the average vector of all feature vectors in each category in the large sample data set, and use it as the central support point C _k of the category. The category is divided according to the label of the image, that is, the category refers to the classification according to the label, such as the data set of handwritten digit recognition, all the labels are 0 is a class, the label is 1 is a class, and so on.

为嵌入函数，在一些实施例中嵌入函数指卷积神经网络的卷积层。S _k represents the sample set of the K-th category, |S _k | represents the number of samples in the K-th category of the sample set; _xi represents the sample data belonging to the S _k sample set, and _yi is the label corresponding to the sample _xi ;

is an embedding function, which in some embodiments refers to a convolutional layer of a convolutional neural network.

2. Optimal division of similar data:

2.1. Cluster the feature vectors of the samples in each category in the large sample dataset into a dynamic number of clusters (assuming m clusters).

2.2. Calculate the centroid C′ _{k_m_n} of each new cluster (representing the centroid of the nth cluster after dividing the data of category k into m clusters), and denote all samples under the cluster as X∈S _{k_m_n} (S _{k_m_n} represents After dividing the samples in class k into m clusters, the set of all samples in the nth cluster).

2.3. Calculate the new centroid C′ _{k_m} of the category in this case of division through the centroid C′ _{k_m_n} of each new cluster.

Among them, C′ _{k_m} represents the new center support point of the category formed by the centroid of each cluster after the samples of the Kth category are divided into m clusters.

2.4. Calculate the sum of the same-cluster error L _sce and the centroid error L _ce as the total error L _s of the category under this kind of division. The case with the smallest total error is selected as the optimal score for this category of data.

The co-cluster error is the sum of the distances between all samples in each cluster and the centroid of the cluster:

The centroid error refers to the distance between the new centroid of the category constructed by the centroids of each cluster and the centroid of the category in the large sample:

L _ce =C' _{k_m} -C _k (4)

The total error is

L _s =L _sce +L _ce (5)

3. Reconstruct the small sample data set:

Samples are uniformly and quantitatively extracted from the optimally divided clusters according to their sample distribution, as a small sample data set for feature reconstruction.

Specific implementation two:

This embodiment is a storage medium, and the storage medium stores at least one instruction, and the at least one instruction is loaded and executed by a processor to implement a sample extraction method for small sample image recognition.

Specific implementation three:

This embodiment is a sample expansion method for small sample image recognition, which is essentially a sample expansion method based on intra-class deformation information. The main idea is to learn the deformation information between the same type of data, and then use the deformation information on other samples of the same type to generate and expand into a new data set of this type. In order to better learn the deformation information between similar data, the sample extraction method of feature reconstruction is used, and the specific embodiment 1 provides a typical small-sample data set with relatively complete features for data expansion, and each category in the typical small-sample data set is are divided into multi-clusters with characteristic differences, and then expanded based on the small sample data set determined in the specific embodiment 1. Figure 1 is an overview of the sample extraction and expansion method based on feature reconstruction and deformation information, in which (a) represents the large sample data set, (b) represents the division of the large sample data set by the same-cluster error and centroid error, where The dotted blank point represents the centroid of each cluster, the solid blank point represents the centroid of the category, (c) represents the optimal division of the large sample data set, (d) represents the typical small sample data set extracted by the feature reconstruction method, (e) represents the arrow represents the deformation information between each cluster of the small sample data set, (f) represents the new large sample data set expanded based on the deformation information, and the two-color dots represent the newly generated samples.

Specifically, the augmentation of samples consists of two stages: the training stage and the generation stage.

Training phase:

生成模型：利用潜在空间内的变形信息Z和输入的样本x′_n来重构

表示为

判别模型：对真实样本对(x_c，x′_m)和重构样本对

进行判别。分类器：对重构样本

进行分类。A typical small-sample data set is divided into the form of sample pairs (x' _m , x' _n ) according to the optimal division of each class in the sample extraction method for small-sample image recognition, where x' _m and x ' _n belong to the sample data of different clusters in the same category. That is, x′ _m ∈ Q _{k_i_s} , x′ _n ∈ Q _{k_i_t} , s≠t, where Q _{k_i_s} and Q _{k_i_t} respectively represent the sample data of the sth cluster and the tth cluster after the data of category k is optimally divided into clusters i . This ensures that the deformation information between the samples of the s-th cluster and the t-th cluster can be learned under the condition of reasonable division. The advantage of this is that the learned deformation information must be relevant to the class and can be applied to the class. The sample expansion method and core is to train a network structure composed of an inference model, a generative model, a discriminant model and a classifier, as shown in Figure 5, the inference model: sample pairs (x′ _m , x′ _n ) for different clusters of the same type Encoding, the deformation information Z is formed in the latent space, and the generative model reconstructs the sample through the deformation information Z and the sample x′ _n

Generative model: use the deformation information Z in the latent space and the input samples x′ _n to reconstruct

Expressed as

Discriminant model: for the real sample pair (x _c , x′ _m ) and the reconstructed sample pair

make a judgment. Classifier: For reconstructed samples

sort.

The inference model learns the deformation information between samples _x'm and _x'n of the same different clusters. The deformation information is learned by the auto-encoder: use the auto-encoder to reduce the dimension of x' _m and x' _n , retain its key information as much as possible, and then use the retained key information and one of the samples x' _n to The training network generates another sample x′ _m , which is also the process of training generation so that the information retained after dimensionality reduction is deformation information. In other words, the deformation information refers to the additional information required to convert from x' _n to x' _m (the additional information is the low-dimensional vector compressed by the encoder), which exists in the latent space and is expressed as Z = E ( x′ _m , x′ _n ).

表示为

生成模型即对抗网络中的生成网络。The generative model uses the deformation information Z in the latent space and the input samples x′ _n to generate

Expressed as

Generative models are generative networks in adversarial networks.

为生成损失，

表示新生成样本

与真实样本x′_m间的均方误差；in,

To generate a loss,

Represents a newly generated sample

mean squared error with the real sample x'_m;

通过对抗性训练使得网络能够更加准确的重构出该样本。The discriminative model is trained to distinguish between real sample pairs (x _c , x′ _m ) and reconstructed sample pairs

Through adversarial training, the network can reconstruct the sample more accurately.

Among them, L _D is the discriminant loss, D(·,·) is the discriminator, and G(·,·) is the generator.

进行分类。The classifier is used to reconstruct the sample

sort.

Among them, L _cls is the classification loss;

Total loss:

L=L _mse +L _D +L _cls (9)

Build stage:

In the present invention, the sample pair (x' _m , x' _n ) of the same type and different clusters is still used as the input of the model, and then the deformation information Z and the sample x' _u ∈ Q _{k_i_t} (the same cluster as x' _n ) are used as the input of the generation model. input, and finally generate a new sample for that class. By constantly changing the sample pair (x' _m , x' _n ) input to the inference model (but it should be guaranteed to be the sample pair trained during training) or changing the sample x' _u input to the generative model to generate more information for the class There are many new samples (but ensure that the samples x′ _n input by the generation model are of the same type and the same cluster), which can ensure the maximum use of the learned deformation information. The purpose of adversarial training in the training process is to enable the network to generate new samples from existing samples. As long as this sample looks different from the input sample, it is enough to become a new sample of the category.

Figure 2 shows an adversarial data augmentation model based on feature reconstruction and deformation information.

The pseudo code is as follows:

Specific implementation four:

This embodiment is a storage medium, where at least one instruction is stored, and the at least one instruction is loaded and executed by a processor to implement a sample expansion method for small sample image recognition.

Example

The present invention conducts three experiments on two standard benchmark data sets (MNIST and EMNIST), which are the quantification experiment of large sample data and small sample data, the extraction experiment of typical small sample data set, and the expansion of typical small sample data set. experiment. To evaluate the SEFI method and the SECIDI method.

For the MNIST data set, there are 10 categories, the training set has 60,000 pieces of data, and the test set has 10,000 pieces of data. The present invention selects the first 2,000 pieces of data in the training set as a large sample data set. 30% (600 pieces of data in total) extracted from it are used as small sample data. In the small sample data set, the sample size of each category is about 60.

For the balanced dataset within the EMNIST dataset, there are a total of 47 categories (of which there are 10 numeric categories and 37 character categories), with 112,800 pieces of data in the training set and 18,800 pieces of data in the test set. In order to compare with other experiments, the present invention only selects categories 42-47 (6 categories in total) as the character recognition task. Similarly, the present invention selects 200 pieces of data for each category (1200 pieces in total) as a large sample data set.

During the experiment, the present invention uses the extracted data as a small sample data set, and the remaining unextracted data is used as a Rest data set, which is also used for testing.

Using Bootstrapping sampling method, 10% (200 samples)-90% (1800 samples) of sample data were extracted from the large sample data set. The extracted data is trained using the same CNNs model structure. And use the same test set to test, get the relationship between the amount of data and the accuracy of the model, and show the best, average and worst cases of the model accuracy under the same amount of data, as shown in Figure 3.

Analysis of results:

1. From Figure 3, it can be clearly seen that as the amount of data decreases, the accuracy of the model decreases. And when the sample drawn is less than 30% of the large sample, the accuracy of the model drops more seriously.

2. When the same amount of small sample data is extracted from the large sample data set, the models trained by different samples have obvious differences during testing. And as the amount of extraction decreases, the difference increases significantly. It also illustrates the importance of the present invention to provide a typical small sample data set for the model.

3. Through quantitative experiments on large sample data and small sample data, the present invention determines 30% of the large sample data as the number of small sample data sets (ie, each category has fifty or sixty samples). Because when the present invention uses the Bootstrapping method for sampling, when 30% of the large sample data is extracted, there is an effect close to that of large sample data training. If the number is further reduced, the optimal situation will be difficult to achieve the training effect of large sample data.

The present invention divides the data of each category of the large sample data set optimally, and in the process of dividing, it starts to be divided into two clusters and increases successively. When L _n+1 >αL _n (wherein L _n indicates that it will be divided into n clusters The total loss when α is a hyperparameter, the present invention sets α=0.95) in the experiment process, and the present invention determines the optimal number of clusters for this category as n. After determining the optimal number of divisions, each category of data is divided. The division results of the MNIST dataset are shown in Table 1 below.

Table 1: The optimal division of the MNIST dataset:

Secondly, in the optimal division of each category, samples are uniformly and quantitatively extracted to form a typical small sample data set. The remaining unextracted data is used as the Rest dataset. The model is trained using the extracted typical small sample data set, and then the model is tested with the test set and the Rest data set respectively.

Finally, the present invention compares four methods, including large sample data set, Bootstrapping sampling method, uniform sampling method, and SEFR sampling method (SEFR is the sampling method of the present invention, which is tested with the test set and the corresponding Rest data set respectively). 50 experiments were carried out respectively, and the best, average and worst cases were summarized in Table 2 below:

Table 2: Experimental Results on Numerical Datasets

Analysis of results:

1. For images, the SEFR method is significantly better than the Bootstrapping sampling method and the uniform sampling method. First, the average classification accuracy of SEFR method is higher than Bootstrapping sampling method and uniform sampling method. Secondly, the floating range of the SEFR method is significantly smaller than that of the Bootstrapping sampling method, and the effect is more stable. And its worst case is significantly better than Bootstrapping sampling method. The SEFR method can avoid the extreme cases of the sample set extracted, and its worst case can also be higher than the average level of the Bootstrapping sampling method.

2. Using the SEFR method makes the average accuracy drop by 4.5% compared to the average accuracy of large sample data training under the condition that the data volume is reduced by 70%. Because the present invention uses CNNs to learn the features of the samples, from the perspective of features, the small sample data set extracted by the SEFR method can cover most of the features of the large sample data set.

3. When using the SEFR extraction method and testing with the Rest data set, the optimal situation of the typical small sample data set can replace 94.5% of the original large sample data.

4. In the experiments of the SEFR method, the accuracy of multiple experiments can reach more than 90%, and it can even be close to the average accuracy of the model trained on a large sample data set.

The data of the same category is divided into several clusters. Through the visualization method, the present invention takes the sample of label 5 in MNIST as an example, which is divided into four clusters. The centroid of each cluster is displayed, as shown in Figure 4. Figure 4 takes the sample with the label of 5 in the MNIST dataset as an example to visualize the centroid of each cluster in its optimally divided cluster. The shape of the centroid is different, and the deformation information that the present invention seeks is the conversion information between any two clusters (the combination of Fig. 4(a) and Fig. 4(b) or Fig. 4(c) and Fig. 4(d), etc.). and apply this difference information (between Fig. 4(a) and Fig. 4(b)) to other samples in one of the clusters (Fig. 4(b)), and we can also apply this difference information (Fig. 4(c) ) and Fig. 4(d) are used on other samples of one of the clusters (Fig. 4(d)).

The present invention is a typical small sample data set extracted from the MNIST digital data set and the EMNIST character data set. The present invention has carried out multiple data expansion experiments based on the SECIDI method (the expansion method of the present invention), and each time is at the original data volume. On the basis, the data is doubled, and the new large-sample data set after the expansion is compared with the typical small-sample data before the expansion, and the expansion methods such as DAGAN are used, as shown in Tables 3 and 4 below.

Table 3: MNIST SECIDI Classification

Table 4: EMNIST SECIDI Classification

Analysis of results:

The classification accuracies of SECIDI on the MNIST numeric dataset and the EMNIST character dataset are shown in Tables 3 and 4, showing the average accuracies across all experiments. Table 3 shows that after the SECIDI method is used to expand the typical small-sample data set in MNIST, the average accuracy is improved by more than 2.5% compared with that before the expansion. When a typical small-sample dataset has 100 samples per class, the average precision after augmentation is close to that of the original large-sample dataset. The experimental results show that the generalization of the model trained by the expanded large sample data is better, indicating that the deformation information can be used correctly to generate new data, and the newly generated data is different from the original data, thus improving the training of the network. Effect.

Table 4 shows the results after augmentation with the SECIDI method on a typical few-shot dataset in EMNIST, which is significantly better than the results on the typical few-shot dataset before augmentation and the DAGAN method. In the two sets of experiments, the SECIDI method improved by about 1.8% compared with the DAGAN method. The invention divides the optimal cluster for the typical small sample data set, so that the network can learn the intra-class deformation information more easily and correctly. And when discriminating a newly generated sample, the present invention provides it with the centroid of the corresponding cluster, in order to make the newly generated sample more conform to the characteristics and distribution of the cluster.

Figure 5. New samples generated on the MNIST numeric dataset (labeled 3) and EMNIST character dataset (labeled 46) using the network structure in the SECIDI method. Figures 5(a) and 5(b) correspond to MNIST Numeric dataset and EMNIST character dataset.

Visualization experiment: The lower left pictures in Figure 5(a) and Figure 5(b) are newly generated samples. Taking Figure 5(b) as an example, the deformation information between the upper left and upper right samples and the lower right sample are used to generate new samples of this category. The objective of the present invention is first to ensure that the new samples generated must belong to this category, and the new samples generated must be different from the provided samples, thus ensuring the availability of the newly generated samples. The expanded large sample data set improves the generalization of the network.

The present invention can also have other various embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art can make various corresponding changes and deformations according to the present invention, but these corresponding changes and deformations are all It should belong to the protection scope of the appended claims of the present invention.

Claims (10)

1. a sample extraction method for small sample image recognition, is characterized in that, comprises the following steps: S1. Calculate the center support point C _k of each category of the image large sample data set; S2. Divide the same sample data into a dynamic number of clusters from the perspective of sample characteristics, and first calculate the centroid of each cluster according to each division situation; According to the centroid of each cluster, the new center point of the category under this kind of division is calculated; and the sum of the error of the same cluster and the centroid error is calculated as the total error of the category in this kind of division; the case with the smallest error is selected as the The optimal way of dividing the category; S3 , extracting samples uniformly and quantitatively from each optimally divided cluster according to its sample distribution, as a small sample data set for feature reconstruction. 2. a kind of sample extraction method for small sample image recognition according to claim 1, is characterized in that, the process that calculates each category center support point C _k of image large sample data set comprises the following steps: For the image large sample data set, calculate the average vector of all feature vectors in each category in the large sample data set, and use it as the central support point C _k of the category:

S _k represents the sample set of the K-th category, |S _k | represents the number of samples in the K-th category of the sample set; _xi represents the sample data belonging to the S _k sample set, and _yi is the label corresponding to the sample _xi ;

is an embedded function. 3. a kind of sample extraction method for small sample image recognition according to claim 1 and 2, is characterized in that, step S2 comprises the following steps: 2.1. Cluster the eigenvectors of the samples in each category in the large sample data set into a dynamic number of clusters, with m clusters; 2.2. Calculate the centroid C′ _{k_m_n} of each new cluster, and denote all samples under the cluster as X∈S _{k_m_n} ; C′ _{k_m_n} indicates the centroid of the nth cluster after dividing the data of category k into m clusters; S _{k_m_n} represents the set of all samples in the nth cluster after dividing the samples in category k into m clusters; 2.3. Calculate the new centroid C′ _{k_m} of the category in this case of division through the centroid C′ _{k_m_n} of each new cluster;

Among them, C′ _{k_m} represents the new center support point of the category formed by the centroid of each cluster after the samples of the Kth category are divided into m clusters; 2.4. Calculate the sum of the same-cluster error L _sce and the centroid error L _ce as the total error L _s of the category in this division; select the case with the smallest total error as the optimal score for the category of data. 4. A sample extraction method for small sample image recognition according to claim 3, wherein the total error L _s =L _sce +L _ce ; wherein The co-cluster error is the sum of the distances between all samples in each cluster and the centroid of the cluster:

The centroid error is the distance between the new centroid of the category constructed by the centroids of each cluster and the centroid of the category in the large sample: L _ce =C' _{k_m} -C _k . 5. A sample expansion method for small sample image recognition, characterized in that a small sample data set is determined by a sample extraction method for small sample image recognition according to one of claims 1 to 4, and based on the small sample The data set is expanded, and the expansion of the sample includes the training stage and the generation stage; Training phase: A typical small-sample data set is divided into the form of sample pairs ( _x'm , _x'n ) according to the optimal division of each class in the sample extraction method for small-sample image recognition, where _x'm and x ′ _n belong to the sample data of different clusters in the same category; that is, x′ _m ∈ Q _{k_i_s} , x′ _n ∈ Q _{k_i_t} , s≠t, where Q _{k_i_s} and Q _{k_i_t} respectively represent the optimal division of the data of category k into i clusters After, the sample data of the s-th cluster and the t-th cluster; The process of sample expansion is completed by using a sample expansion model, and the sample expansion model is a network structure including an inference model, a generative model, a discriminant model and a classifier; The inference model encodes the sample pairs (x′ _m , x′ _n ) of the same different clusters, and learns the deformation information Z=E(x′ _m , x′ _n between the samples x′ _m and x′ _n of the same different clusters ); The generative model uses the deformation information Z in the latent space and the input samples x′ _n to generate

Expressed as

The discriminative model is trained to distinguish between real sample pairs (x _c , x′ _m ) and reconstructed sample pairs

Through adversarial training, the network can reconstruct the sample more accurately; The classifier is used to reconstruct the sample

sort; Build stage: Take the sample pair (x′ _m , x′ _n ) of the same type and different clusters as the input of the model, and then take the deformation information Z and the sample x′ _u ∈ Q _{k_i_t} as the input of the generative model, and finally generate a new sample for this category; More new samples for the class are generated by constantly changing the sample pair ( _x'm , _x'n ) input to the inference model or changing the sample _x'u input to the generative model. 6. The sample expansion method for small sample image recognition according to claim 5, wherein, in the process of changing the sample pair (x' _m , x' _n ) input by the inference model, it is necessary to ensure that the sample pair ( x′ _m , x′ _n ) are pairs of samples trained during training. 7. The sample expansion method for small sample image recognition according to claim 6, wherein the process of changing the input sample x' _u of the generative model is to ensure that x' _u and the sample input of the generative model are x' _n are of the same type and the same cluster. 8. The sample expansion method for small sample image recognition according to claim 7, wherein the total loss of the sample expansion model in the training phase is as follows: L=L _mse +L _D +L _cls Among them, L _mse is the generation loss, L _D is the discriminative loss, and L _cls is the classification loss. 9. A storage medium, characterized in that, the storage medium stores at least one instruction, and the at least one instruction is loaded and executed by a processor to implement a method for A sample extraction method for small sample image recognition. 10. A storage medium, wherein at least one instruction is stored in the storage medium, and the at least one instruction is loaded and executed by a processor to implement the method for small samples according to one of claims 5 to 8 A sample augmentation method for image recognition.

Images