CN117079024A - Image class increment learning algorithm integrating uncertainty estimation and increment stage discrimination - Google Patents

Abstract

The application relates to an image class increment learning algorithm integrating uncertainty estimation and increment stage discrimination, which comprises the following steps: a large-scale pre-training network (such as a CLIP network) is adopted as a characteristic extraction network of the classification model, parameters of the characteristic extraction network are frozen in the training process, a classification head is independently trained for each incremental stage, and meanwhile, the classification head trained before is reserved. And in the classification, the category with the highest confidence in the classification head with the smallest classification uncertainty is used as the classification result of the image. According to the application, the parameters of the feature extraction network are shared in each incremental stage, so that the parameter calculation amount of the model is greatly reduced. The large-scale pre-training network is used as the characteristic extraction network of the classification model, so that the classification model has good generalization in all incremental stages, has good discrimination on images of new and old categories, and greatly improves the classification accuracy of the incremental learning model.

Description

Image class incremental learning algorithm integrating uncertainty estimation and incremental stage discrimination

Technical field

The invention relates to the field of image class incremental learning. Specifically, it is an image class incremental learning algorithm that integrates classification uncertainty estimation and incremental stage discrimination.

Background technique

Humans and animals can accept and learn new knowledge in response to constant changes in the environment, and supplement and modify the original knowledge. In many classification tasks, classification algorithms also need to have the ability of humans and animals to continuously learn new knowledge and avoid forgetting old knowledge. This type of algorithm is called an incremental learning algorithm. The present invention studies the problem of incremental learning of image classes, that is, the classification model can continuously learn new categories while maintaining the discriminative power of old categories. The current mainstream image classification algorithm is to identify a limited number of images in a static environment. However, new categories will continue to appear in the actual application process, and most neural networks cannot recognize untrained images. That is, the neural network model can only recognize pictures of known categories. For untrained picture types, the neural network model will mistakenly classify the object into a known category, resulting in a decrease in recognition accuracy. Therefore, it is necessary to design an image incremental learning algorithm. , so that the model can continuously learn the knowledge of new categories while maintaining the discriminative power of old categories.

On the one hand, the incremental learning classification model must show the ability to learn new knowledge from new data and refine existing knowledge, and on the other hand, it must avoid significant interference of new knowledge on existing knowledge. How to make the model better balance between the two is a challenge faced by incremental learning algorithms. Traditional incremental learning algorithms train classification models from scratch in order to continuously acquire new knowledge every time new categories of data arrive. Some category incremental learning strategies have been proposed. The mainstream method is to store a small number of old category samples and jointly fine-tune the model parameters with the new category samples on the base class model trained from scratch. However, this type of algorithm will make the model The parameters are constantly changing, which can easily cause the model to forget the knowledge of old categories. Recently, large-scale pre-training models have made major breakthroughs in the field of deep learning. They use massive data sets for training. The powerful feature extraction capabilities and generalization capabilities of large-scale pre-training models make them well suited for incremental learning algorithms. middle.

The present invention provides an image class incremental learning algorithm that integrates uncertainty estimation and incremental stage discrimination, uses a large-scale pre-training network as the feature extraction network of the classification model, and freezes the parameters of the feature extraction network during the training process to ensure The feature extraction network generalizes to the categories at each stage in the incremental process. At the same time, in order to be able to distinguish the image categories at each stage, a separate classification head is trained for each stage. When classifying, first calculate the classification uncertainty of the image on each classification head to determine the incremental stage to which the image belongs. The stage to which the classification head with the smallest classification uncertainty belongs is regarded as the incremental stage to which the image belongs, and then select The category with the highest confidence in the classification head corresponding to this incremental stage is used as the classification result of the image.

Contents of the invention

In response to the urgent need to continuously learn the knowledge of new data while not forgetting the knowledge of old data during the incremental process, the present invention provides an image class incremental learning algorithm that integrates uncertainty estimation and incremental stage discrimination. The present invention adopts the following technical solutions:

An image class incremental learning algorithm that integrates uncertainty estimation and incremental stage discrimination, including the following steps:

1) Use a large-scale pre-trained network (such as CLIP network) as the feature extraction network of the classification model, freeze the parameters of the feature extraction network, input the data of the base class into the feature extraction network, and use the cross-entropy loss function L _CE to train the base class The classification head parameters of the stage, the number of categories of the classification head is the number of categories of base class data N ₀ ;

2) For the training of the kth incremental stage, freeze the parameters of the feature extraction network and retain the classification heads after training in the previous incremental stage. Enter the new category data into the feature extraction network and use the cross-entropy loss function L′ _CE Classification head parameters in the incremental phase of training, the number of categories of the classification head is the number of categories N _k of the incremental data in this phase;

3) In the test phase, calculate the classification uncertainty H _k (x) of the classification head of the k-th incremental stage of the image x to be classified, and assign the classification head with the smallest classification uncertainty to the incremental stage to which as the incremental stage to which this image belongs;

4) General The category with the highest classification confidence in the classification heads of the incremental stages is used as the classification result of image x

The calculation formula of the cross entropy loss function L _CE in step 1) is:

Among them, _yi represents the output of the model classification head, _gi represents the real label value, and N ₀ is the number of categories of the classification head in the base class stage.

The calculation formula of the cross entropy loss function L′ _CE in step 2) is:

Among them, _yi represents the output of the model classification head, _gi represents the real label value, and N _k is the number of categories of the classification head in the kth incremental stage.

The calculation formula for the classification uncertainty H _k (x) of image x on the k-th classification head in step 3) is:

Among them, y _ki (x) represents the i-th output of the classification head of image x in the k-th incremental stage, and N _k represents the number of categories of the classification head in the k-th incremental stage.

The incremental stage to which the classification head with the smallest classification uncertainty belongs in step 3) The calculation formula is:

Among them, n represents the number of stages in the incremental stage so far. When the value of incremental stage k is 0, it means that the incremental stage is the base class stage and is the starting point for incremental learning model training.

Classification results of image x in step 4) The calculation formula is:

in, Represents the image x in the incremental phase /> The i-th output of the classification head,/> Represents the incremental stage/> The number of categories in the category header.

The present invention has the following beneficial effects: 1) Using a large-scale pre-training network as the feature extraction network of the classification model enables the classification model to maintain good generalization in all incremental stages, and the feature extraction network has powerful feature extraction capabilities , both new and old categories of images can maintain good discrimination; 2) A separate classification head is trained for the data in each incremental stage, and the classification uncertainty is used to determine which number the image to be tested belongs to. In the incremental stage, it can determine the classification head that should be used and greatly improve the classification accuracy of the incremental learning model. 3) Since the parameters of the feature extraction network are frozen, parameters can be shared in each incremental stage, so the trainable parameters are greatly reduced.

Description of the drawings

Figure 1 is a schematic flow chart of an image class incremental learning algorithm that combines uncertainty estimation and incremental stage discrimination in an embodiment of the present invention.

Figure 2 shows the comparison results between the method in the embodiment of the present invention and other methods.

Detailed ways

The invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. The embodiments are a clear and complete description of the technical solutions in the present invention. The described embodiments are part of the embodiments of the present invention but not all of them. The specific embodiments described here are only used to explain the present invention and are not used to limit the scope of the present invention.

1) This invention uses the CIFAR-10 data set, which is a classic data set used for image recognition and classification. It contains 10 categories and a total of 60,000 color pictures of 32x32 size, of which the training set has 50,000 pictures. The test set The set has 10,000 pictures. The present invention adopts CLIP large-scale pre-training network as the feature extraction network of the classification model. CLIP is a multi-modal model based on contrastive learning. It uses a large-scale data set to train a feature extraction network with strong generalization capabilities. The present invention includes multiple stages of image classification model training. However, in order to explain the relevant technical features of the present invention more clearly, the present invention uses the incremental stages of the zeroth incremental stage, the first incremental stage and the second incremental stage. Quantitative learning model training is introduced as an example;

2) Incremental learning model training in the zeroth incremental stage: The model trained in this stage is the base class model, which is the starting point for incremental learning model training. Six categories in the CIFAR-10 data set are randomly selected as the zeroth For the base class data set in the incremental stage, the feature extraction network of the base class model is a CLIP pre-training network. The parameters of the feature extraction network are frozen during the training process, and the cross-entropy loss L _CE is used to train the classification head of the base class model:

Among them, _yi represents the output of the classification head of the base class model, _gi represents the real label value of the base class data set, and N ₀ is the number of categories of the classification head in the base class stage.

3) Training of the incremental learning model in the first incremental stage: It is category incremental learning training based on the training in the first incremental stage. Use the feature extraction network of the zeroth incremental stage as the feature extraction network of the first incremental stage, also freeze the parameters of the feature extraction network, and retain the classification head trained in the zeroth incremental stage. Randomly select the remaining two categories of data in the CIFAR-10 data set as the incremental data set in the first incremental stage, and use the cross-entropy loss L′ _CE to train the classification head of the incremental learning model in the first incremental stage. parameter:

Among them, _yi represents the output of the classification head of the incremental learning model in the first incremental stage, _gi represents the real label value of the incremental data set in the first incremental stage, and N _k is the classification head of the first incremental stage. number of categories.

4) Training of the incremental learning model in the second incremental stage: It is category incremental learning training based on the training in the first incremental stage. Use the feature extraction network in the first incremental stage as the feature extraction network in the second incremental stage, also freeze the parameters of the feature extraction network, and retain the classification after training in the zeroth incremental stage and the first incremental stage head. Select the remaining two categories of data in the CIFAR-10 data set as the incremental data set in the second incremental stage, and use the cross-entropy loss L′ _CE to train the classification head parameters of the incremental learning model in the second incremental stage. :

Among them, _yi represents the output of the classification head of the incremental learning model in the second incremental stage, _gi represents the real label value of the incremental data set in the second incremental stage, and N _k is the classification head of the second incremental stage. number of categories.

5) In the test phase, calculate the classification uncertainty H _k (x) of the classification head of the k-th incremental stage of the image x to be classified, and assign the classification head with the smallest classification uncertainty to the incremental stage to which As the incremental stage this image belongs to:

Among them, y _ki (x) represents the i-th output of the classification head of image x in the k-th incremental stage, N _k represents the number of categories of the classification head in the k-th incremental stage, and n represents the increment so far. The number of stages for the stage.

6) General The category with the highest classification confidence in the classification heads of the incremental stages is used as the classification result of image x

in, Represents the image x in the incremental phase /> The i-th output of the classification head,/> Represents the incremental stage/> The number of categories in the category header.

7) Figure 2 shows the traditional incremental learning method PASS trained from scratch (retaining a prototype for each old category and using prototype-based region expansion in the feature space to maintain the classification boundaries of the old category) and this From the recognition accuracy of the inventive method, it can be seen that the inventive method significantly improves the recognition accuracy of the model in each stage. The recognition rate in the zeroth incremental stage is increased by 2.22%, and the recognition rate in the first incremental stage is increased by 12.55%. %, the recognition rate increased by 8.74% in the second incremental stage.

It can be seen that the image-based incremental learning algorithm proposed by the present invention that integrates uncertainty estimation and incremental stage discrimination can effectively improve the incremental learning recognition accuracy and provide a basis for subsequent incremental learning algorithm research and engineering applications. Provide new ideas.

The above is only a preferred embodiment of the present invention and is not used to limit the protection scope of the present invention. Any person of ordinary skill in the relevant technical field can make equivalent changes or equivalent substitutions within the technical scope disclosed in this application. All solutions shall be covered by the protection scope of this application. The protection scope specified in the claims is the protection scope of this application.

Claims (6)

1. An image class increment learning algorithm integrating uncertainty estimation and increment stage discrimination is characterized in that the method comprises the steps of performing multi-stage increment learning training on an image classification model to identify images of new classes and old classes, obtaining a classification head corresponding to each stage after each stage training, and training each stage by using an image data set of different classes, wherein each stage comprises the following steps:

1) A large-scale pre-training network (such as a CLIP network) is used as a feature extraction network of a classification model, parameters of the feature extraction network are frozen, data of a base class is input into the feature extraction network, and a cross entropy loss function L is used _CE Training classification head parameters of the basic class stage, wherein the number of classes of the classification head is the number N of classes of the basic class data ₀ ；

2) For training in the kth incremental stage, freezing parameters of the feature extraction network, reserving classification heads after training in the previous incremental stage, inputting new class data into the feature extraction network, and using a cross entropy loss function L' _CE Training the classification head parameters of the increment stage, wherein the classification head class number is N of the class number of the increment data of the stage _k ；

3) In the test stage, the classification uncertainty H of the image x to be classified on the classification head of the kth increment stage is calculated _k (x) Incremental stage to which classification head with minimum classification uncertainty belongsAs an incremental stage to which the image belongs;

4) Will be the firstThe classification head of each increment stage has the highest classification confidence class as the classification result of image x +.>

2. The image class increment learning algorithm combining uncertainty estimation and increment stage discrimination according to claim 1, wherein the cross entropy loss function L in step 1) _CE The calculation formula of (2) is as follows:

wherein y is _i Representing the output of the model classification head g _i Representing the true tag value, N ₀ The number of classes that are the classification heads of the base class stage.

3. The image class delta learning algorithm incorporating uncertainty estimation and delta phase discrimination as set forth in claim 1, wherein the cross entropy loss function L 'in step 2)' _CE The calculation formula of (2) is as follows:

wherein y is _i Representing the output of the model classification head g _i Representing the true tag value, N _k The number of categories for the category header for the kth incremental stage.

4. The image class increment learning algorithm combining uncertainty estimation and increment stage discrimination according to claim 1, wherein in step 3) the classification uncertainty of image x on the kth classification headH _k (x) The calculation formula of (2) is as follows:

wherein y is _ki (x) An ith output on the classification head representing image x at the kth incremental stage, N _k Representing the number of categories of the classification header for the kth incremental stage.

5. The image class increment learning algorithm combining uncertainty estimation and increment stage discrimination according to claim 1, wherein the increment stage to which the classification head with the smallest classification uncertainty belongs in step 3)The calculation formula of (2) is as follows:

wherein n represents the number of stages of the increment stage up to now, and when the value of the increment stage k is 0, the increment stage is a base class stage and is the starting point of the increment learning model training.

6. The image class increment learning algorithm combining uncertainty estimation and increment stage discrimination according to claim 1, wherein the classification result of the image x in step 4) is thatThe calculation formula of (2) is as follows:

wherein,representing image x at delta stage +.>I-th output on classification head, < ->Representing delta phase +.>The number of categories of the classification header.