CN110443162A - A kind of two-part training method for disguised face identification - Google Patents

Abstract

A kind of two-part training method for disguised face identification, includes the following steps: step S1, pre-processes the data set used needed for training, obtains collection Set_F、Set_S；Step S2, the first stage, by Set_FAs training set and use ArcFace loss function training network；Step S3, the full articulamentum of the last layer of network is cancelled；Step S4, second stage, by Set_SAs training set and use ArcFace loss function training network.The work of model is applicable in domain present invention utilizes a small amount of disguised face data and has gone to disguised face identification from Generic face identification, there is good recognition effect in DFW test benchmark.

Description

A two-stage training method for disguised face recognition

technical field

The invention belongs to the technical field of face recognition, and in particular relates to a two-stage training method for camouflaged face recognition.

Background technique

Face recognition technology through convolutional neural networks has achieved great success in recent years. As a prominent achievement in the research, the use of face feature vectors obtained through network mapping for face recognition has a very good effect, and is generally considered to be the most advanced method at present. With the continuous development of advanced network structures, high-quality datasets, and more sophisticated loss functions, the resulting feature vectors are more and more discriminating: the difference between the feature vectors of different people gradually increases, and vice versa for the same person The difference between the eigenvectors is decreasing.

Despite the impressive achievements in face recognition, disguised face recognition is still a challenging topic. After applying makeup, hats, masks and other objects to the face, the difficulty of identification will be greatly increased. And in the case that the recognition itself is difficult, the overall quality of the data set that deep learning relies on is also unsatisfactory, which further increases the difficulty of the subject. Compared with the high-quality results such as FaceNet, SphereFace, and ArcFace in the general face recognition field, the achievements in the field of camouflage face recognition are much less. The newer results are obtained by using the DFW camouflage face dataset MiRA-Face. It uses a two-stage training, first uses the general face recognition training method to obtain a network, and then uses the training set provided by DFW to use PCA to reduce the dimensionality of the feature vector, thereby obtaining a certain camouflage aspect. Information. The shortcomings of MiRA-Face are: (1) The first stage training adopts the method proposed by CosFace, which is not the best choice at present; (2) The information extracted by PCA is less. These two lead to the performance of the algorithm still has room for improvement.

SUMMARY OF THE INVENTION

The technical problem to be solved by the present invention is to overcome the deficiencies of the prior art, and to provide a two-stage training method for camouflaged face recognition, by using ArcFace to train the basic convolutional neural network, and then using the Joint loss function to minimize The intra-class distance of DFW training samples is quantized and the intra-class distance is enlarged, which has a good effect of camouflaged face recognition.

The present invention provides a two-stage training method for disguised face recognition, comprising the following steps:

Step S1, preprocessing the data set required for training to obtain sets Set _F and Set _S ;

Step S2, the first stage, take Set _F as the training set and use the ArcFace loss function to train the network;

Step S3, cancel the last fully connected layer of the network;

Step S4, the second stage, takes Set _S as the training set and uses the ArcFace loss function to train the network.

As a further technical solution of the present invention, the model adopted in the first stage is composed of ResNet50IR residual network, BatchNorm layer and Dropout layer, fully connected layer and an output module composed of another BatchNorm layer, and a classification module composed of a fully connected layer and Softmax classification layer and ArcFace loss function, the ResNet50IR residual network and the output module are used as the backbone network for feature extraction.

Further, the ResNet50IR residual network uses a residual unit based on the 50-layer ResNet. The residual unit is a 6-layer composite structure of BatchNorm-convolution-BatchNorm-PRelu-convolution-BatchNorm, and the output size is determined by the fifth layer volume. The step size of the product layer is determined; when the step size is 1, the output is the same size as the input; when the step size is 2, the output size is half of the input; the ResNet50IR residual network consists of an Input and 4 convolution modules, 4 Each convolution module has 3, 4, 14, and 3 residual units respectively. The first residual unit of each convolution module is responsible for reducing the dimension of the output. The parameter of the Dropout layer of the output module is 0.5, and the output of the fully connected layer is is a 512-dimensional vector, and then passes through a BatchNorm layer to obtain the final feature vector v.

Further, the step feature vector V needs to be normalized before being input to the fully connected layer, so that ||v||=1; the dimension of the weight of the fully connected layer is determined according to the number of label categories in the training set, when the number of categories is P , the dimension of the weight matrix w is D*P, and when the MS-Celeb-1M dataset is used as the training set, P is 85K, D is the length of the feature vector V, and the length is 512; set the bias b of the fully connected layer to Normalizing each column of zero and w, the output vector of the fully connected layer is w _i is the i-th column of the weight matrix W.

Further, use the ArcFace loss function to train the network, the function formula is Among them, the hyperparameters s and m are 64 and 0.5 respectively, θ _j,i is the angle between the feature vector v _i generated by the ith input and the weight vector w _j , and y _i is the correct label value corresponding to v _i .

Further, the model in the second stage is composed of a feature extraction network and a joint loss function. The feature extraction network is the backbone network of the first stage, and the joint loss function formula is

In the formula, the former part is Triplet loss, and the latter part is Pair loss; f(x _i ) is the feature vector v _i output by the feature extraction network after normalization, <f(x ₁ ), f(x ₂ ) > is the vector product of the eigenvectors, that is, the cosine value of the angle between the vectors v ₁ and v ₂ , and the parameters α and λ are both positive values.

Further, the training set of the second stage is the training set of the DFW data set, and a triplet needs to be performed before training. pairing, first select a Normal image as Then select Validation or Disguised in the same directory as Normal as Finally, select Impersonator in the same directory as Normal as the

Further, the pictures in the same directory in the DFW dataset are divided into Normal, Validation, Disguised and Impersonator. Among them, Normal, Validation and Disguised are the same person, and Impersonator and the first three are different people who are similar in appearance.

The invention utilizes a small amount of camouflaged face data to change the working applicable domain of the model from general face recognition to camouflaged face recognition, and has a good recognition effect on the DFW test benchmark.

Description of drawings

Fig. 1 is the training flow schematic diagram of the present invention;

Fig. 2 is the backbone network structure schematic diagram of the present invention;

3 is a structural diagram of a residual unit of the present invention;

4 is a comparison diagram of different loss functions in stage 2 of the present invention;

Figure 5 is an example diagram of the DFW dataset;

Figure 6 is a comparison chart of DFW test results.

Detailed ways

Please refer to FIG. 1. The overall process of this embodiment is divided into two stages. The network model adopted in stage 1 consists of the following four parts: (1) ResNet50IR residual network; (2) BatchNorm layer, Dropout layer, fully connected layer and Another output module composed of a BatchNorm layer; (3) a classification module composed of a fully connected layer and a Softmax classification layer; (4) ArcFace loss function. Among them, the two parts (1) and (2) are used as the backbone network for feature extraction, and the specific network structure and the dimension of a single output are given in Figure 2. The detailed analysis of stage 1 is as follows:

(1) ResNet50IR uses the improved residual unit shown in Figure 3 on the basis of the conventional 50-layer ResNet. The residual unit uses a 6-layer composite structure of BatchNorm-Convolution-BatchNorm-PRelu-Convolution-BatchNorm. The output size of the entire residual unit is controlled by the stride of the 5th convolutional layer. When the stride is 1, the output is the same size as the input, and when the stride is 2, the output is half the size of the input.

(2) ResNet50IR consists of 5 parts, an Input part and 4 convolution modules, the 4 convolution modules have 3, 4, 14, and 3 residual units respectively, and the first residual unit in each module Responsible for reducing the dimensionality of the output (set the stride of the second convolutional layer in this unit to 2). In Figure 2 we present the output dimension of each module when a single input dimension is [112*112*3].

(3) The parameter of the Dropout layer in the output module is set to 0.5, which means that the output of half of the random units in this layer will be set to zero, which increases the robustness of the network. The output of the fully connected layer is a 512-dimensional vector, and after a BatchNorm layer, the final feature vector v is obtained.

(4) Before inputting the feature vector v into the fully connected layer, it needs to be normalized so that ||v||=1. The dimension of the weight of the fully connected layer depends on the number of label categories in the training set. When the number of categories is PP, the W dimension of the weight matrix is D*P (D row and P column). When using the MS-Celeb-1M dataset as the training set, P is 85k, and D is the length of the feature vector v, which is 512 here. The bias b of the fully connected layer is set to zero and each column of W is normalized. Assuming that the i-th column of W is written as w _i , the i-th bit of the output vector of the fully connected layer is:

v·w _i =‖v‖·||w _i ||·cosθ=cosθ (1.1)

cos(θ) is the angle between the two vectors v and _wi .

(5) Use the loss function proposed by ArcFace to train the entire network, the formula is as follows:

The hyperparameters s and m in the formula are set to 64 and 0.5 respectively, θ _j,i refers to the angle between the feature vector v _i generated by the ith input and the weight vector w _j , and y _i represents the corresponding value of v _i . Correct label value. Relative to the often used Softmax loss function shown in formula (1.3):

Equation (1.2) is improved as follows:

The bias b of the fully connected layer is set to 0, and the feature vector v and the weight vector _wi are normalized. At this time, the vector product of v and _wi can be regarded as the cosine value of the angle between the two vectors, as shown in the formula ( 1.1). Formula (1.2) is regarded as a function of the included angle θ _j,i , and the gradient is calculated, and the following results are obtained:

The direction that makes the loss function fall the fastest is reduce while Increase, it can be considered that the feature vector v _i will be as close as possible to the weight vector representing its label category during training At the same time, stay away from the rest of the weight vectors w _j , j≠y _i that do not represent the class. In this way, the feature vectors with the same label will be gathered in the same area with training, and the feature vectors of different labels will be pulled apart by the angle, that is, the intra-class distance is reduced and the inter-class distance is increased.

use instead of This further reduces the intra-class distance of the feature vectors. consider in small when At this time, using formula (1.2) as the loss function will still maintain a large value. In order to make the loss function continue to decrease, it is necessary to make further reduced.

In addition to normalizing the feature vector v and the weight vector w _i , a hyperparameter s is also set. Using a larger s value will make the training easier and the network easier to converge. It is generally set to 64. In the case of not using s, just use replace It is often difficult for the model to converge during actual training. When s is set large, it can be found that the loss function will be larger when the classification is wrong than when s is not used, forcing it to iterate in the correct direction, while when the classification is correct, the loss function will be smaller than the original, making the training easier to converge .

The first stage of training generally uses larger face datasets, such as VGG2, MS-Celeb-1M, etc. This stage is to obtain a better feature extraction network applied to non-camouflaged faces (that is, remove the last part of the network with fully connected layers).

The network model used in stage 2 is (1) feature extraction network; (2) Joint loss function. The feature extraction network is the backbone network obtained after stage 1. The detailed analysis of stage 2 is as follows:

(1) The Joint loss function is shown in the following formula:

Obviously, formula (1.5) consists of two parts, the former part is called Triplet loss, and the latter part is called Pair loss. In the formula, f(x _i , x _i ) is used to represent the feature vector v _i output by the feature extraction network after normalization, and <f(x ₁ ), f(x ₂ )> represents the vector product of the feature vectors, That is, the cosine of the angle between the two vectors v ₁ and v ₂ . The parameters α and λ both take positive values. When using this loss function for training, it is necessary to organize the pictures into groups of three for input, that is, in the formula the two-tuple of is called a positive sample pair and needs to have the same label, while Then different labels are required, which are called negative sample pairs. Triplet loss controls that the distance between positive sample pairs is smaller than the distance between negative sample pairs, and the specific difference is controlled by the parameter α, generally α is about 0.3. The Pair loss controls the distance between pairs of positive samples and further limits the intra-class distance. Its existence avoids the situation that only the Triplet loss can control the inter-class distance but cannot reduce the intra-class distance. Figure 4 shows the comparison of the angle distribution between the positive sample pairs after training using only Triplet loss and using Joint loss. It can be found that after using Pair loss, the angle distribution between positive sample pairs is significantly better . For the parameter λ, generally take 0.3 or 0.4.

(2) The training set used in stage 2 is the training set of the DFW dataset, which needs to be tripled before training. Pairing: 1) Pick a Normal image as 2) Select Validation or Disguised in the same directory as Normal as 3) Select Impersonator in the same directory as Normal as (Note: The pictures in the same directory in the DFW dataset are divided into 4 types: Normal, Validation, Disguised and Impersonator, Normal, Validation and Disguised are the same person, Impersonator and the first three are different people with similar looks, see Figure 5 for an example)

The present invention obtains a feature extraction network for general faces through the training in stage 1, and then in stage 2, aiming at the situation that the current camouflaged face data set is generally small, the triplet pairing training is performed using the idea of the Triplet loss function, and Pair loss is used to make up for the insufficiency of Triplet loss, and the migration of the application scope of the network to disguised faces is completed. The combination of the two treatments enables the present invention to achieve a greater improvement in results.

The model of the present invention (stage 1 training on MS-Celeb-1M data set, stage 2 training on DFW training set) is tested on the camouflaged face recognition test set provided by DFW, and the FAR is between 1% and 0.1 The GARs in % cases are: 1) protocol-1: 97.98% and 60.23%; 2) protocol-2: 90.37% and 82.84%; 3) protocol-3: 90.4% and 81.18%. The following is the description of GAR and FAR:

The DFW test dataset provides a batch of two-by-two face images. Among these paired images, some of the two images in the pair are the same person, and these pairs are used as positive samples. The other picture pairs belong to different people. What measures the similarity of two images is the distance between the image feature vectors, but now there is only one distance, and it is obvious that it is impossible to judge whether the two images are the same person at that time. At present, the more common method is to add a threshold value as a threshold. When the distance is less than this threshold value, it is considered as a positive sample, otherwise it is a negative sample.

when a threshold is given. The values of TP, TN, FP, FN can be calculated:

TP: the number of positive samples correctly identified by the algorithm;

TN: the number of negative samples correctly identified by the algorithm;

FP: the number of negative samples identified as positive samples;

FN: the number of positive samples identified as negative samples

Next, the values of GAR and FAR can be obtained through these values:

Figure 6 is a comparison of the model of the present invention with other models. Overall, the performance of the present invention is better than most existing algorithms. (Note: The DFW dataset provides three different sets of positive and negative sample pairs, protocol-1, protocol-2, and protocol-3, where protocol-3 is the synthesis of the first two sets of sample pairs.)

The foregoing has shown and described the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above-mentioned specific embodiments. The above-mentioned specific embodiments and descriptions in the specification are only to further illustrate the principle of the present invention. There are also various changes and modifications which fall within the scope of the claimed invention. The claimed scope of the present invention is defined by the claims and their equivalents.

Claims (8)

1. a two-stage training method for camouflaging face recognition, is characterized in that, comprises the steps, Step S1, preprocessing the data set required for training to obtain sets Set _F and Set _S ; Step S2, the first stage, take Set _F as the training set and use the ArcFace loss function to train the network; Step S3, cancel the last fully connected layer of the network; Step S4, the second stage, takes Set _S as the training set and uses the ArcFace loss function to train the network. 2. a kind of two-stage training method for disguised face recognition according to claim 1, is characterized in that, the model that described first stage adopts is by ResNet50IR residual network, BatchNorm layer and Dropout layer, fully connected Layer and another BatchNorm layer composed of output module, fully connected layer and Softmax classification layer composed of classification module and ArcFace loss function, the ResNet50IR residual network and the output module are used as the backbone network for extracting features. 3. a kind of two-stage training method for disguised face recognition according to claim 2, is characterized in that, ResNet50IR residual network uses residual unit on the basis of 50 layers of ResNet, and residual unit is BatchNorm- Convolution-BatchNorm-PRelu-Convolution-BatchNorm 6-layer composite structure, the output size is determined by the stride of the fifth convolutional layer; when the stride is 1, the output is the same size as the input; when the stride is 2, The size of the output is half of the input; the ResNet50IR residual network consists of an Input and 4 convolution modules, and the 4 convolution modules have 3, 4, 14, and 3 residual units respectively. A residual unit is responsible for reducing the dimension of the output. The parameter of the Dropout layer of the output module is 0.5, and the output of the fully connected layer is a 512-dimensional vector. After a BatchNorm layer, the final feature vector v is obtained. 4. A two-stage training method for disguised face recognition according to claim 3, wherein the feature vector V needs to be normalized before being input to the fully connected layer, so that ||v||= 1; The dimension of the weight of the fully connected layer is determined according to the number of label categories in the training set. When the number of categories is P, the dimension of the weight matrix w is D*P, and when the MS-Celeb-1M dataset is used as the training set, P is 85K , D is the length of the feature vector V, and the length is 512; if the bias b of the fully connected layer is set to zero and each column of w is normalized, the output vector of the fully connected layer is w _i is the i-th column of the weight matrix W. 5. a kind of two-stage training method for disguised face recognition according to claim 1 and 2, is characterized in that, uses ArcFace loss function training network, and the function formula is Among them, the hyperparameters s and m are 64 and 0.5 respectively, θ _j,i is the angle between the feature vector v _i generated by the ith input and the weight vector w _j , and y _i is the correct label value corresponding to v _i . 6. A two-stage training method for disguised face recognition according to claim 1, wherein the model in the second stage is composed of a feature extraction network and a joint loss function, and the feature extraction network is the first stage The backbone network of , the Joint loss function formula is In the formula, the former part is Triplet loss, and the latter part is Pair loss; f(x _i ) is the feature vector v _i output by the feature extraction network after normalization, <f(x ₁ ), f(x ₂ ) > is the vector product of the eigenvectors, that is, the cosine value of the angle between the vectors v ₁ and v ₂ , and the parameters α and λ are both positive values. 7. a kind of two-stage training method for disguised face recognition according to claim 1, is characterized in that, the training set of described second stage is the training set of DFW data set, need to carry out ternary before training Group pairing, first select a Normal image as Then select Validation or Disguised in the same directory as Normal as Finally, select Impersonator in the same directory as Normal as the 8. a kind of two-stage training method for camouflaging face recognition according to claim 7, is characterized in that, the picture under the same catalogue in DFW data set divides Normal, Validation, Disguised and Impersonator, wherein, Normal, Validation The same person as Disguised, Impersonator and the first three are different people who are similar in appearance.