CN111582057A - Face verification method based on local receptive field - Google Patents

Abstract

The invention discloses a face verification method based on local receptive fields, and belongs to the technical field of calculation, calculation or counting. The method comprises the following steps: establishing an external data set, and performing data enhancement on samples in the data set; establishing a convolutional neural network, wherein the input of the convolutional neural network is a color picture, the output of the convolutional neural network is a characteristic vector corresponding to a face region in the picture and a prediction frame coordinate of a face position, and the characteristic vector of the corresponding region is output according to the position of the prediction frame in the picture during testing; and testing the pre-trained convolutional neural network by using the test set and finely adjusting the convolutional neural network according to the test result. According to the translation invariance of the deep neural network, the invention effectively extracts the characteristics of the face region by using one network, so that the receptive field of the characteristic vector just only contains the face, thereby effectively reducing the noise brought by background information, ensuring the accuracy of face verification, simultaneously improving the parallelism of network calculation and greatly simplifying the training process.

Description

A face verification method based on local receptive field

technical field

The invention discloses a face verification method based on a local receptive field, relates to a computer vision technology for face verification, and belongs to the technical field of calculation, calculation or counting.

Background technique

Face recognition is a very important part of computer vision technology, and its goal is to correctly identify the identity of a person in a face photo. The current mainstream method is to use a classification neural network to classify face pictures. However, the classification network needs to be designed according to a fixed category, and the identity of the person cannot be added after the training is completed, which is very inflexible in practical use. Therefore, we use face verification for face recognition, use neural network to extract features from face pictures to generate feature vectors of the faces in the pictures, and then calculate the Euclidean distance between different face feature vectors, and finally pass Set a threshold to judge whether it is the same person. In this way, if a new identity is added, it is only necessary to use the network to generate its facial features and save them, and then calculate with the newly input sample feature vector to perform identity recognition.

However, the current face verification algorithm is divided into two steps. First, the picture needs to go through a face detection network to obtain the position coordinates of the face, and then cut out the face part in the picture through the coordinates, so as to reduce the background band. noise, and then input the face image into the face verification network for subsequent calculations. That is to say, in the process of face verification, two networks need to be used to complete it, which means that a face detection network and a face verification network need to be trained separately during training, which will bring Inconvenient, at the same time, because there are two separate networks, the parallelism of the network will be reduced in actual use, which is essentially a two-stage face verification algorithm.

The convolutional neural network has translation invariance, so it only needs to adjust the convolution kernel and step size to obtain the features of the specified area of the picture, that is to say, the receptive field of these features corresponds to the specific area in the picture. Using this property of the convolutional neural network, only the features of the face area in the original image can be obtained. This application aims to propose a face verification method based on local receptive fields to improve the parallelism of the network and reduce the complexity of the training steps.

SUMMARY OF THE INVENTION

The purpose of the present invention is to address the deficiencies of the above-mentioned background technology, and to provide a face verification method based on local receptive field, which utilizes the receptive field output by the network to obtain the feature vector of the face region, and only uses a convolutional neural network. Realize face detection and verification, improve the parallelism of detection operations and verification operations while effectively reducing the noise caused by the background of the picture, and solve the technical problems of low parallelism and complex training steps in the existing two-stage face verification method.

The present invention adopts following technical scheme for realizing above-mentioned purpose of invention:

A face verification method based on local receptive field, comprising the following steps:

Step 1: Divide the public face verification data set or the data set collected by yourself into training set, verification set and test set;

Step 2, performing data enhancement on the samples in the data set, using at least one of the following methods for data enhancement: translation, zoom, rotation, flip;

Step 3, establish a face verification convolutional neural network based on local receptive field, the input of the convolutional neural network is a color image, and the output during training is the identity category of the face in the image and the predicted frame coordinates of the face position, loss The function uses softmax loss, and outputs the feature vector of the corresponding area according to the position of the predicted frame in the image during testing;

Step 4, use the test set to test the pre-trained convolutional neural network in step 3, and fine-tune the convolutional neural network according to the test results.

In the above step 1, the face verification training set adopts CASIA-WebFace, and the test set adopts the LFW data set.

In the above step 1, after scaling all the images in the dataset to the input size of the convolutional neural network, normalization is performed.

In the above step 3, the convolutional neural network for face verification based on local receptive field consists of two parts, one part is used to detect the face area in the input picture, and the other part is used to extract the feature vector of the face area. During training, the network extracts face features from different regions in the input image, and then selects the feature vectors corresponding to the face regions according to the face detection results, and classifies and trains the selected feature vectors through the fully connected layer. The identity category to which the face belongs and the predicted frame coordinates of the face position use softmax loss as the loss function, and use the Adam optimizer for training. When the accuracy rate no longer rises, save the convolutional neural network model and obtain the trained convolutional neural network. network.

In the above step 4, when testing and using the network, the fully connected layer is removed, and the test image sample is input into the network. The face feature vector directly output by the network is the identity feature of the face in this image, and then different tests are calculated by calculating. The Euclidean distance between the feature vectors of the picture samples. If the distance is less than a certain threshold, the two face pictures are considered to be pictures of the same person; otherwise, they are pictures of different people.

In the above convolutional neural network, after calculating the last convolutional layer of the network, a feature vector of N*N*K dimension is generated, where N*N dimension indicates that the picture is divided into N regions and the size of the feature vector of each region is It is 1*1*K. At the same time, in order to solve the problem that the face appears at the junction of the region so that the region cannot contain the entire face, a large convolution kernel is used and the convolution step size is smaller than the width of the convolution kernel. Accumulate the kernel so that the local images targeted by the two adjacent convolution operations overlap each other; during training, a fully connected layer needs to be added after the filtered feature vector, and the feature vector filtered each time is passed through the fully connected layer and softmax is used loss is used as a loss function to calculate the network prediction error.

In the above-mentioned convolutional neural network, while extracting the face features, it is also necessary to use the method of face detection to judge whether there is a face in the input picture, that is, to predict the position coordinates of the face in the input picture, and then according to the coordinates. Select the corresponding feature vector for the determined face area; during training, add the loss function of face detection to the loss function of face classification and train at the same time.

The present invention adopts the above-mentioned technical scheme, and has the following beneficial effects:

(1) The present invention proposes a method that can use a single deep neural network for face verification. The method uses the translation invariance of the deep convolutional neural network to extract the feature vectors of different areas of the image, and the detected face area is The local receptive field of the feature vector uses the receptive field of the feature vector to filter from the feature vectors in different areas of the image. Compared with the feature vector extraction method of detecting first and then cropping, it can effectively reduce the noise caused by background information and reduce the follow-up. Double counting in calculations.

(2) The present invention realizes the feature vector screening based on the receptive field through the mask matrix, and can realize the detection and verification of the face through a neural network. Compared with the two-stage face verification algorithm, the training process is simplified, and the Computational parallelism.

Description of drawings

FIG. 1 is a structural diagram of a neural network for face verification according to the present invention.

FIG. 2 is a schematic diagram of feature vector mapping of the present invention.

Detailed ways

The technical solutions of the invention will be described in detail below with reference to the accompanying drawings.

The present invention provides a face verification method based on local receptive field, comprising the following steps:

Build an external dataset: Build an external dataset based on the public fine-grained classification database of the research institution or the data collected by yourself. Exemplarily, the face verification training set can use CASIA-WebFace, and the test set can use the LFW data set. Each picture should contain an identity label, indicating which category the picture belongs to; each picture also needs to have face frame coordinates, indicating the position of the face in the picture. As many faces of different identities as possible should be collected, each identity containing as many samples as possible, while reducing the number of mislabeled samples in the dataset.

Data augmentation: Using deep neural networks to complete face verification tasks can easily lead to overfitting, but the number of training samples is usually much smaller than the required number of samples. Manual data augmentation can reduce overfitting. There are generally four types of data augmentation methods used to enlarge datasets: translation, scaling, rotation, and flipping.

Training model: Build a convolutional neural network. The input of the convolutional neural network is a color image. During training, the output is the identity category of the face in the image and the predicted frame coordinates of the face position. The loss function uses softmax loss. The position of the predicted frame in the image outputs the feature vector of the corresponding area; the structure of the face verification network based on the local receptive field consists of two parts, one is the detection part of the face in the figure; the other is the feature vector of the face area part; during training, the network will extract face features for different areas in the picture, and then select the feature vector corresponding to the face region in the picture according to the results of face detection; during training, the selected feature vector After adding a fully connected layer, the feature vector filtered each time passes through the fully connected layer, using softmax loss as the loss function, and using the Adam optimizer for training. When the accuracy rate no longer increases, save the convolutional neural network model and get trained. When testing and using the network, remove the fully connected layer and directly output the feature vector of the face area in the picture, and calculate the Euclidean distance between the feature vectors of different pictures. If the distance is less than a certain threshold, it is considered that the two A face picture is a picture of the same person; otherwise, it is a picture of a different person.

After calculating the last convolutional layer of the network, a feature vector of N*N*K dimension is generated, where N*N dimension means that the image is divided into N regions and the feature vector size of each region is 1*1*K, At the same time, in order to solve the problem that the face appears at the junction of the region so that the region cannot contain the entire face, a large convolution kernel and a convolution step size smaller than the width of the convolution kernel are used, so that the convolution kernel is used for sliding calculation. They can overlap each other; during training, a fully connected layer needs to be added after the filtered feature vector, and each filtered feature vector is passed through the fully connected layer and softmax loss is used as the loss function to calculate the network prediction error.

The technical solutions of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

The face verification method based on the local receptive field of the present invention includes the following three steps.

Step 1. Establish an external data set: The training set uses CASIA-WebFace, and the test set uses the LFW data set. There are a total of 10575 identities in the CASIA-WebFace dataset, and each identity has multiple photos; there are 5749 identities in the LFW dataset, with a total of 13233 images. All images are scaled to 250*250*3 size.

Step 2: Perform data enhancement: perform a translation operation on the samples in the obtained data set, and perform random translation on each image. In the process of deep neural network training, it is easy to produce overfitting, and at the same time, the face is placed in any position in the picture through random translation. Therefore, random translation is used to increase the robustness of the network.

Step 3: Establish a convolutional neural network: The deep neural network used in the present invention is shown in Figure 1, and the input neural network is a color picture with a pixel of 250*250*3, and the color picture is first obtained through a series of convolutional layers. The feature map of 13*13*896 is then divided into two branches, one of which continues to extract image features, and generates corresponding feature vectors for different areas of the picture. As shown in Figure 1, 3*3 128-dimensional images are generated. Feature vector, that is, the original image is divided into 9 fixed areas, each area corresponds to a 128-dimensional vector, if the number of generated feature vectors is increased, the corresponding areas in the original image will be more, and the size of each area will be smaller. ; Another branch is used to predict the position of the face. As shown in Figure 1, the 13*13*896 feature map is convolved to generate a prediction frame for the coordinates of the face and whether the prediction frame is a face or a background For the classification prediction, the frame with the highest face confidence is the frame where the face is located, and the pixel where the center point of the frame where the face is located is set to 1 and the rest of the pixels are set to 0 to get a 15*15 mask matrix, and then go through a The maximum pooling operation with a size of 5 and a step size of 5 turns the mask matrix into 3*3, where the position of 1 is the position of the feature vector corresponding to the face area in the 3*3*128-dimensional vector, so by masking The code matrix can pick out the feature vector corresponding to the face region. During training, it is necessary to add a fully connected layer for classification and softmax loss calculation after filtering out the feature vectors. It should be noted that only the feature vector corresponding to the face area of each image calculates the softmax loss. The loss function of the network is The sum of the softmax loss and the error in the face detection branch. After the network training is completed, the fully connected layer is removed, and the network directly outputs the feature vector of the test image. For different input samples, the Euclidean distance between the feature vectors is calculated. If it is greater than a certain threshold, it is considered that it is not the same person. Otherwise the same person.

The present invention also provides a method for solving the situation that the face position is at the boundary of the region. The mapping relationship between the nine feature vectors of the face and the corresponding region in the picture is shown in Figure 2. If the face is in the middle of a certain region, it can be compared However, if the face is at the junction of the two regions, it will have an impact on the feature extraction. The present invention adopts a large convolution kernel and a small step size in the convolution layer for extracting the feature vector to alleviate this situation, The height and width of the large convolution kernel used are 2/3 of the height and width of the feature map and rounded down. Since the number of output feature vectors is fixed, the convolution step size can be determined according to formula (1). In formula (1), feature is the size of the finally generated feature vector, which is 3 in Figure 1, input is the size of the input feature map, which is 11 in Figure 1, and the selected convolution kernel size kernel is 7 , then the stride size can be calculated to be 2.

For example, the input feature map of the last layer is 11*11, the convolution kernel uses a 7*7 step size of 2, and the output is 3*3; since the convolution kernel only slides a small step during the convolution calculation The convolutions overlap, so that the facial features located at the junction of the regions can also be better extracted.

To sum up the above, the present invention is a face verification method based on local receptive field, which utilizes the translation invariance of deep convolutional neural network to extract features from different regions in the image, effectively reducing the impact of background information on face feature extraction. At the same time, the face detection method is used to predict the area where the face is located, and the feature vector corresponding to the face area is selected for face verification. Compared with the two-stage face verification method, there is no need to use the face detection network to cut out the face part and then perform face verification, which effectively improves the parallelism of face verification, and does not need to train two networks during network training. , which greatly simplifies the training process of the network.

The above embodiments are only to illustrate the technical idea of the present invention, and cannot limit the protection scope of the present invention. Any changes made on the basis of the technical solution according to the technical idea proposed by the present invention all fall within the protection scope of the present invention.

Claims (10)

1. A human face verification method based on local receptive fields is characterized in that a neural network for performing human face verification on an input picture is trained, the neural network detects a human face region of the input picture and extracts feature vectors of each region of the input picture, the feature vectors of the human face region of the input picture are screened out from all the extracted feature vectors, the trained neural network is used for extracting the feature vectors of the human face regions of different test pictures, and when the Euclidean distance between the feature vectors of the human face regions of the two test pictures is smaller than a threshold value, the two test pictures are output as the verification result of the same human face picture.

2. The local receptive field-based face verification method according to claim 1, wherein the neural network detects a face region of the input picture by reading face prediction box information of the input picture, and when the selected region of the face prediction box is a face classification result, the selected region of the face prediction box with the highest confidence is selected as the face region of the input picture.

3. The local receptive field-based face verification method as claimed in claim 1, characterized in that a mask matrix representing the face position is used to screen out the feature vectors of the face region of the input picture from all extracted feature vectors.

4. The method of claim 1, wherein in the course of training the neural network for face verification, the feature vectors of the face regions of the input pictures are input into a full connection layer to obtain classification results.

5. The method according to claim 1, wherein during training of the neural network for face verification of the input picture, softmax loss is used as a loss function to reversely propagate the sum of the classification error of the feature vector of the face region of the input picture and the detection error of the face region of the input picture to correct the network parameters.

6. The local receptive field-based face verification method as claimed in claim 1, characterized in that the input picture face region is detected by sliding a large convolution kernel with a convolution step smaller than the convolution kernel width.

7. The method for verifying human face based on local receptive field as claimed in claim 1, wherein the input picture and the test picture are subjected to size scaling, normalization and data enhancement, and the data enhancement includes but is not limited to translation, scaling, rotation and flipping.

8. The method as claimed in claim 3, wherein the mask matrix has an element of 1 corresponding to the face position and the remaining elements are 0.

9. A computer-readable storage medium on which a computer program is stored, the program, when executed by a processor, implementing the face verification method of claim 1.

10. Terminal equipment, characterized by, includes: a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the face verification method of claim 1 when executing the program.

Images