CN111488806A - Multi-scale face recognition method based on parallel branch neural network - Google Patents

Abstract

The invention discloses a multi-scale face recognition method based on a parallel branch neural network, which can output the recognized face end-to-end after inputting any picture. Generate a face picture and input it into the face recognition network to prepare for recognition; step 2, input the unlabeled original data set picture into the face detection network obtained in step 1, detect the face picture and perform preprocessing; step 3, put the step 2. The obtained pictures are marked with everyone's names, and input the face recognition network for training; step 4, fix all the parameters of the training convergent face recognition network without any changes, and obtain the final face recognition network, and The face detection networks obtained in step 1 are combined and finally form an end-to-end face recognition network.

Description

A multi-scale face recognition method based on parallel branched neural network

technical field

The invention relates to the field of deep learning in the direction of artificial intelligence, and mainly relates to a method for neural network detection and multi-scale face recognition with a parallel branch structure.

Background technique

With the development of computer technology, great progress has been made in the field of computer vision, and video surveillance is becoming more and more intelligent. An important task of an intelligent video surveillance system is to detect and recognize faces in video images. The correct detection and identification of the target is the premise of video surveillance, and its effect will affect the subsequent tracking analysis and other operations.

There are many traditional face recognition methods, the most commonly used are SVM classifier and KNN clustering. SVM is a linear classifier that maximizes the classification interval. It projects the linearly inseparable data into a high-dimensional space to make it linearly separable, and then finds a classification line (surface) with the largest edge distance based on the linear function given by humans. Complete the classification of the data. However, it relies on the experience of engineers, the parameters set have a decisive impact on the final result and it is difficult to solve the multi-classification problem. KNN compares the features of the test data with the corresponding features in the training set, and finds the most similar top K data in the training set, then the category corresponding to the test data is the category with the most occurrences in the K data. However, it has a large amount of calculation, poor tolerance to data, and low accuracy.

In recent years, the rapid development of deep learning algorithms has made it comprehensively surpass traditional recognition algorithms in the field of computer vision, and the end-to-end approach is convenient for people with different knowledge backgrounds to operate. Therefore, at present, the mainstream method of deep learning neural network is used for face detection and recognition. However, most of these algorithms have the problems of too deep network, complex structure, huge amount of parameters and time-consuming. There is a contradiction between detection and recognition accuracy and computational overhead.

SUMMARY OF THE INVENTION

By improving the existing deep learning algorithm, the method uses a lightweight network to solve the common problems of complex and bloated neural network structure and huge and redundant parameters. A neural network structure with parallel branches is proposed. At the same time, different from the existing neural network face recognition algorithm, it no longer uses the characteristics of a certain layer of network as the classification basis, but uses multi-scale features to learn more abundant faces. features to improve accuracy. In addition, the image preprocessing aligns the image and grayscales it to further control the computational overhead, and obtain an ideal face recognition accuracy with a small computational cost.

The purpose of this invention is to realize through the following technical solutions:

A multi-scale face recognition method based on a parallel branched neural network, which can output the recognized face end-to-end after inputting any picture, and specifically includes the following steps:

Step 1: Train a face detection network to prepare for generating a face image and input it into a face recognition network for recognition; it is divided into the following parts:

Step 101: Annotate the images of the training data set, including completely enclosing the face with a rectangular frame and marking five points of the face: left eye, right eye, nose, left corner of mouth, and right corner of mouth;

Step 102: Use the lightweight network, namely the MobileNet network, as the face detection network, and input the labeled pictures for training;

Step 103: Fix all parameters of the converged face detection network after training to obtain a final face detection network, and prepare for inputting the detected face into the face recognition network;

Step 2: Input the unlabeled original data set image into the face detection network obtained in step 1, detect the face image and perform preprocessing;

Step 3: Label the pictures obtained in Step 2 with everyone's name, and input the face recognition network for training;

Step 4: Fix all the parameters of the converged face recognition network, without any changes, to obtain the final face recognition network, which is combined with the face detection network obtained in step 1 to form an end-to-end face recognition network. .

Further, step 2 specifically includes the following steps

Step 201: Input the original data set picture into the face detection network obtained in step 1, and cut and save the face picture after detecting it;

Step 202: Scale the cropped face pictures of different sizes to a uniform size of 112x112, and use a similar transformation method to correct and align the faces with inclined angles;

Step 203: Grayscale the aligned pictures of the same size.

Further, the face recognition network consists of two parts; the first part uses a lightweight network, namely the MobileNet network, as the basic network, and the second part adds three layers of parallel branched feature maps with inconsistent sizes behind the basic network to be used. Carry out multi-scale face recognition; the branch structure in the second part above uses ordinary convolution and hole convolution to carry out feed-forward propagation to learn face features with different focuses.

Compared with the prior art, the beneficial effects brought by the technical solution of the present invention are:

1. In the process of image preprocessing, the cropped face pictures of different sizes are scaled to a uniform size of 112×112, and the faces with inclined angles are corrected and aligned by similar transformation; this is a very simple mathematical operation. It can simplify the work of the face recognition network and make it more focused on learning the positions and internal connections of the five key points of the face, without considering the additional learning overhead due to the size and angle of the face.

2. In the process of image preprocessing, the aligned images of uniform size are grayscaled. Because face recognition, the most important thing is to learn the connection of five key points, so color can not improve the accuracy of face detection and recognition, but it will affect the calculation cost. Specifically, a general color image has three primary color components, red, green, and blue, each of which has 256 possible values, and a total of 256×256×256=16777216 colors. The grayscale image has no color, the value of each color channel is the same, only the color depth, a total of 256 color depths. Therefore, in this way, the subsequent neural network can be made to pay attention to the color depth change, rather than the specific color, which can greatly speed up the operation speed, and can also improve the subsequent network's emphasis on learning effective information and improve the accuracy.

3. This method is based on the deep learning algorithm, selects a lightweight network, avoids the problem of too deep network and too many parameters; the detection and recognition are divided into two independent stages, without interfering with each other, to ensure that the network can target specific functions. And in the face recognition network design parallel network branches to learn different features, the parallel network branch selects ordinary convolution and hole convolution to pass and recognize faces in parallel, ordinary convolution can learn the overall information; hole convolution The receptive field has been increased, and more detailed information has been learned. The combination of overall information and detailed information can improve the accuracy of face recognition and learn more abundant information; and then use multi-scale prediction to ensure that the lightweight network runs fast while solving the frequent recognition problems. The problem of poor performance.

Description of drawings

Figure 1 is a schematic diagram of a face detection network training process;

Figure 2 is a schematic diagram of a picture preprocessing process;

Figure 3 is a schematic diagram of a face recognition network training process;

Figure 4 is a schematic diagram of the complete structure of the face recognition algorithm.

Detailed ways

The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention.

The present invention provides a multi-scale face recognition method based on a parallel branch neural network, the details are as follows:

Step 1: Train a face detection network to prepare for generating a face picture and inputting it into a face recognition network for recognition. Figure 1 is a detailed explanation of step one. Specifically:

Step 101. Annotate the images of the training data set, including completely enclosing the face with a rectangular frame and marking the five point positions of the face: left eye, right eye, nose, left corner of the mouth, and right corner of the mouth. The tool is LabelImg, and the information file with the suffix .xml is generated after labeling, which corresponds to the original image one by one.

Step 102. Use a lightweight network such as the currently most popular MobileNet as a face detection network, and input the marked .xml file and the image together for training.

Step 103. The converged face detection network after training can effectively detect the face in the picture. When all parameters of all layers are fixed and no longer change, the final face detection network is obtained, in preparation for detecting the face and inputting the face recognition network.

The second step is image preprocessing. Input the unlabeled original data set image into the face detection network obtained in step 1, detect the face image and perform a series of processing. Figure 2 is a detailed explanation of step 2 image preprocessing. Specifically:

Step 201. Input the unlabeled original picture into the face detection network obtained in step 1, and cut and save the face picture after detecting it.

Step 202. Uniformly scale the face pictures of different sizes to 112x112 size, and then correct and align the faces with inclined angles by means of similar transformation. The basic principle of similarity transformation is as follows:

The original image coordinates (x, y), the transformed image coordinates (x', y'), the public representation is as follows:

Among them, s=1 is direction-preserving, and s=-1 is reverse. There are 4 degrees of freedom (1 rotation angle θ, 2 translations t _x , _ty , 1 scaling scale). In this way, the work of the face recognition network can be simplified with very simple mathematical operations, so that it can focus more on learning the positions and internal connections of the five key points of the face, without considering the additional size and angle of the face. learning cost.

Step 203. Grayscale the aligned pictures of the same size. Because face recognition, the most important thing is to learn the connection of five key points, so color can not improve the accuracy of face detection and recognition, but it will affect the calculation cost. Each pixel in the color image is determined by three components, R, G, and B, and each component has 256 value methods. At this time, one pixel has more than 16 million (256×256×256) color changes. scope. The grayscale image is a special color image with the same three components of R, G, and B. At this time, there are 256 variations of a pixel. Grayscale processing of the image before face expression recognition can reduce the amount of calculation in subsequent image processing.

Step 3: Label the pictures obtained in Step 2 with everyone's names, and input them into the face recognition network for training. Figure 3 is a detailed explanation of the face recognition network designed in step 3. The face recognition network is composed as follows:

Consists of two parts. The first part uses a lightweight network such as MobileNet as the basic network, and the second part adds multi-scale feature maps generated by different convolutional layers with 3 parallel branches behind the basic network for multi-scale face recognition. These two branches, one uses ordinary convolution, and the other uses hole convolution for feedforward propagation, aiming to learn facial features with different focuses. Ordinary convolution can learn overall information; dilated convolution increases the receptive field and learns more detailed information. The combination of overall information and detailed information can improve the accuracy of face recognition. Especially for the recognized small face, it is also enlarged to 112×112 size, which will inevitably cause information loss. After adding the atrous convolution branch, the accuracy of small face recognition is greatly improved. Both branches use downsampling operations to change the size of the feature map. The dimensions used in this example are as follows: feature maps a1, b1 are 52 × 52; feature maps a2, b2 are 26 × 26; feature maps a3, b3 are 13 × 13. A large feature map is helpful for recognizing small objects, and a small feature map is favorable for recognizing large objects. Using the method of feature map prediction, it can better recognize the face transformed from each scale. Avoiding a certain layer of feature maps only has a good effect on faces of a certain size. Finally, the name with the highest confidence in the predictions of the three scales is used as the final face recognition result.

Step 4. "Frozen" the training convergent face recognition network parameters, that is, all parameters of the network are fixed without any changes, and then the final face recognition network is obtained, which is combined with the face detection network obtained in step 1. An end-to-end face recognition network is formed.

FIG. 4 is a schematic diagram of the complete structure of the end-to-end face recognition algorithm finally proposed by the present invention. After obtaining the face detection network and the face recognition network from steps 1 and 3, the final end-to-end structure can be expressed as: (1) input the picture to be detected; (2) the face detection network detects the face and cuts it out; (3) Scale the cropped face image to 112x112 size and align the face with similarity transformation; (4) Grayscale the aligned face; (5) Input the face image into the face recognition network for multi-scale Identify and analyze; (5) output the final identification result.

The present invention is not limited to the embodiments described above. The above description of the specific embodiments is intended to describe and illustrate the technical solutions of the present invention, and the above-mentioned specific embodiments are only illustrative and not restrictive. Without departing from the spirit of the present invention and the protection scope of the claims, those of ordinary skill in the art can also make many specific transformations under the inspiration of the present invention, which all fall within the protection scope of the present invention.

Claims (4)

1. A multi-scale face recognition method based on a parallel branch neural network can output a recognized face end to end after any picture is input, and is characterized by specifically comprising the following steps of:

training a face detection network to prepare for inputting a face picture into the face recognition network for recognition; the method is divided into the following parts:

step 101: marking the picture of the training data set, wherein the method comprises the following steps of completely surrounding the face by using a rectangular frame and marking five point positions of the face: left eye, right eye, nose, left mouth corner, right mouth corner;

step 102: taking a lightweight network as a face detection network, and inputting the marked pictures for training;

step 103: fixing all parameters of the face detection network after the training convergence to obtain a final face detection network, and preparing for detecting a face and inputting the face into a face recognition network;

inputting the unmarked original data set picture into the face detection network obtained in the step one, detecting the face picture and preprocessing the face picture;

step three, marking the names of all people on the pictures obtained in the step two, and inputting the pictures into a face recognition network for training;

and step four, fixing all parameters of the face recognition network with convergence training without any change to obtain a final face recognition network, and combining the final face recognition network with the face detection network obtained in the step one to finally form an end-to-end face recognition network.

2. The method for multi-scale face recognition based on the parallel branch neural network as claimed in claim 1, wherein the second step specifically comprises the following steps

Step 201: inputting the original data set picture into the face detection network obtained in the first step, and cutting and storing the detected face picture;

step 202, zooming the cut human face pictures with different sizes to be in the uniform 112 × 112 size, and correcting and aligning the human face with inclined angle in a similar transformation mode;

step 203: and graying the aligned pictures with the same size.

3. The method for multi-scale face recognition based on the parallel branched neural network as claimed in claim 1, wherein the face recognition network is composed of two parts; the first part uses a lightweight network as a basic network, and the second part adds three layers of parallel branched feature maps with different sizes behind the basic network for carrying out multi-scale face recognition; and the branch structure of the second part is subjected to feedforward propagation in a common convolution and a cavity convolution mode respectively so as to learn the face features of different emphasis points.

4. The method according to claim 1, wherein the lightweight network is a MobileNet network.

Images