CN110543846A - A method of frontalizing multi-pose face images based on generative adversarial networks - Google Patents

Abstract

The invention discloses a multi-pose human face frontalization method based on a generative confrontation network. In the training stage, face pictures of various poses are first collected as a data set, and then multiple groups of frontal face images and non-frontal faces of the same person are input. For images, through the newly designed loss function, alternately train the generative network and the discriminative network until the value of the loss function converges stably. In the test stage after the training is completed, the present invention can correct the input face pictures in various postures into frontal face images. The rectified image is not only clear, but also retains the identity features of the original face, which can be used for face recognition. The invention can effectively slow down the negative impact of posture factors on face recognition, and is beneficial to the development of practical application of face recognition under unrestricted conditions.

Description

A method of frontalizing multi-pose face images based on generative adversarial networks

technical field

The present invention relates to the technical field of image processing, in particular to a method for frontalizing multi-pose face images based on generative confrontation networks.

Background technique

At present, face recognition technology has been widely used in many fields such as access control security, social networking and finance. However, in most practical scenarios, face recognition technology needs to be used efficiently in a strictly standard environment. Usually, the person to be detected needs to be in a scene with sufficient and uniform lighting, maintain a calm expression, and adjust the standard posture with the image acquisition device. However, in many practical application fields, such as suspect tracking, the above conditions are often difficult to meet, which leads to a significant decline in the performance of many face recognition technologies, making it difficult to promote face recognition technologies in these fields. Among the unfavorable factors affecting the performance of face recognition technology, the posture of the face in the photo is the most important. Handle the posture problem well, and the application of face recognition technology in an unrestricted environment will take a big step forward.

One of the methods to deal with the posture problem is to perform frontal correction on the input side face image, that is, to correct a side face image into a front face image of the same person, and then identify the identity of the person on the synthesized front face image. At present, most of the frontalization methods for multi-pose face images lack the ability to process face images with deflection angles exceeding 60°. Work is difficult. At present, the multi-pose face image frontalization method with good effect is basically based on generative adversarial network.

Compared with other multi-pose face image frontalization methods based on generative confrontation networks, the present invention adopts different network structures and loss functions. Even if a face image with a deflection angle of more than 60° is input, this model can synthesize a realistic face image and retain more identity information, which greatly improves the efficiency of subsequent face recognition work.

Contents of the invention

The purpose of the present invention is to overcome the deficiencies of the existing multi-pose face image frontalization method, and proposes a multi-pose face image frontalization method based on generative confrontation network, which solves the situation where the input face deflection angle exceeds 60° in similar methods It improves the fidelity of the synthesized face and preserves more identity information of the face in the image.

In order to achieve the above object, the technical solution provided by the present invention is: a method for frontalizing multi-pose face images based on generative confrontation network, comprising the following steps:

1) Collect face images of each pose as a training set and a test set. It must be ensured that each input face image I ^a of any pose can find a non-synthesized front face image I ^g of the same person in the data set;

2) In the training phase, input the face image I ^a of any pose in the training set into the generator G, obtain the corrected code X ₂ and the synthesized front face image I ^f , and input the non-synthesized front face image I ^g into the generator G , to get the front face code X ₃ ;

3) Input the synthesized front face image I ^f or the non-synthesized front face image I ^g into the discriminant network D, and the discriminant network D judges whether the input face image is synthesized or non-synthesized, and then inputs the synthesized front face image I ^f Or non-synthetic positive face image ^Ig input face identity feature extractor F, extract the character identity feature of face image by F;

4) Bring the discriminant result of step 3) and the extracted identity features, synthesized frontal image If, non-synthesized frontal image ^Ig , modified code X ₂ and ^frontal code X ₃ into each pre-designed loss function, alternately train the generator G and the discriminant network D until the end of training;

5) In the test phase, input a face image I ^a of any pose into the trained generator G, and a synthesized front face image I ^f can be obtained, which can be verified by directly observing the quality of the synthesized front face image ^If Effect.

In step 1), the face images of each posture used in the data set are all derived from the data set Multi_Pie; the number of pictures in the data set exceeds 750,000, including images under 20 kinds of illumination and 15 kinds of postures of 337 people; The illumination of the picture changes from dark to bright from illumination labels 01 to 20, wherein the illumination label 07 is the standard illumination condition; label all face images in the dataset as I ^a , and find the same person in the dataset for each image I ^a , For the images whose face deflection angle is 0° and the illumination number is 07, mark them as I ^g .

In step 2), the generator G is composed of a pose estimator P, an encoder En, an encoded residual network R and a decoder D; the pose estimator P uses the PnP algorithm to estimate the face pose, and then obtains The deflection angle of the face in the yaw direction is realized by the function cv2.solvePnP () of the opencv library; the encoder En is a convolutional neural network, and the encoding residual network R is a two-layer fully connected neural network, The decoder De is a deconvolutional neural network;

The process of the generator G synthesizing pictures is as follows: input the face image I ^a of any pose to the generator G, the encoder En converts it into the initial code X ₁ , and the coded residual network R estimates the code by the initial code X ₁ The residual R(X ₁ ), the attitude estimator P obtains the precise deflection angle γ of the face in the input image in the yaw direction, and the deflection angle γ is input into the function Y to obtain the weight Y(γ) of the coding residual, and the initial coding X ₁ and the coded residual are fused to obtain the modified code X ₂ , where X ₂ =X ₁ +Y(γ)×R(X ₁ ), the modified code X ₂ is input to the decoder De, and the front face image I is generated by deconvolution ^f .

In step 3), the discriminant network D is a binary classifier based on a convolutional neural network, which is used to determine whether the input image is from the generator G or the original image data; the face identity feature extractor F Using the open source Light-CNN-29, Light-CNN29 is a lightweight convolutional neural network with a depth of 29 layers and 12 million parameters.

In step 4), the goal of the loss function is to minimize the difference between the synthesized ^frontal image If and the non-synthesized frontal image ^Ig , so that the synthesized ^frontal image If can retain more input The identity information of the face image; the loss function used in step 4) includes a self-created encoding loss function in addition to the pixel loss function, identity loss function, symmetric loss function and adversarial loss function commonly used in the same type of method. The first goal of the encoding loss function is to make the encoding of the input face image ^Ia and the non-synthesized frontal image ^Ig respectively through the encoder En closer, because the current face pose correction method is in the input face deflection The smaller the angle, the better the effect, so the closer the encoding of the input face image is to the encoding of the same face at 0° deflection angle, the better the effect of the synthesized frontal face; therefore, the first part of the encoding loss function formula as follows:

In the formula, N is the dimension of each encoding, En is the encoder, En(I ^a ) ⁱ is the value of the i-th dimension of the initial encoding En(I ^a ); R is the encoding residual network, R(En(I ^a )) ⁱ is the value of the i-th dimension of the coding residual R(En(I ^a )); En(I ^a ) ⁱ +R(En(I ^a )) ⁱ is equal to the value of the i-th dimension of the correction code; Is the value of the i-th dimension of the frontal encoding X ₃ obtained by the non-synthesized frontal image through the encoder; the first part of the encoding loss function refers to the Manhattan distance between the correction encoding and the frontal encoding;

The second goal of the encoding loss function is to differentiate the corrected encodings between different characters, and build a fully connected layer C behind the corrected encoded En(I ^a )+R(En(I ^a )), fully connected The number of neurons in layer C is equal to the number M of characters in the training set, and the second part of the encoding loss function uses the cross-entropy loss function:

In the formula, M is the number of characters in the training set; y _i is the value of the i-th dimension of the one-hot vector y, and the vector y indicates which character in the training set the input face image I ^a belongs to. If the image I ^a belongs to For the jth person, then the value of the jth dimension of the vector y is 1, the value of the remaining dimensions is 0, and the dimension of y is M; En(I ^a )+R(En(I ^a )) is a modified code; C( En(I ^a )+R(En(I ^a ))) _i is the i-th feature vector C(En(I ^a )+R(En(I ^a ))) obtained through the fully connected layer C after the correction code the value of the dimension;

Therefore, the complete encoding loss function is:

L _code = L _code1 + _{λL code2}

In the formula, λ is a constant with a value of 0.1, representing the weight.

In step 4), alternately training the generator G and the discriminant network D can optimize and improve each other in the confrontation; in the initial stage, the face image generated by the generator G is blurred, and the discriminant network D can easily judge the input image The source of the source, thereby encouraging the generator G to generate a clearer image and improve the quality of the generator G; in the subsequent stage, the image generated by the generator G is clearer and closer to the original image data, thereby stimulating the discriminative network to make more Precise judgment improves the discriminative ability of the discriminant network D.

Compared with the prior art, the present invention has the following advantages and beneficial effects:

1. The present invention makes full use of the rotation angle information of the input face image, and defines a related encoding loss function, which helps to synthesize a higher-quality frontal face image.

2. The present invention can also generate a clear and realistic frontal face image without deformation when the deflection angle of the input face image exceeds 60°.

3. The front face image synthesized by the present invention can retain the identity information of the input face picture, so it helps to reduce the adverse effects of face posture transformation on face recognition, and brings convenience to the follow-up face recognition work .

4. From the analysis of actual application scenarios, the present invention is expected to promote the development of suspect tracking and other fields. By correcting the side face image of the target person into a front face image, the efficiency of related work is improved.

Description of drawings

Figure 1 is a flow chart of the method of the present invention.

Figure 2 is a flow chart of the generator compositing images.

Figure 3 is a structural diagram of the encoder neural network.

Figure 4 is a structural diagram of the decoder neural network.

Figure 5 is a structure diagram of the discriminant network.

Fig. 6 is a diagram showing the synthesis effect of the present invention.

Detailed ways

In order to describe the present invention more specifically, the technical solution of the present invention will be described in detail below in conjunction with steps, drawings and specific implementation methods.

As shown in Figure 1, the multi-pose face image frontalization method based on generating confrontation network provided by this embodiment includes the following steps:

1) Collect face images of various poses as the training set and test set. It must be ensured that each input face image I ^a of any pose can find the same face in the data set and the face deflection angle is 0° Non-synthesized frontal image I ^g .

The images of the data set used come from the Multi_Pie face data set, which contains images of 337 people under 20 kinds of lighting and 15 postures. The number of pictures in the data set exceeds 750,000, including 20 kinds of lighting and 15 kinds of 337 people. The image under the attitude; the illumination of the picture changes from dark to bright from the illumination label 01 to 20, and the illumination label 07 is the standard illumination condition; mark all the face images in the data set as I ^a , for each image I ^a , in the data Find the images of the same person, the face deflection angle is 0°, and the illumination label is 07, and mark them as I ^g . The data set has been subjected to preprocessing steps such as face detection and face interception before use. Select 13 poses with deflection angles within 90° and images under all lighting conditions as the data set, and divide the images of the first 200 people into the training set, and divide the images of the remaining 137 people into the test set. Normalize and resize all images in the dataset. Among them, normalization refers to dividing the value of all pixels of the image by 255.0, so that the value range of all pixels of the image is [0,1], and resize refers to adjusting the dimensions of all images in the data set by bilinear interpolation. At 128x128x3.

2) In the training phase, input the face image I ^a of any pose in the training set into the generator G, and obtain the corrected code X ₂ and the synthesized front face image I ^f . Input the non-synthesized front face image I ^g into the generator G to obtain the front face code X ₃ .

The function of the generator G is to convert the input face image I ^a into a ^synthetic front face image If, which consists of a pose estimator P, an encoder En, an encoded residual network and R, and a decoder De. As shown in Figure 2, the process of the generator G synthesizing pictures is as follows: input the face image I ^a of any pose to the generator G, the encoder En converts it into the initial code X ₁ , and the coded residual network R passes the initial code X ₁ Estimate the encoding residual R(X ₁ ), the pose estimator P obtains the precise deflection angle γ of the face in the input image in the yaw direction, and the deflection angle γ is input into the function Y to obtain the weight Y(γ) of the encoding residual , the initial code X ₁ and the coded residual are fused to obtain a modified code X ₂ , where X ₂ =X ₁ +Y(γ)×R(X ₁ ), the modified code X ₂ is input to the decoder De, and generated by deconvolution Front ^face image If.

The role of the pose estimator P is to find the more accurate face deflection angle γ of the input face image in the yaw direction, and then obtain the weight of the coding residual: The pose estimator P uses the PnP algorithm to estimate the pose of the face. The cv2.solvePnP() function in the opencv library is directly used in the implementation. The parameters required by this function include the 2D feature points of the face image and the 3D positions corresponding to each feature point. The 2D feature points of the face are directly obtained by the face feature point detection algorithm provided by the open source dlib library. The 3D position of each feature point adopts the position of the feature point on the average face model. These 3D positions are fixed and provided by the relevant documents of the 2D feature point detection algorithm in the dlib library.

The encoder En is a convolutional neural network, and its network structure is shown in Figure 3. Its function is to convert the input face I ^a into an initial code X ₁ . The dimension of the input image ^Ia is 128×128×3, the activation function of the last layer of the encoder is _maxout , and the dimension of the output initial coded X1 is 256.

The coded residual network R is a two-layer fully connected neural network, and the number of neurons in both layers is 256. Assuming that the code obtained by the non-synthesized frontal image I ^g through the encoder is X ₃ , the function of the coding residual network R is to estimate the coding residual R(X ₁ ) between the initial coding X ₁ and the frontal coding X ₃ . The coded residual is multiplied by the weight and the initial code X ₁ is fused to obtain the modified code X ₂ , where X ₂ =X ₁ +Y(γ)×R(X ₁ ).

The decoder De belongs to the deconvolutional neural network, and its network structure is shown in Figure 4. Its function is to decode the correction code X ₂ and obtain the synthesized front face image If through the ^{deconvolution} step. For the current face correction method based on generative confrontation network, the smaller the deflection angle of the input face, the higher the quality of the synthetic face image, and the more identity information of the person can be retained in the image. Since the modified code X ₂ is closer to the frontal face code X ₃ than the initial code X ₁ , the modified code X ₂ is used instead of the initial code X ₁ to input the decoder to synthesize a higher-quality face image.

3) The synthesized front face image If or the non-synthesized front face image ^{Ig is} input into the discrimination network D, and the discrimination network D judges whether the input face image is synthesized or non-synthesized. Then input the synthesized front face image I ^f or non-synthesized front face image I ^g into the face identity feature extractor F to obtain the character identity feature of the input face image.

The discriminant network D is a binary classifier based on the convolutional neural network. Its network structure is shown in Figure 5. Its function is to judge whether the input image comes from the generator G or the original image data. The final output of the discriminant network is a The value is used to indicate the possibility that the input image comes from the generator G. The larger the value, the greater the possibility that the input image comes from the generator G.

The face identity feature extractor F uses the open source Light-CNN-29. Light-CNN29 is a lightweight convolutional neural network with a depth of 29 layers and about 12 million parameters. It can extract the identity features of the input face image, and the final extracted identity feature dimension is 256.

4) Bring the discriminant result of step 3) and the extracted identity features, synthesized frontal image If, non-synthesized frontal image ^Ig , modified code X ₂ and ^frontal code X ₃ into each pre-designed loss function, alternately train the generator G and discriminative network D until the end of training.

In addition to the pixel loss function, identity loss function, symmetric loss function, and adversarial loss function commonly used in similar methods, the loss function of the training generator also includes a self-created encoding loss function.

The first is the pixel loss function, which represents the pixel gap between the synthesized ^frontal image If and the non-synthesized frontal image ^Ig , the formula is as follows:

Where W and H represent the width and height of the image, respectively. and represent the pixel values of the synthesized frontal image If and the non-synthesized frontal image ^Ig at the coordinates ( ^x , y), respectively.

Then there is the symmetric loss function. In view of the symmetry of the human face, the synthesized front face image If should be as close as possible to the image I ^{sym obtained} after it has been flipped left and right. The formula of the symmetric loss function is as follows:

Where W and H represent the width and height of the image, respectively. and Denote the pixel values of the synthesized front face image If and the image I ^sym obtained after it has been flipped left and right respectively under the coordinates ( ^x , y).

Then there is the identity loss function. The face identity feature extractor F can efficiently find the identity features of the front face image. The synthesized front face image If and the non-synthesized front face image I ^g are respectively input into F, and their identity features F(I ^f ) and F(I ^{g )} are obtained. In order to ensure that the synthesized frontal image If can contain the identity information of the non-synthesized frontal image ^Ig , it is necessary to minimize the Manhattan distance between the last two layers of identity feature maps obtained after inputting them into ^F. The identity loss function formula is as follows:

W _i and H _i represent the width and height of the face identity feature map of the penultimate i-th layer, respectively. F is a face identity feature extractor. and ^{represent the values of the coordinates (x, y) of the front face image If and the non-synthesized front face image I g on} the identity feature map of the i-th layer from the bottom respectively.

Finally, there is the confrontation loss function, whose goal is to enable the synthetic image to confuse the discriminative network D, so that the synthetic image is closer to the real image and enhances the realism of the synthetic image. The adversarial loss formula is as follows:

where N is the size of the current training batch, G and D are the generator and discriminative network, respectively, and I ^a and G(I ^a ) represent the input face image and the front face image synthesized by the generator, respectively. The value of D(G(I ^a )) reflects the possibility that the synthesized image G(I ^a ) is judged as a synthesized picture by the discriminant network D. The purpose of minimizing the loss function L _adv is to make the frontal image G(I ^a ) synthesized by the generator pass the detection of the discriminative network, thereby improving the realism of the synthesized frontal image G(I ^a ).

The first goal of the encoding loss function is to make the encoding of the input face image ^Ia and the non-synthesized frontal image ^Ig respectively through the encoder En closer, because the current face pose correction method is in the input face deflection The smaller the angle, the better the effect, so the closer the encoding of the input face image is to the encoding of the same face at 0° deflection angle, the better the effect of the synthesized frontal face; therefore, the first part of the encoding loss function formula as follows:

In the formula, N is the dimension of each encoding, En is the encoder, En(I ^a ) ⁱ is the value of the i-th dimension of the initial encoding En(I ^a ); R is the encoding residual network, R(En(I ^a )) ⁱ is the value of the i-th dimension of the coding residual R(En(I ^a )); En(I ^a ) ⁱ +R(En(I ^a )) ⁱ is equal to the value of the i-th dimension of the correction code; is the value of the i-th dimension of the frontal coding X ₃ obtained by the non-synthesized frontal image through the encoder; the first part of the coding loss function refers to the Manhattan distance between the modified coding and the frontal coding.

The second goal of the encoding loss function is to distinguish the corrected encodings between different characters. A fully connected layer C is constructed behind the corrected encodings En(I ^a )+R(En(I ^a )). The number of neurons in the connection layer C is equal to the number M of characters in the training set, and the second part of the encoding loss function uses the cross-entropy loss function:

In the formula, M is the number of characters in the training set; y _i is the value of the i-th dimension of the one-hot vector y, and the vector y indicates which character in the training set the input face image I ^a belongs to. If the image I ^a belongs to For the jth person, then the value of the jth dimension of the vector y is 1, the value of the remaining dimensions is 0, and the dimension of y is M; En(I ^a )+R(En(I ^a )) is a modified code; C( En(I ^a )+R(En(I ^a ))) _i is the i-th feature vector C(En(I ^a )+R(En(I ^a ))) obtained through the fully connected layer C after the correction code dimension value.

Therefore, the complete encoding loss function is:

L _code = L _code1 + _{λL code2}

In the formula, λ is a constant with a value of 0.1, representing the weight.

In summary, the loss function of the training generator is:

L _total ＝L _pixel +λ ₁ L _sym +λ ₂ L _id +λ ₃ L _adv +λ ₄ L _code

where L _pixel , L _sym , L _id , L _adv and L _code represent pixel loss function, symmetric loss function, identity loss function, adversarial loss function and encoding loss function, respectively. λ ₁ , λ ₂ , λ ₃ and λ ₄ represent the weights of different loss functions.

Referring to the parameter settings of the same type of methods and a large amount of experimental experience, the weights λ ₁ , λ ₂ , λ ₃ and λ ₄ of each loss function are set to 0.2, 0.003, 0.001 and 0.002, respectively. While training the generator, it is necessary to train the discriminant network D. The goal of training D is to enable the discriminant network to distinguish whether the input frontal image is from the generator G or the original data set. The loss function of the trained discriminant network is as follows:

where N is the size of the current training batch, G and D are the generator and discriminative network, respectively, and I ^a is the input image. G(I ^a ) and I ^g denote the synthesized and non-synthesized frontal images, respectively. The values of logD(G(I ^a )) and logD(I ^a ) respectively reflect the synthetic frontal image G(I ^a ) and the non-synthetic frontal image I ^g judged by the discriminative network D as the synthetic frontal image possibility. The purpose of minimizing L _adv2 is to allow the discriminant network D to accurately reflect the possibility that the input picture is a synthetic picture.

Alternately training the generator G and the discriminant network D can optimize and improve each other in the confrontation. In the initial stage, the face image generated by the generator G is blurred, and the discriminant network D can easily judge the source of the input image, thereby motivating the generator G to generate a clearer image and improving the quality of the generator G. In the subsequent stage, the image generated by the generator G is clearer and closer to the original image data, thereby stimulating the discriminant network to make more accurate judgments on the input image and improving the discriminative ability of the discriminant network D.

After the loss function design of the generator G and discriminant network D is completed, the parameters of the generated confrontation network are minimized by the Adam descent method, the learning rate is set to 0.0002, and the batch size is set to 12. After each training of the generator G, the discriminant network D is trained once. With the increase of the training times, the quality of the image generated by the generator is continuously improved, and the ability of the discriminant network to discriminate the input image is continuously strengthened, and the training is finally completed. The deep learning framework used in this experiment is Tensorflow, the graphics card of the computer is 1080ti, and the training stops when the training reaches 20,000 batches.

5) In the test phase, for the input face image I ^a of any ^pose , the trained generator G can synthesize the front face image If of the same face, and then directly ^observe the synthesized front face image If The quality can verify the effect of the present invention. The effect of the generated image is shown in Figure 6. For each row of pictures, the first one is a face image with an input deflection angle exceeding 45°, and the second one is a frontal face image synthesized by the generator , and the third is a non-synthesized frontal face image of the same person in the dataset. It can be seen from the figure that for the face images whose deflection angle exceeds 45°, the present invention can synthesize their front face images, and can preserve the identity information of the original face.

The above-described embodiments are only preferred embodiments of the present invention, and are not intended to limit the scope of the present invention. Therefore, all changes made according to the shape and principles of the present invention should be covered within the protection scope of the present invention.

Claims (6)

1. A method for frontalization of multi-pose face images based on generation confrontation network, characterized in that, comprising the following steps: 1) Collect face images of each pose as a training set and a test set. It must be ensured that each input face image I ^a of any pose can find a non-synthesized front face image I ^g of the same person in the data set; 2) In the training phase, input the face image I ^a of any pose in the training set into the generator G, obtain the corrected code X ₂ and the synthesized front face image I ^f , and input the non-synthesized front face image I ^g into the generator G , to get the front face code X ₃ ; 3) Input the synthesized front face image I ^f or the non-synthesized front face image I ^g into the discriminant network D, and the discriminant network D judges whether the input face image is synthesized or non-synthesized, and then inputs the synthesized front face image I ^f Or non-synthetic positive face image ^Ig input face identity feature extractor F, extract the character identity feature of face image by F; 4) Bring the discriminant result of step 3) and the extracted identity features, synthesized frontal image If, non-synthesized frontal image ^Ig , modified code X ₂ and ^frontal code X ₃ into each pre-designed loss function, alternately train the generator G and the discriminant network D until the end of training; 5) In the test phase, input a face image I ^a of any pose into the trained generator G, and a synthesized front face image I ^f can be obtained, which can be verified by directly observing the quality of the synthesized front face image ^If Effect. 2. A kind of method for frontalization of multi-pose face images based on generation confrontation network according to claim 1, characterized in that: in step 1), the face images of each pose used in the data set are all derived from Dataset Multi_Pie; the number of pictures in the data set exceeds 750,000, including images of 337 people under 20 kinds of lighting and 15 postures; the lighting of the picture changes from dark to bright from light number 01 to 20, of which light number 07 is the standard light Conditions: label all face images in the data set as I ^a , for each image I ^a , find the same person in the data set, the face deflection angle is 0° and the image with the illumination label 07, and mark them as I ^g . 3. A kind of multi-pose face image frontalization method based on generation confrontation network according to claim 1, is characterized in that: in step 2), described generator G is composed of pose estimator P, encoder En, The coded residual network R and the decoder D are composed; the pose estimator P adopts the PnP algorithm to estimate the face pose, and then obtains the deflection angle of the face in the yaw direction, and the function cv2.solvePnP( ) implementation; the encoder En is a convolutional neural network, the encoding residual network R is a two-layer fully connected neural network, and the decoder De is a deconvolutional neural network; The process of the generator G synthesizing pictures is as follows: input the face image I ^a of any pose to the generator G, the encoder En converts it into the initial code X ₁ , and the coded residual network R estimates the code by the initial code X ₁ The residual R(X ₁ ), the attitude estimator P obtains the precise deflection angle γ of the face in the input image in the yaw direction, and the deflection angle γ is input into the function Y to obtain the weight Y(γ) of the coding residual, and the initial coding X ₁ and the coded residual are fused to obtain the modified code X ₂ , where X ₂ =X ₁ +Y(γ)×R(X ₁ ), the modified code X ₂ is input to the decoder De, and the front face image I is generated by deconvolution ^f . 4. A kind of multi-pose face image frontalization method based on generation confrontation network according to claim 1, is characterized in that: in step 3), described discriminant network D is a based on convolutional neural network The two classifiers are used to judge whether the input image comes from the generator G or the original image data; the face identity feature extractor F uses the already open source Light-CNN-29, and Light-CNN29 is a lightweight convolutional neural network The network has a depth of 29 layers and a number of parameters of 12 million. 5. a kind of multi-pose face image frontalization method based on generation confrontation network according to claim 1, is characterized in that: in step 4) in, the target of described loss function is to minimize the synthetic frontal image I The difference between ^f and the non-synthesized front face image I ^g , so that the synthetic front face image I ^f can retain more identity information of the input face image; the loss function used in step 4) is not only commonly used in the same type of method The pixel loss function, identity loss function, symmetric loss function and adversarial loss function, also includes self-created encoding loss function, the first goal of the encoding loss function is to let the input face image I ^a and the non-synthesized positive The encoding obtained by the face image I ^g through the encoder En is closer, because the current face posture correction method works better when the deflection angle of the input face is smaller, so the encoding of the input face image is closer to the same person. The encoding of the face in the case of 0° deflection angle, the better the effect of the synthetic frontal face; therefore, the first part of the encoding loss function formula is as follows: In the formula, N is the dimension of each encoding, En is the encoder, En(I ^a ) ⁱ is the value of the i-th dimension of the initial encoding En(I ^a ); R is the encoding residual network, R(En(I ^a )) ⁱ is the value of the i-th dimension of the coding residual R(En(I ^a )); En(I ^a ) ⁱ +R(En(I ^a )) ⁱ is equal to the value of the i-th dimension of the correction code; Is the value of the i-th dimension of the frontal encoding X ₃ obtained by the non-synthesized frontal image through the encoder; the first part of the encoding loss function refers to the Manhattan distance between the correction encoding and the frontal encoding; The second goal of the encoding loss function is to differentiate the corrected encodings between different characters, and build a fully connected layer C behind the corrected encoded En(I ^a )+R(En(I ^a )), fully connected The number of neurons in layer C is equal to the number M of characters in the training set, and the second part of the encoding loss function uses the cross-entropy loss function: In the formula, M is the number of characters in the training set; y _i is the value of the i-th dimension of the one-hot vector y, and the vector y indicates which character in the training set the input face image I ^a belongs to. If the image I ^a belongs to For the jth person, then the value of the jth dimension of the vector y is 1, the value of the remaining dimensions is 0, and the dimension of y is M; En(I ^a )+R(En(I ^a )) is a modified code; C( En(I ^a )+R(En(I ^a ))) _i is the i-th feature vector C(En(I ^a )+R(En(I ^a ))) obtained through the fully connected layer C after the correction code the value of the dimension; Therefore, the complete encoding loss function is: L _code = L _code1 + _{λL code2} In the formula, λ is a constant with a value of 0.1, representing the weight. 6. A kind of multi-pose face image frontalization method based on generation confrontation network according to claim 1, is characterized in that: in step 4), alternately training generator G and discriminant network D can make both confrontation mutual optimization and improvement; in the initial stage, the face image generated by the generator G is blurred, and the discriminant network D can easily judge the source of the input image, thereby motivating the generator G to generate a clearer image and improving the quality of the generator G; In the subsequent stage, the image generated by the generator G is clearer and closer to the original image data, thereby stimulating the discriminant network to make more accurate judgments on the input image and improving the discriminative ability of the discriminant network D.