CN112949565A - Single-sample partially-shielded face recognition method and system based on attention mechanism - Google Patents

Abstract

The invention belongs to the field of single-sample partially occluded face recognition, and in particular relates to a single-sample partially occluded face recognition method and system based on an attention mechanism, comprising: acquiring a partially occluded test face image and an unoccluded single-sample face image Gallery set and preprocess it; input the preprocessed data into the differential network composed of ReseNet‑34, and extract the shallow feature map through a convolutional layer; use the spatial attention module to adjust the shallow depth from the spatial position information The weight of the layer feature map, and multiply the weight with the feature map output by the last convolution layer to highlight its local detail features; subtract the feature maps of the occluded image and the unoccluded image after highlighting the local detail features; After the difference value is processed by absolute value, it is calibrated by channel on the original feature map through the channel attention module; the calibrated feature map is sent to the full connection layer to output the classification result; Identification.

Description

A single-sample partial occlusion face recognition method and system based on attention mechanism

technical field

The invention belongs to the field of single-sample partial occlusion face recognition, and particularly relates to a single-sample partial occlusion face recognition method and system based on an attention mechanism.

Background technique

Face recognition is a biometric recognition technology based on human facial feature information. By using cameras or cameras to collect images or videos containing human faces, a series of technical processing is performed on the collected faces to achieve recognition. The purpose of the identity of different people. After years of research, face recognition has achieved good results under controlled conditions, but face recognition under unconstrained conditions still faces many challenges. In some special application scenarios, such as ID card management system, criminal investigation and law enforcement system, passport verification and registration port identification, etc., only one face image of each person (face image of ID photo) can be collected as a training sample. When the test samples may be affected by drastic facial changes such as illumination changes, posture changes, expression changes, and occlusion by external objects, etc., these influences will cause the facial features of the face to have greater intra-class differences than inter-class differences, so The single-sample partial occlusion face recognition problem occurs, that is, the only single-sample design face recognition method is used to identify unknown face images.

In real life, the situations that cause face occlusion can be divided into the following three categories: 1) occlusion by external objects (such as sunglasses, scarves, hats, etc.); 2) extreme lighting (such as shadows); 3) self-confidence caused by pose changes Occlusion (such as a side face), the occlusion we say below is mainly face recognition that is occluded by external objects.

Although face recognition has been well handled in the research field of occlusion, there are still some problems: 1) Current methods still cannot completely remove the impact of occlusion. If the influence of occlusion can be completely removed, the recognition results will be more ideal. 2) When the current method faces the single-sample problem, some existing mature face recognition algorithms cannot use a single training sample to extract intra-class variation information, so their recognition effect will be much worse.

Most of the research work now focuses on how to improve the accuracy of the recognition system, ignoring the problems of face databases, such as the difficulty of collecting samples or the storage limitations of the system, each person in the database may have only one sample image. In this case, most traditional methods, such as PCA and LDA, will suffer from performance degradation or even fail to work, and when the deep learning route is adopted, each person only stores a face database of one image, lacking a large number of samples to learn rich intra-class Change information, under this premise, dealing with occlusion problems does not perform well. To sum up, single-sample partial occlusion face recognition is unavoidable in real-world application scenarios, and face recognition under single-sample constraints still faces huge challenges.

SUMMARY OF THE INVENTION

In view of the above-mentioned problems of lack of a large number of samples to learn rich intra-class variation information and the loss of facial feature information due to the existence of occlusion affecting recognition accuracy, the present invention proposes a single-sample partial occlusion face recognition based on attention mechanism A method and system, the method comprising:

Input the face image pair set with category labels as the source domain dataset, the face image pair includes a clean frontal face and a face with the same identity with partial occlusion, and preprocess the face image pair dataset;

Input the preprocessed face image into the differential network composed of two ResNet-34, and extract the shallow feature map through a convolutional layer;

Input the above shallow feature map into a residual network composed of four residual module groups cascaded in sequence to extract the global features of the face image;

Embed a spatial attention module between the second layer and the fourth layer residual module, adjust the weight of the pixels in the unoccluded area of the shallow feature map, and output a spatial position weight feature map;

The spatial position weight feature map and the feature map output by the fourth-layer residual module are connected by multiplication, and the local detail features from the lower layer are obtained through the fusion of cross-layer information;

The absolute value of the difference between the occluded image after highlighting the local detail features and the feature map of the unoccluded image is used as the input of the channel attention module;

According to the absolute value of the input, the channel attention module calibrates the feature maps of the occluded image and the unoccluded image after highlighting the local detail features by channel, and the calibrated feature map is sent to the fully connected layer to output the classification result;

The network is iteratively trained by jointly optimizing the cross-entropy loss function and the contrast loss caused by the difference between the face and the image;

After multiple rounds of training, the network loss tends to be stable, the iterative training process ends, and the trained network model is obtained;

The single-sample face gallery set of the target domain and the partially occluded test face image are input into the trained network model, and the model calculates and outputs the category of the final occluded face according to the cosine distance of the face image feature.

Further, preprocessing the face image data set includes cropping the face image to a size of 128×128, and performing a pixel normalization operation on the cropped face image, which is expressed as:

X _pix = (X _pix -128)/128;

Among them, X _pix is the image pixel value corresponding to the face image.

Further, extracting a shallow feature map through a convolutional layer includes: inputting a face image with a channel number of 3 into a convolutional layer with a convolution kernel size of 3 × 3, a channel number of 64, and a stride of 1 to perform features. Extract, output a feature map with a size of 128×128 and an output channel of 64, and pass the feature map through the maximum pooling layer to obtain a shallow feature map of the face image.

Further, in the four residual module groups that are cascaded in sequence, each residual module group includes 3, 4, 6, and 3 residual modules in the cascade order, and each residual module group is in the cascade order. The output feature map sizes are 64×64, 32×32, 16×16, 8×8.

Further, the process of obtaining local detail features from lower layers includes:

A spatial attention module is embedded between the second-layer residual module and the fourth-layer residual module;

The input 3D tensor of h′×w′×c′ is obtained by global average pooling and global maximum pooling to obtain the weight vector feature map;

The weight vector feature map is processed using a convolutional layer with a convolution kernel of 7 × 7, a padding size of 3, and a number of channels of 1 and a Sigmoid nonlinear activation layer;

The size of the output of the fourth-layer residual group module is 8 × 8 feature maps through bilinear interpolation method upsampling into 32 × 32 feature maps;

The upsampled feature map is multiplied by the processed weight vector feature map, and then the 8×8 feature map is output by downsampling to obtain the local detail features from the lower layers.

Further, the channel attention module performs channel-by-channel calibration on the feature maps of the occluded image and the unoccluded image after highlighting the local detail features according to the absolute value of the input feature map difference, including:

Perform a subtraction operation on the feature map of the occluded image after highlighting the local detail features and the feature map of the unoccluded image, and use the absolute value of the difference obtained by the subtraction as the input of the channel attention module;

The channel attention module adopts global average pooling and global maximum pooling operations to obtain two 1×1×C channel descriptions;

The two channel descriptions are respectively sent to a shallow neural network. In the shallow neural network, the number of neurons in the first layer is C/r, the activation function is ReLU, and the number of neurons in the second layer is C;

Add and merge the two feature maps extracted by the shallow neural network, and use the Sigmoid activation function to obtain the weight coefficients of each channel;

After the weight of the feature channel is obtained, it is weighted to the original feature by multiplication channel by channel, and the original feature re-calibration in the channel dimension is completed;

Among them, C is the number of channels, and r is the dimension reduction factor.

Further, the cross-entropy loss function is expressed as:

为交叉熵损失函数；

表示网络中样本yⁱ的身份分类概率，

表示第i个人脸图像对中存在遮挡的人脸图像，F为差分网络中最后一个卷积层后的全连接层，

表示进行通道注意力掩模运算后的存在局部遮挡的人脸的特征，μ(·)表示通道注意力输出的权重特征图，是值为[0,1]之间的掩膜；f(·)代表卷积层最后输出的特征；n表示人脸图像的训练集总样本数。in,

is the cross entropy loss function;

represents the identity classification probability of the sample ^yi in the network,

Indicates the occluded face image in the i-th face image pair, F is the fully connected layer after the last convolutional layer in the differential network,

Represents the feature of the face with partial occlusion after the channel attention mask operation, μ(·) represents the weight feature map of the channel attention output, which is a mask with a value between [0, 1]; f(· ) represents the final output feature of the convolutional layer; n represents the total number of samples in the training set of face images.

Further, the contrastive loss function expresses:

为由于遮挡区域的存在所造成的两张图片对比损失，μ(·)表示通道注意力输出的权重特征图，是值为[0,1]之间的掩膜；

表示第i个人脸图像对中存在遮挡的人脸图像；xⁱ表示第i个人脸图像对中不存在遮挡的人脸图像；f(·)代表卷积层最后输出的特征；n表示人脸图像的训练集总样本数。in,

is the contrast loss of the two images caused by the existence of the occluded area, μ( ) represents the weight feature map of the channel attention output, which is a mask with a value between [0, 1];

represents the occluded face image in the ith face image pair; x ⁱ represents the ith face image pair without occlusion; f( ) represents the final output feature of the convolutional layer; n represents the face The total number of samples in the training set of images.

The present invention also proposes a single-sample partial occlusion face recognition system based on an attention mechanism, including an image acquisition module, a data preprocessing module, a neural network module and an output module, wherein:

The image acquisition module is used to input the data set and obtain the face image information or the face image to be tested;

The data preprocessing module is used to normalize the pixels of the face image, and at the same time, perform data enhancement in the source domain data set, and expand the source domain training set by randomly adding occluders;

Neural network module for building and training a differential neural network formed by two identical ReseNet-34 embedded with attention mechanism;

The output module is used to output the final identity category of the face image to be tested. The model trained in the source domain will be transferred to the target domain data set, and the single-sample face gallery set and the partially occluded test face image will be sent into The model determines the identity of the partially occluded test face image.

Beneficial technical effects of the present invention:

(1) The present invention has the effects of high speed and high precision, and can accurately identify the identity of any input face image with partial occlusion.

(2) The present invention proposes a novel feature extraction architecture that utilizes differential networks while taking into account high-level-low-level information fusion. Through the connection of global information and local information, the fusion of cross-layer information is obtained, which enhances the feature representation capability of the network. In order to obtain a more discriminative representation and achieve a high-precision single-sample partial occlusion face recognition effect.

(3) The present invention embeds spatial attention and channel attention modules in the differential network. The spatial attention module guides the model to focus on where the features are meaningful, and focuses on the features of the unoccluded areas; the channel attention module passes the The correlation between the modeling channels is completed, the original feature re-calibration in the channel dimension is completed, the channels that respond positively to the occlusion area are suppressed, and the defects existing in the existing single-sample partial occlusion face recognition methods are overcome.

Description of drawings

1 is a schematic process diagram of a single-sample partial occlusion face recognition method based on an attention mechanism provided by an embodiment of the present invention;

2 is a schematic diagram of a spatial attention module according to an embodiment of the present invention;

3 is a schematic diagram of a channel attention module according to an embodiment of the present invention;

4 is a schematic diagram of a training process according to an embodiment of the present invention;

5 is a schematic structural diagram of a single-sample partial occlusion face recognition network based on an attention mechanism according to an embodiment of the present invention;

FIG. 6 is an application effect diagram of an embodiment of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

The present invention proposes a single-sample partial occlusion face recognition method based on an attention mechanism, which specifically includes the following steps:

Input the face image pair set with category labels as the source domain dataset, the face image pair includes a clean frontal face and a face with the same identity with partial occlusion, and preprocess the face image pair dataset;

Input the preprocessed face image into the differential network composed of two ResNet-34, and extract the shallow feature map through a convolutional layer;

Input the above shallow feature map into a residual network composed of four residual module groups cascaded in sequence to extract the global features of the face image;

Embed a spatial attention module between the second layer and the fourth layer residual module, adjust the weight of the pixels in the unoccluded area of the shallow feature map, and output a spatial position weight feature map;

The spatial position weight feature map and the feature map output by the fourth-layer residual module are connected by multiplication, and the local detail features from the lower layer are obtained through the fusion of cross-layer information;

The absolute value of the difference between the occluded image after highlighting the local detail features and the feature map of the unoccluded image is used as the input of the channel attention module;

According to the absolute value of the input, the channel attention module calibrates the feature maps of the occluded image and the unoccluded image after highlighting the local detail features by channel, and the calibrated feature map is sent to the fully connected layer to output the classification result;

The network is iteratively trained by jointly optimizing the cross-entropy loss function and the contrast loss caused by the difference between the face and the image;

After multiple rounds of training, the network loss tends to be stable, the iterative training process ends, and the trained network model is obtained;

The single-sample face gallery set of the target domain and the partially occluded test face image are input into the trained network model, and the model calculates and outputs the category of the final occluded face according to the cosine distance of the face image feature.

In one embodiment, the present invention intends to use a trained model, and the data set used in the training phase is the CASIA_WebFace face data set, which serves as the source domain and includes 494,414 face images collected from the Internet, From 10575 people, the labels are represented by the same numerical value for the same identity. In order to deal with the problem that the recognition accuracy decreases due to the existence of occluded areas, the present invention randomly adds black blocks or real occlusions such as masks and leaves to the CASIA_WebFace data set for data enhancement, and performs a training on the model with the CASIA_WebFace data set after data enhancement. After the model is trained, the model is migrated to the target domain data set. In order to ensure the fairness of the division of the training set and the test set, all the comparative experiments in this embodiment are the target domain single-sample face Gallery set and partial occlusion test. The division structure of the face image set is the same. The target domain datasets include three single-sample face datasets: AR, Extended Yale B, and CAS-PEAL-R1.

Preprocess the source domain and target domain datasets: uniformly crop all images to 128×128 size, and normalize the pixels of the processed face images. The formula includes:

X _pix = (X _pix -128)/128;

Wherein, X _pix corresponds to the input face image pixel value, specifically, the face image pixel value input to the difference network.

The preprocessed sample images are sequentially sent to the neural network, and the loss function is minimized by backpropagation to train the network. Compared with the traditional single-sample partial occlusion face recognition algorithm, the present invention adopts ResNet-34 to reduce the size of the model and improve the accuracy of the model. ResNet-34 adds a shortcut branch outside the original convolution layer to form a basic residual. Difference module, the original mapping H(X) is expressed as H(X)=F(X)+x, where F(X) is the residual mapping, and x is the input signal. The learning of H(X) is transformed into the learning of F(X), and the learning of F(X) is simpler than that of H(X). This structure reduces the amount of calculation and effectively solves the problem due to the number of network layers. Attenuation problem caused by too deep.

Input the face image into the ResNet-34 network, and perform shallow feature extraction through a convolutional layer as the input feature map of the next residual network. Specifically, the feature map with the input channel number of 3 first passes through the kernel size of 3. ×3, the number of channels is 64, the stride is 1 for feature extraction, the size of the output feature map is 128 × 128, the output channel becomes 64, and then after the maximum pooling layer, the output feature map at this time is as Subsequent residual modules input feature maps.

After the shallow feature map is extracted, four groups of residual modules connected in sequence are connected to form a residual network, and the residual network is used as a global branch to extract the global features of the face image.

It can be understood that the core improvement of the present invention lies in the two modules of attention mechanism and cross-layer information fusion proposed by the present invention, and the attention mechanism module is divided into spatial attention mechanism and channel attention mechanism, which are mainly embedded in In ResNet-34; in order to input a pair of face images at the same time, ResNet-34 is improved at the same time as a differential network. At first, the convolutional layer extracts the shallow feature map, and in the maximum pooling layer There are four groups of residual modules connected in sequence to form a residual network, and the residual network is used as a global branch to extract the global features of the face image; on the other hand, the second and fourth layers of the residual network are used. A spatial attention module is embedded between them and the feature maps output by the two are connected, that is, cross-layer information fusion, and the absolute value of the difference of the fourth layer feature map of the differential network is used as the input of the next channel attention module. In the present invention, if not specifically emphasized, the residual network of the present invention mainly refers to the differential network composed of multiple groups of residual modules and attention mechanism modules behind ResNet-34. Of course, the above division refers only to The improvements of the present invention are more prominently embodied, and those skilled in the art can make adaptive understanding according to the overall embodiment of the present invention and the accompanying drawings.

Further, the difference network is composed of a residual module and an attention mechanism module.

The process of building a differential network includes the following steps:

The shallow feature map output by the first convolutional layer is input into two ResNet-34 network branches. The ResNet-34 network branch is composed of 4 groups of residual modules in series. The number of input channels of each group of residual modules is 64, 128, 256, 512, and each residual module is composed of convolution operations, batch normalization (BN) operations and Rectified Linear Unit (ReLU) operations, which act on the mapping of global features , the corresponding output channels are 64, 128, 256, 512;

A spatial attention module is embedded between the second-layer residual module and the fourth-layer residual module of each branch of the differential network. The embedding process of the spatial attention module includes: inserting a spatial attention module into the residual network to guide the model Pay attention to where the meaningful features are. Specifically, the three-dimensional tensor of h′×w′×c′ output by the convolutional layer of the second-layer residual module is used for global average pooling and global maximum pooling. The difference is , here is the operation in the dimension of the channel, that is to say, pool all input channels into 2 real numbers, and get two (h'×w'×1 from the input of (h'×w'×c') shape ) weight vector, at this time, two (h'×w'×1) weight vectors are spliced into a (h'×w'×2) weight vector feature map based on the channel, where h' and w' are input respectively. The length and width of the face image, c' is the number of channels;

A 7×7 convolution kernel is used for convolution, the padding size is 3, the number of channels is compressed to 1, and then the Sigmoid nonlinear activation operation is performed to form a new (h′×w′×1) weight after convolution. vector feature map;

The 8 × 8 feature map output by the residual module of the fourth layer is upsampled to a 32 × 32 feature map through bilinear interpolation, and the (h' × w' × 1) weight vector feature map is at the channel level. Multiply up, and then output an 8×8 feature map by downsampling to obtain new features after scaling;

The scaled new feature maps are subtracted at the two branches at the end of the network, and the absolute value of the difference is obtained as the input of the channel attention module.

In one embodiment, after the second-layer residual module performs the convolution operation, the input is a 32×32×128 three-dimensional tensor, which are the length, width, and number of channels of the input feature map, respectively, and the three-dimensional tensor is first passed through the global maximum pool. Pooling and global average pooling, that is, pooling in the dimension of the column channel, taking the maximum and average value of a column of channels, pooling a column of channels at a time and becoming a value is a channel, the length and width are unchanged. The input feature map is 32×32×128, and after one pooling, it becomes a 32×32×1 feature map. At this time, two (32×32×1) weight vectors are spliced into a (32×32×1) weight vector feature map based on the channel.

A 7×7 convolution kernel is used for convolution, the padding size is 3, the number of channels is compressed to 1, and then the Sigmoid nonlinear activation operation is performed to form a new (32×32×1) weight vector feature after convolution picture;

The 8 × 8 feature map output by the residual module of the fourth layer is upsampled to a 32 × 32 feature map through bilinear interpolation, and the (32 × 32 × 1) weight vector feature map is at the channel level. The upper multiplication, that is, the features from the lower layer are fused, and then the feature map of 8×8 size is output by downsampling, and the new features after scaling are obtained;

The specific process of the channel attention module adjusting the weight of each channel is as follows:

The scaled new feature maps are subtracted at the two branches at the end of the network, and the absolute value of the feature difference is used as the input of the channel attention module. In order to aggregate spatial features, the channel attention module receives the new feature map after subtraction, and adopts global average pooling and global max pooling to obtain two 1×1×512 channel descriptions, two different pooling It means that the extracted high-level features are richer;

The two channel descriptions are respectively sent to a shallow neural network. The number of neurons in the first layer is 512/16, the activation function is ReLU, and the number of neurons in the second layer is 512. the network is shared;

The two obtained feature maps are combined by addition, and a sigmoid activation function is used to obtain the weight coefficients of each channel;

After the weight of the feature channel is obtained, it is weighted to the original feature by multiplication channel by channel, and the original feature re-calibration in the channel dimension is completed;

The feature re-calibrated by the channel attention module is used as the final face feature representation, and the classification result is output according to the fully connected layer.

The joint cross-entropy loss function and the face-to-image contrast loss caused by the difference are solved for the network, and the loss function is minimized through backpropagation, the entire difference network is jointly optimized, and the network is iteratively trained.

Further, the cross entropy loss function is expressed as follows:

The contrastive loss is expressed as follows:

为交叉熵损失函数；

表示网络中样本yⁱ的身份分类概率，

表示第i个人脸图像对中存在遮挡的人脸图像，xⁱ表示第i个人脸图像对中不存在遮挡的人脸图像，F为差分网络中最后一个卷积层后的全连接层，

为由于遮挡区域的存在所造成的两张图片对比损失，

表示进行通道注意力掩模运算后的存在局部遮挡的人脸的特征，μ(·)表示通道注意力输出的权重特征图，是值为[0,1]之间的掩膜；f(·)代表卷积层最后输出的特征；n表示人脸图像的训练集总样本数。in,

is the cross entropy loss function;

represents the identity classification probability of the sample ^yi in the network,

Indicates the occluded face image in the ith face image pair, x ⁱ represents the ith face image pair without occlusion in the face image, F is the fully connected layer after the last convolutional layer in the differential network,

is the comparison loss of the two images caused by the existence of the occluded area,

Represents the feature of the face with partial occlusion after the channel attention mask operation, μ(·) represents the weight feature map of the channel attention output, which is a mask with a value between [0, 1]; f(· ) represents the final output feature of the convolutional layer; n represents the total number of samples in the training set of face images.

The Adam optimizer is used for training adjustment. After multiple rounds of training, the network tends to be stable, and the iterative process ends, and a trained network model is obtained. The training process is shown in Figure 4.

After acquiring the image dataset, preprocess the face image;

A differential network model based on the attention mechanism is constructed, that is, the network model constructed by the present invention;

Train the network using the dataset and perform multiple iterations;

The result of the network output and the real identity value label corresponding to the face image are used to solve the loss until the loss tends to be stable.

At this point, end the training and output the trained network model.

The trained network model is shown in Figure 5.

When using the trained neural network model, input the target domain single-sample face gallery set and the partially occluded test face image into the trained network model, and the model calculates and outputs the final occluded face according to the cosine distance of the face image features. category.

The present invention also proposes a single-sample partial occlusion face recognition system based on an attention mechanism, including an image acquisition module, a data preprocessing module, a neural network module and an output module, wherein:

The image acquisition module is used to input the data set and obtain the face image information or the face image to be tested;

The data preprocessing module is used to normalize the pixels of the face image, and at the same time, perform data enhancement in the source domain data set, and expand the source domain training set by randomly adding occluders;

A neural network module for building and training a differential neural network formed by two identical residual networks embedded with an attention mechanism;

The output module is used to output the final identity category of the face image to be tested. The model trained in the source domain will be transferred to the target domain data set, and the single-sample face gallery set and the partially occluded test face image will be sent into The model determines the identity of the partially occluded test face image.

Further, the neural network module includes ResNet-34. When the data output by the data preprocessing module is input into the neural network module, a shallow feature map is extracted through a convolutional layer, and the shallow feature map is input in the order of A residual network composed of four residual module groups connected to extract the global features of face images; a spatial attention module is embedded between the second-layer residual module and the fourth-layer residual module to adjust shallow features The weight of the pixels in the unoccluded area of the map is output, and a spatial position weight feature map is output; the spatial position weight feature map is connected with the feature map output by the fourth-layer residual module, and the information from the lower layer is obtained through the fusion of cross-layer information. local detail features;

Perform a subtraction operation on the feature map of the occluded image after highlighting the local detail features and the feature map of the unoccluded image, and use the absolute value of the difference obtained by the subtraction as the input of the channel attention module;

The channel attention module adjusts the weights of each channel, including:

The subtracted value received by the channel attention module adopts global average pooling and global max pooling to obtain two channel descriptions;

The two channel descriptions are respectively sent to a shallow neural network to extract features, and the extracted features are combined by addition;

The combined features use a sigmoid activation function to obtain the weight coefficients of each channel;

After the weight of the feature channel is obtained, it is weighted by multiplication to the feature map of the occluded image and the unoccluded image after highlighting the local detail features, and the original feature re-calibration in the channel dimension is completed;

The features re-calibrated by the channel attention module are used as the final face feature representation, and the classification results are output according to the fully connected layer.

Jointly optimize the cross-entropy loss function and the contrast loss caused by the difference between the face and the image, and iteratively train the network. The training process is detailed in the Methods section, and will not be repeated here.

Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, and substitutions can be made in these embodiments without departing from the principle and spirit of the invention and modifications, the scope of the present invention is defined by the appended claims and their equivalents.

Claims (9)

1. A single-sample partial occlusion face recognition method based on an attention mechanism, characterized in that it specifically comprises the following steps: Input the face image pair set with category labels as the source domain dataset, the face image pair includes a clean frontal face and a face with the same identity with partial occlusion, and preprocess the face image pair dataset; Input the preprocessed face image into the differential network composed of two ResNet-34, and extract the shallow feature map through a convolutional layer; Input the above shallow feature map into a residual network composed of four residual module groups cascaded in sequence to extract the global features of the face image; Embed a spatial attention module between the second layer and the fourth layer residual module, adjust the weight of the pixels in the unoccluded area of the shallow feature map, and output a spatial position weight feature map; The spatial position weight feature map and the feature map output by the fourth-layer residual module are connected by multiplication, and the local detail features from the lower layer are obtained through the fusion of cross-layer information; The absolute value of the difference between the occluded image after highlighting the local detail features and the feature map of the unoccluded image is used as the input of the channel attention module; The channel attention module calibrates the feature maps of the occluded image and unoccluded image after highlighting the local detail features by channel according to the absolute value of the input, and the calibrated feature map is sent to the fully connected layer to output the classification result; The network is iteratively trained by jointly optimizing the cross-entropy loss function and the contrast loss caused by the difference between the face and the image; After multiple rounds of training, the network loss tends to be stable, the iterative training process ends, and the trained network model is obtained; The single-sample face gallery set of the target domain and the partially occluded test face image are input into the trained network model, and the model calculates and outputs the category of the final occluded face according to the cosine distance of the face image feature. 2. The single-sample partial occlusion face recognition method based on the attention mechanism according to claim 1, wherein the preprocessing of the face image to the dataset comprises cropping the face image to a size of 128×128, and The pixel normalization operation is performed on the cropped face image, which is expressed as: X _pix = (X _pix -128)/128; Among them, X _pix is the pixel value corresponding to the face image. 3. the single-sample partial occlusion face recognition method based on the attention mechanism according to claim 1, is characterized in that, extracting the shallow feature map through a convolutional layer comprises: the face image input that the number of channels is 3 The convolutional layer with the convolution kernel size of 3 × 3, the number of channels is 64, and the stride size of 1 is used for feature extraction, and the output size is 128 × 128, and the output channel becomes a feature map of 64. The layer is used to obtain the shallow feature map of the face image. 4. The single-sample partial occlusion face recognition method based on an attention mechanism according to claim 1, wherein in the four residual module groups cascaded in sequence, each residual module group is cascaded in sequence. It includes 3, 4, 6, and 3 residual modules in sequence, and the size of the feature map output by each residual module group is 64 × 64, 32 × 32, 16 × 16, 8 × 8 according to the cascade order. 5. The single-sample partial occlusion face recognition method based on an attention mechanism according to claim 4, wherein the process of obtaining the local detail features from the lower layer comprises: A spatial attention module is embedded between the second-layer residual module and the fourth-layer residual module; The input 3D tensor of h′×w′×c′ is obtained by global average pooling and global maximum pooling to obtain the weight vector feature map; The weight vector feature map is processed using a convolutional layer with a convolution kernel of 7 × 7, a padding size of 3, and a number of channels of 1 and a Sigmoid nonlinear activation layer; Upsampling the 8 × 8 feature map output by the fourth layer residual group module into a 32 × 32 feature map through bilinear interpolation; The upsampled feature map is multiplied by the processed weight vector feature map, and then the 8×8 feature map is output by downsampling to obtain the local detail features from the lower layers. 6. The single-sample partial occlusion face recognition method based on the attention mechanism according to claim 1, is characterized in that, the channel attention module according to the absolute value of the input is the occlusion image and the unoccluded image after highlighting the local detail feature. The channel-by-channel calibration on the feature map includes: Perform a subtraction operation on the feature map of the occluded image after highlighting the local detail features and the feature map of the unoccluded image, and use the absolute value of the difference obtained by the subtraction as the input of the channel attention module; The channel attention module adopts global average pooling and global maximum pooling operations to obtain two 1×1×C channel descriptions; The two channel descriptions are respectively sent to a shallow neural network. In the shallow neural network, the number of neurons in the first layer is C/r, the activation function is ReLU, and the number of neurons in the second layer is C; Add and merge the two feature maps extracted by the shallow neural network, and use the Sigmoid activation function to obtain the weight coefficients of each channel; After the weight of the feature channel is obtained, the channel-by-channel weighting is applied to the features of the occluded image and the unoccluded image after highlighting the local detail features, and the original feature re-calibration in the channel dimension is completed; Among them, C is the number of channels, and r is the dimension reduction factor. 7. The single-sample partial occlusion face recognition method based on an attention mechanism according to claim 1, wherein the cross-entropy loss function is expressed as:

in,

is the cross entropy loss function;

represents the identity classification probability of the sample ^yi in the network,

Indicates the occluded face image in the i-th face image pair, F is the fully connected layer after the last convolutional layer in the differential network,

Represents the feature of the face with partial occlusion after the channel attention mask operation, μ(·) represents the weight feature map of the channel attention output, which is a mask with a value between [0, 1]; f(· ) represents the final output feature of the convolutional layer; n represents the total number of samples in the training set of face images. 8. The single-sample partial occlusion face recognition method based on an attention mechanism according to claim 1, wherein the contrast loss function represents:

in,

is the contrast loss of the two images caused by the existence of the occluded area, μ( ) represents the weight feature map of the channel attention output, which is a mask with a value between [0, 1];

represents the occluded face image in the ith face image pair; x ⁱ represents the ith face image pair without occlusion; f( ) represents the final output feature of the convolutional layer; n represents the face The total number of samples in the training set of images. 9. A single-sample partial occlusion face recognition system based on an attention mechanism, characterized in that it includes an image acquisition module, a data preprocessing module, a neural network module and an output module, wherein: The image acquisition module is used to input the data set and obtain the face image information or the face image to be tested; The data preprocessing module is used to normalize the pixels of the face image, and at the same time, perform data enhancement in the source domain data set, and expand the source domain training set by randomly adding occluders; A neural network module for building and training a differential neural network consisting of two identical ReseNet-34 embedded with attention mechanisms; The output module is used to output the final identity category of the face image to be tested, that is, the model trained in the source domain is transferred to the target domain dataset, and the single-sample face gallery set and the partially occluded test face image are sent into The model determines the identity of the partially occluded test face image.

Images