CN112016437A - A method of living body detection based on key frames of face video - Google Patents

Abstract

The invention provides a living body detection method based on a face video key frame, which comprises the following steps: acquiring a region image corresponding to a face in each frame of face image; acquiring face feature key points corresponding to the region images, and acquiring the position variation of the face key points according to the face feature key points in the multi-frame face images; acquiring a face video key frame according to the position variation, performing living body identification on the face video key frame, and outputting an identification result; the invention can effectively improve the accuracy and efficiency of the living body detection.

Description

A method of living body detection based on key frames of face video

technical field

The invention relates to the field of intelligent recognition, and in particular to a method for detecting a living body based on a face video key frame.

Background technique

As the identity authentication system using face recognition as a means is more and more widely used in daily social life, its security is always facing challenges and more and more attention is paid to it. The traditional face recognition method does not include a module for verifying the authenticity of the face. It only needs to input the collected face image to output the corresponding recognition result; and with the rapid rise of digital imaging equipment and the vigorous development of the Internet, people The difficulty of obtaining face images is also getting lower and lower. These factors make it easy for attackers to obtain a copy of a specific face through some tools or means (such as printed photos, electronic display screens, three-dimensional masks, etc.), so as to pretend to be a legitimate user, so as to achieve the purpose of invading the system. This security problem is particularly prominent in unattended closed application environments.

Therefore, face live detection technology came into being, which aims to distinguish whether the face in the currently collected video or image is a live face (real life face) or a prosthetic face (a fake person pretending to be a real person). face), in order to prevent criminals from fraudulently using the face information of legitimate users. Today, for a typical face recognition system, liveness detection is an essential preprocessing module in its front-end. When the system collects the face video or image through the input device, the face liveness detection is carried out first, and the subsequent identity verification can only be carried out under the condition that the current face is confirmed to be a real face, so its security has been greatly improved. guarantee.

Most of the common face live detection algorithms focus on image texture analysis, motion information analysis, interactive identification and other directions. They can usually be divided into two implementation methods, a single-frame model that takes a single image as input and a video frame stream as input. Multi-frame model: The complexity of the single-frame model is relatively low, and the computational efficiency is high, but there is a certain possibility of false detection; while the multi-frame model is relatively high in complexity, with good overall performance, but low computational efficiency.

With the reduction of equipment cost and the advancement of imaging technology, more and more live detection algorithms have turned to face video as model input, and each acquired face video is taken as a single data sample. However, generally collected face videos usually have a very high frame rate, and even a short video contains hundreds or even thousands of frames in total. Due to the limitation of model size, it is usually necessary to extract a small number of image frames from face videos as input using an appropriate sampling strategy. Among them, the more common methods include uniform sampling, continuous sampling, random sampling, etc., but these sampling strategies are not based on any features related to faces, nor can they ensure that the extracted image frames contain significant features that can be used to discriminate faces. The key information of authenticity, thus potentially affecting the performance of subsequent live detection algorithm modules.

SUMMARY OF THE INVENTION

In view of the above problems in the prior art, the present invention proposes a method for detecting a living body based on key frames of face video, which mainly solves the problem of low recognition rate caused by information loss caused by sampling.

In order to achieve the above objects and other objects, the technical solutions adopted in the present invention are as follows.

A liveness detection method based on face video key frames, comprising:

Obtain the area image of the corresponding face in each frame of face image;

Obtaining the face feature key points corresponding to the regional images, and obtaining the change amount of the face key point position according to the face feature key points in the multi-frame face images;

Acquire a face video key frame according to the position change amount, perform in vivo recognition on the face video key frame, and output a recognition result.

Optionally, before acquiring the area image corresponding to the face in each frame of face image, the position of the bounding box of the largest face in each frame of face image is obtained, and the position of the bounding box corresponding to the multiple frames of face images is obtained. the area image.

Optionally, setting an envelope frame including all the frame positions in the multi-frame images, and setting an image cropping frame according to the geometric center position of the envelope frame;

A region image corresponding to the face in each frame of the face image is acquired according to the image cropping frame.

Optionally, the image cropping frame includes a square frame.

Optionally, a frame of face image is selected as the reference frame, and the position difference between the corresponding face feature key points in the multi-frame face images and the face feature key points in the reference frame is obtained to obtain the position. amount of change.

Optionally, the extreme point of the position change is acquired, and the corresponding face video key frame is acquired according to the extreme point.

Optionally, the number of face video key frames to be extracted is set, and when the number of the extreme value points is greater than the number of the face video key frames to be extracted, the extreme value points are merged, Obtain significant extreme points, and obtain face video key frames according to the significant extreme points; when the number of the extreme points is equal to the number of face video key frames to be extracted, the extreme points are directly The face image at the corresponding moment is used as the face video key frame; when the number of the extreme points is less than the number of the face video key frames to be extracted, from the face image corresponding to the non-extreme point Extract the corresponding number of face images to supplement the face video key frames.

Optionally, when acquiring the significant extremum points, the extremum points that satisfy the preset condition are merged according to the time difference of the adjacent multiple extreme value points and the position difference in the corresponding face image.

Optionally, when the number of the significant extreme points is greater than the number of the face video key frames to be extracted, obtain from the significant extreme points the number of face video key frames to be extracted. The corresponding significant extreme points are obtained, and the corresponding face video key frames are obtained; when the number of the significant extreme points is equal to the number of the face video key frames to be extracted, the significant extreme points are directly obtained. The face image at the corresponding moment is used as a face video key frame; when the number of the significant extreme points is less than the number of the face video key frames to be extracted, from the face image corresponding to the non-significant extreme point Extract the corresponding number of face images to supplement the face video key frames.

Optionally, a living body recognition model is trained, and the recognition result is obtained according to the output probability of the living body recognition model.

As described above, the present invention proposes a method for detecting a living body based on a face video key frame, which has the following beneficial effects.

Face recognition and information interaction through the combination of visible light images, near-infrared images and depth images can effectively improve the security and accuracy of face recognition.

Comprehensively considering the position changes of face regions in multi-frame face images to extract key frames, it can effectively solve the problem of information loss in traditional sampling methods; using single-frame face images for parallel operations can effectively improve detection efficiency.

Description of drawings

FIG. 1 is a flowchart of a method for detecting a living body based on a face video key frame according to an embodiment of the present invention.

FIG. 2 is a schematic diagram of a region image in an embodiment of the present invention.

FIG. 3 is a schematic diagram of the positions of key points of a face in an embodiment of the present invention.

FIG. 4 is a schematic diagram of an extreme point in an embodiment of the present invention.

FIG. 5 is a schematic diagram of a significant extreme point in an embodiment of the present invention.

FIG. 6 is a schematic diagram of acquiring a face video key frame according to an embodiment of the present invention.

Detailed ways

The embodiments of the present invention are described below through specific specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments may be combined with each other under the condition of no conflict.

It should be noted that the drawings provided in the following embodiments are only used to illustrate the basic concept of the present invention in a schematic way, so the drawings only show the components related to the present invention rather than the number, shape and number of components in actual implementation. For dimension drawing, the type, quantity and proportion of each component can be changed at will in actual implementation, and the component layout may also be more complicated.

Referring to FIG. 1 , this embodiment provides a method for detecting a living body based on a face video key frame, including steps S01-S03.

In step S01, the region image corresponding to the face in each frame of face image is obtained:

In one embodiment, before obtaining the area image corresponding to the face in each frame of the face image, the frame position of the largest face in each frame of the face image is obtained, and the area is obtained according to the corresponding bounding frame position in the multiple frames of face images. image. Specifically, a piece of face video can be collected by an image collection device such as a camera, and face detection is performed on each frame of the original image in the face video. If no face is detected in any frame of the original image, the face is collected again. face video; otherwise, save the border position of the largest face in each frame of the original image.

The largest face refers to the face with the largest pixel area in the image; in the specific implementation, the largest face in each frame of face image can be detected as the target face, or a better solution can be adopted, that is, to detect the first face first. The largest face in a frame of images is used as the target face, and then its position is tracked, which can avoid the loss or dislocation of the target face when multiple faces are detected at the same time.

In one embodiment, the position of the face bounding box can be represented as two pairs of coordinates on the two diagonal vertices of the face rectangle:

[(X ₁ ,Y ₁ ),(X ₂ ,Y ₂ )]

Among them, X and Y represent the index of the column and row in the original image pixel, respectively.

In one embodiment, referring to FIG. 2 , an envelope frame 2 including all frame positions 3 in the multi-frame image is set, and an image cropping frame 1 is set according to the geometric center of the envelope frame; each image cropping frame is obtained according to the image cropping frame. The area image of the corresponding face in the frame face image. Among them, the image cropping border can be set as a square border. Specifically, according to the position of the face bounding box corresponding to each frame of the face image, under the same pixel coordinate scale, the minimum rectangular envelope (that is, the envelope frame 2) that can contain all the face frames corresponding to the multiple frames of face images is obtained. ), and then obtain the smallest square area that can contain the smallest rectangular envelope and is centered on it (ie, image cropping frame 1), and cut each frame of the original face image into a square of the same pixel size according to the smallest square area , so as to facilitate subsequent input into the deep neural network for living body recognition.

In one embodiment, according to the position of the smallest square area used for cropping the image in the original image, the detected face border position can be transformed from the original face image coordinate space to the cropped image coordinate space through coordinate translation.

Specifically, remember the position coordinates of the face frame as:

Then the position coordinates of the minimum rectangular envelope in the original image space can be expressed as:

In general, the center of the smallest square area coincides with the center of the smallest rectangular envelope, and the side length of the smallest square area is equal to the long side of the smallest rectangular envelope. It is translated to within the bounds of the original image.

The final effect is that each frame of the original image is cropped according to a fixed square area in the picture, and the target face always appears within the cropped image area, which is convenient for subsequent image scaling as the input of the deep neural network.

When performing coordinate translation transformation, remember that the pixel coordinate origin of the original image is (0,0), and the position coordinates of the face frame before the transformation are:

If the coordinates of the origin of the pixel coordinates of the cropped image in the original image space are (X ₀ , Y ₀ ), the position coordinates of the transformed face bounding box are:

In step S02, the facial feature key points corresponding to the regional images are obtained, and the position variation of the facial key points is obtained according to the facial feature key points in the multi-frame face images:

In one embodiment, after completing the cropping of the face image, one frame of the cropped face image is selected as the reference frame, and the corresponding face feature key points in the multi-frame face images and the face feature key points in the reference frame are obtained. The position difference of gets the position change amount. Specifically, according to the position of the face frame transformed by the coordinate translation, face key point detection is performed on the regional image in each frame of the cropped image (see Figure 3 for the specific position of the face key point), and each frame of regional image The corresponding keypoint positions of all faces are merged and saved as a matrix.

According to the obtained face key point position matrix, take the regional image corresponding to the first frame of face image as the reference frame, calculate the difference between the position of the face key point of each frame in the multi-frame regional image and the position of the face key point of the reference frame, The position changes of the face key points corresponding to each frame are obtained, which are merged and saved as the face key point position change matrix.

According to the obtained face key point position change matrix, calculate the modulo of the face key point position change in each frame, obtain the face key point position change amount corresponding to each frame, and save it as a face key point position change vector.

Specifically, in the specific implementation, there are different schemes for the face key points to choose from, respectively 68 points, 19 points, 5 points, etc. (for the specific positions of the face key points, please refer to Figure 3); The number of points is L, then L can be 68 or 19 or 5, etc. The algorithm implementation of face detection and face key point detection can use existing toolkits (such as OpenCV, dlib, etc.) or other neural network models or algorithm codes that can be used for face detection or face key point detection.

The face key point position matrix can be expressed as:

与

表示在第i帧区域图像中第j个人脸关键点所对应的像素空间坐标；in,

and

Indicates the pixel space coordinates corresponding to the jth face key point in the ith frame area image;

The face key point position change matrix can be expressed as:

The face key point position change vector can be expressed as:

In step S03, the key frames of the face video are acquired according to the position change, the living body recognition is performed on the key frames of the face video, and the recognition result is output.

In one embodiment, the extreme point of the position change is acquired, and the corresponding face video key frame is acquired according to the extreme point.

In one embodiment, the number of face video key frames to be extracted can be set, and when the number of extreme points is greater than the number of face video key frames to be extracted, the extreme points are merged to obtain significant extreme points. value points, and obtain the face video key frames according to the significant extreme points; when the number of extreme points is equal to the number of face image key frames to be extracted, the face image at the time corresponding to the extreme point is directly used as the face Video key frames; when the number of extreme points is less than the number of face video key frames to be extracted, a corresponding number of face images are extracted from the face images corresponding to the non-extreme points to supplement the number of face video key frames.

Specifically, extremum points can be extracted according to the three-neighborhood or five-neighborhood principle, that is, each extremum point that meets the conditions needs to satisfy: obtain the maximum value or minimum value;

The extreme point can be written in the form of (i, ‖ΔLM‖ _i ), which means that the change vector of the key point position of the face has obtained the extremum in the ith frame, and the corresponding change is ‖ΔLM‖ _i , that is, ‖ΔLM‖ in ith element.

In one embodiment, when obtaining significant extreme points, the extreme points that meet the preset conditions are merged according to the time difference of the adjacent multiple extreme points and the position difference in the corresponding face image.

The time difference between extreme points refers to the number of frames between the corresponding moments of two extreme points;

The position difference between the extreme points refers to the difference between the position changes of the key points of the face corresponding to the two extreme points;

In the specific implementation, for the extreme point (i, ‖ΔLM‖ _i ) and the extreme point (j, ‖ΔLM‖ _j ), if

|i-j|≤α×N

and

Then the two extreme points are merged into the same cluster (where α is the scale coefficient, which can be 0.05). After all extreme points are merged, several relatively independent clusters are obtained, and only one extreme point is randomly selected from each cluster as a significant extreme point (see Figure 5 for an example).

In one embodiment, referring to FIG. 6 , when the number of significant extreme points is greater than the number of face video key frames to be extracted, the number of face video key frames to be extracted is obtained from the significant extreme points. The corresponding significant extreme points are obtained, and the corresponding face video key frames are obtained; when the number of significant extreme points is equal to the number of face video key frames to be extracted, the face at the corresponding moment of the significant extreme points is directly obtained. The image is used as a face video key frame; when the number of salient extreme points is less than the number of face video key frames to be extracted, a corresponding number of face images are extracted from the face images corresponding to the non-salient extreme points to supplement the human face. Face video keyframes. Due to the acquisition frame rate, image noise and other factors, the selected extreme points often have certain redundancy, and this part of redundancy can be removed to a large extent through the screening of significant extreme points, and finally a number of relative points are obtained. Independent significant extreme points.

In one embodiment, a living body recognition model is trained, and the recognition result is obtained according to the output probability of the living body recognition model.

Specifically, according to the extracted key frames of the face video, the regional images at the corresponding moment of each key frame are scaled to the same pixel size as the input of the living body recognition model.

Each key frame image is input into the pre-trained living body recognition model for face living body detection, and the output probability obtained by each operation is used as the output probability corresponding to each key frame, and the output probability represents the current face video key. The probability that the largest face in the frame is judged to be a real face, the value range is between 0 and 1, tending to 0 means a fake face, and tending to 1 means a real face; record the maximum and minimum values of each output probability and the average value for backup.

Judging according to the obtained maximum value, minimum value and average value of output probability, if the average value of output probability is less than the false face threshold, and the maximum value is less than the true face threshold, it will prompt a false face; if the average output probability is greater than the true face threshold, and If the minimum value is greater than the false face threshold, it indicates that it is a real face; if the above two conditions are not met, the image acquisition is performed again.

Specifically, in the specific implementation, the pixel size to which the regional image corresponding to each key frame needs to be scaled may be 224×224;

Residual network (ResNet) can be used as a deep neural network model framework for face recognition.

The size of the network input channel can be set to 224×224, which corresponds to the scaling of the image in the key frame area; the output dimension of the last fully connected layer is set to 2, corresponding to the real face and fake face in the category; finally, the network passes Softmax The operation calculates the probability that the currently input face image is judged to be a real face; the convolution kernel size, step size, number of channels, pooling kernel size, step size, fully connected layer dimension, activation function and other parameters of other layers can be It can be adjusted flexibly according to the experimental effect.

The convolutional neural network framework used for face recognition can also be replaced by pre-trained network structures such as LeNet, AlexNet, GoogLeNet, and VGG;

Before the actual test, the network needs to use a certain amount of pre-annotated face video keyframes for model training.

The model operation for each face video key frame can be performed in parallel, which can be achieved by pre-initializing K models that are identical or share network weights with each other.

Let the fake face threshold be Th _fake , the real face threshold be Th _real , the maximum output probability corresponding to each key frame is P _max , the minimum value is P _min , and the average value is P _avg :

If P _avg <Th _fake , and P _max <Th _real , it is determined to be a fake face;

If P _avg >Th _real , and P _min >Th _fake , it is determined as a real face;

If the above two conditions are not met, it will be regarded as pending, and the face video needs to be collected again for judgment.

In a specific implementation, the fake face threshold Th _fake used for determination can be set to 0.4, and the real face threshold Th _real can be set to 0.6.

To sum up, the present invention proposes a method for detecting a living body based on key frames of face video, which overcomes the defects of traditional video sampling strategies, and can better extract key frames containing significant discriminant information in video clips, reducing the need for Compared with the general single-frame model, the multiple discrimination results of the input face key frames can be combined, which can improve the recognition accuracy of the single-frame living detection model to a certain extent. The performance is also more robust; compared with the general multi-frame model, splitting each key frame and using the single-frame model for parallel computing can reduce the model's computing time to a certain extent and improve the detection efficiency; To a certain extent, the living body recognition model used can be replaced by the vast majority of face living body recognition algorithms or networks based on single-frame images. Therefore, the present invention effectively overcomes various shortcomings in the prior art and has high industrial utilization value.

The above-mentioned embodiments merely illustrate the principles and effects of the present invention, but are not intended to limit the present invention. Anyone skilled in the art can make modifications or changes to the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or changes made by those with ordinary knowledge in the technical field without departing from the spirit and technical idea disclosed in the present invention should still be covered by the claims of the present invention.

Claims (10)

1. A living body detection method based on face video key frames is characterized by comprising the following steps:

acquiring a region image corresponding to a face in each frame of face image;

acquiring face feature key points corresponding to the region images, and acquiring the position variation of the face key points according to the face feature key points in the multi-frame face images;

and acquiring a face video key frame according to the position variation, performing living body identification on the face video key frame, and outputting an identification result.

2. The live body detection method based on the key frame of the human face video as claimed in claim 1, wherein before the region image of the corresponding human face in each frame of the human face image is obtained, the position of the bounding box of the largest human face in each frame of the human face image is obtained, and the region image is obtained according to the position of the corresponding bounding box in the plurality of frames of the human face image.

3. The live body detection method based on the key frame of the human face video, according to claim 2, characterized by setting an envelope border containing all the border positions in a multi-frame image, and setting an image clipping border according to the geometric center of the envelope border;

and obtaining a region image corresponding to the face in each frame of face image according to the image cutting frame.

4. The live body detection method based on the key frame of the human face video, according to claim 3, characterized in that the image cropping frame comprises a square frame.

5. The living body detection method based on the face video key frame as claimed in claim 1, wherein a frame of face image is selected as a reference frame, and the position variation is obtained by obtaining the position difference between the corresponding face feature key point in the face images of multiple frames and the face feature key point in the reference frame.

6. The living body detection method based on the face video key frame as claimed in claim 1, wherein an extreme point of the position variation is obtained, and the corresponding face video key frame is obtained according to the extreme point.

7. The living body detection method based on the face video key frames according to claim 6, characterized in that the number of the face video key frames to be extracted is set, when the number of the extreme points is larger than the number of the face video key frames to be extracted, merging processing is performed on the extreme points to obtain significant extreme points, and the face video key frames are obtained according to the significant extreme points; when the number of the extreme points is equal to the number of the face video key frames to be extracted, directly taking the face images at the moment corresponding to the extreme points as the face video key frames; and when the number of the extreme points is less than the number of the face video key frames to be extracted, extracting a corresponding number of face images from the face images corresponding to the non-extreme points to complement the face video key frames.

8. The living body detection method based on the face video key frame as claimed in claim 7, wherein when the significant extreme point is obtained, the extreme points meeting the preset condition are merged according to the time difference between the adjacent extreme points and the position difference in the corresponding face image.

9. The living body detection method based on the face video key frames according to claim 8, wherein when the number of the significant extreme points is larger than the number of the face video key frames to be extracted, the significant extreme points corresponding to the number of the face video key frames to be extracted are obtained from the significant extreme points, and the corresponding face video key frames are obtained; when the number of the significant extreme points is equal to the number of the face video key frames to be extracted, directly acquiring the face images corresponding to the significant extreme points as the face video key frames; and when the number of the significant extreme points is less than the number of the face video key frames to be extracted, extracting a corresponding number of face images from the face images corresponding to the non-significant extreme points to complement the face video key frames.

10. The living body detection method based on the human face video key frame as claimed in claim 1, characterized in that a living body recognition model is trained, and the recognition result is obtained according to the output probability of the living body recognition model.

Images