CN117854160A - Human face living body detection method and system based on artificial multi-mode and fine-granularity patches - Google Patents

Abstract

The invention provides a human face living body detection method and a human face living body detection system based on artificial multi-mode and fine-granularity patches. The invention adopts the patch cut out from the facial image to identify the local characteristics, thereby improving the detection accuracy. The invention adopts the classification loss and the self-supervision similarity loss based on the asymmetric edges to standardize the feature embedding space, thereby improving the stability and the reliability of detection.

Description

A method and system for face liveness detection based on artificial multimodality and fine-grained patches

Technical Field

The present invention belongs to the field of face recognition, and in particular relates to a face liveness detection method and system based on artificial multimodality and fine-grained patches.

Background technique

In recent years, face recognition technology has been widely used in daily life, especially in the fields of smartphone login, access control, mobile payment, etc. However, with the popularity of social media, the risk of personal face photos being leaked has increased. Attackers can easily create photos, videos or 3D masks of target victims. This attack seriously threatens the reliability and security of face recognition systems. Therefore, with the further maturity of face recognition technology, face liveness detection has become a key link in ensuring the security and reliability of face recognition systems, and has important application value.

At present, the closest existing technology is the face liveness detection technology based on deep learning. This technology mainly identifies whether the face is alive by performing deep learning on the face image.

Although existing face liveness detection technologies can guarantee security to a certain extent, they have many limitations. These technologies usually adopt a responsive approach to capture image information of a single modality and require users to cooperate in completing specified random actions, such as nodding, shaking their heads, opening their mouths, etc. By inputting multiple frames of continuous images, dynamic spatiotemporal features are extracted for detection. However, this approach is not applicable under non-cooperative deployment conditions that require rapid decision-making. In addition, if the attacker pre-records a video of the corresponding action or wears a fake headgear, liveness detection can also be passed. Therefore, non-cooperative face liveness detection methods have more practical application value and advantages, and can achieve faster and more effective detection. In addition, methods based on a single modality cannot utilize data from other modalities during the training phase, resulting in the discarding of complementary information between modalities, which affects the detection performance.

In addition, with the popularity of consumer-grade sensors such as depth cameras, some emerging liveness detection methods have begun to emerge. These methods use multimodal information such as depth images and infrared images to extract complementary features, and fuse the features of multiple modalities to improve detection accuracy. However, multimodal face liveness detection has the following limitations, which restrict its widespread use in certain application scenarios: 1) It is difficult to obtain multimodal face images. Each modality requires a corresponding sensor to capture, and the high cost of near-infrared and depth information sensors makes it difficult to obtain corresponding modality data; 2) Multimodal sensors are difficult to integrate and difficult to be widely deployed on mobile terminals.

Summary of the invention

The purpose of the present invention is to address the deficiencies in the prior art.

The technical solution adopted by the present invention is specifically as follows:

A human face liveness detection method based on artificial multimodality and fine-grained patches, comprising:

Collecting a sequence of color images of the target area;

Perform facial key point detection on the color image sequence to obtain the initial face frame;

Based on the acquired initial face frame and face key point information, the collected color image sequence is aligned and cropped to obtain a valid face image sequence;

Extracting several modalities based on a valid face image sequence, including one or more of a moiré modality, a depth map modality, a dense optical flow modality, and a temporal pooling modality;

Input a valid face image sequence and several modalities into a pre-trained face liveness detection model to obtain a detection result; the face liveness detection model includes several artificial modality feature extraction modules, a CBAM attention module, a cropping module, a fusion feature extraction module and a fully connected layer; wherein the several artificial modality feature extraction modules correspond to the valid face image sequence and several modalities one by one, and are respectively used to extract features from the valid face image sequence and several modalities; the CBAM attention module fuses the features output by the several artificial modality feature extraction modules; the cropping module is used to randomly horizontally flip and randomly rotate the fusion features output by the CBAM attention module and then crop them into multiple face blocks of fixed size; the fusion feature extraction module is used to extract features from multiple face blocks respectively, and the fully connected layer is used to output the detection result based on the features output by the fusion feature extraction module;

The face liveness detection model is trained based on the collected data set by minimizing the loss function. Each sample of the data set contains a valid face image sequence and several modalities corresponding to the color image sequence collected by N patch types, as well as the corresponding real labels; the N patch types are composed of k real face classes and N-k deception classes; the loss function includes the classification loss of asymmetric edges and the self-supervised similarity loss. Among them, the real face class is the class corresponding to the real face collected by the image collection device under different parameters, such as different capture resolutions; the deception class is the class corresponding to the face containing different deception media collected by the image collection device under different parameters.

Furthermore, a pre-trained color image face detection model is used to detect face candidate frames and face key points on the color image sequence to obtain initial face frame and face key point information; the color image face detection model is constructed based on the SCRFD face detection algorithm.

Furthermore, the moiré modality is extracted from a valid face image sequence using a pre-trained MoireDet network.

Furthermore, the depth map modality is extracted from a valid face image sequence using a pre-trained PRNet network.

Furthermore, the method for extracting the dense optical flow modality is as follows:

A pyramid-based optical flow algorithm is used to calculate the motion vector of each pixel between the first frame and the last frame of a valid face image sequence, and the dense optical flow modality is obtained by calculating the size and direction of the motion vector.

Furthermore, the extraction method of the time pooling mode is as follows:

The Rank Pooling method is used to encode the valid face image sequence, and the valid face image sequence is converted into a dynamic/time-series image to obtain the time pooling mode.

Furthermore, the artificial modality feature extraction module adopts ResNet18, Light CNN, Xception or MobileNet.

Furthermore, the classification loss of asymmetric margins is:

Where, the subscript i represents the i-th sample, _ml and _ms are the angle boundaries of the live and spoof classes, is the patch feature corresponding to the t-th face patch extracted by the fusion feature extraction module, t∈p, p is the number of face patches cropped by the cropping module, _Wj is the j-th column of the fully connected layer, _yi is the true label of the ith sample,/> is the value of the corresponding column of _yi in the fully connected layer, s is a hyperparameter used to scale the value, /> are any two different patch features from the same sample, t ₁ ,t ₂ ∈P, n represents the number of samples; L is the set of k real face classes, S is the set of Nk deception classes;

The self-supervised similarity loss is:

|*| ₂ indicates the L2 norm.

A human face liveness detection system based on artificial multimodality and fine-grained patches is used to implement a human face liveness detection method based on artificial multimodality and fine-grained patches, comprising:

Acquisition camera: used to acquire color image sequences of the target area.

Image detection module: including a color face detection unit and an image preprocessing unit; wherein the color face detection unit is used to detect faces and obtain initial face frames and face key point information; the image preprocessing unit is used to perform face alignment on the collected color image sequence based on the face key point information;

Image cropping module: crops the aligned color image sequence based on the obtained initial face frame to obtain a valid face image sequence;

Image artificial modality generation module: extracts several modalities based on valid face image sequences;

Liveness judgment module: includes a pre-trained face liveness detection model, which is used to input a valid face image sequence and several modalities into the pre-trained face liveness detection model to obtain the detection result.

Furthermore, the image artificial modality generation module comprises:

A moiré pattern generation unit, used for generating a moiré pattern based on a valid face image sequence;

A depth map modality generating unit, used for generating a depth map modality based on a valid face image sequence;

A dense optical flow modality generation unit, used to generate a dense optical flow modality based on a valid face image sequence;

The temporal pooling modality generation unit is used to generate a temporal pooling modality based on a valid face image sequence.

The beneficial effects of the present invention are:

1. The present invention artificially creates dense optical flow modality, temporal pooling modality, moiré modality and depth map modality from RGB images and fuses these artificial modalities, aiming to solve the problem of decreased detection accuracy of single RGB images, while avoiding the high deployment cost of using additional sensors to collect multimodal information.

2. The existing technology often ignores the local features in the face image (i.e. the device that collects the image and the material that deceives the image). Therefore, the present invention uses patches cropped from the facial image to identify local features, thereby improving the accuracy of detection.

3. In order to improve the generalization ability of the network to extract deceptive features so that the present invention can be widely used in some unknown attacks, the present invention adopts asymmetric edge-based classification loss and self-supervised similarity loss to standardize the feature embedding space, thereby improving the stability and reliability of detection.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a flow chart of a method for face liveness detection based on artificial multimodality and fine-grained patches according to the present invention;

FIG2 is a structural diagram of a face liveness detection system based on artificial multimodality and fine-grained patches according to the present invention.

Detailed ways

Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

The terminology used in the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application.

As used in this application and the appended claims, the singular forms "a", "an", "said", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

It should be understood that although the terms first, second, third, etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

The present invention provides a method for face liveness detection based on artificial multimodality and combined with fine-grained patches, comprising the following steps:

S1: Collect color image sequences of the target area;

In one embodiment, a color image of the target area is collected by a collection device for a total of 2 seconds. Here, the collection device uses an RGB camera to capture color images. The acquisition frequency of the color images can be the same or different, depending on the functional requirements. For example, the color images can be collected at a frequency of 60FPS.

S2: Perform facial key point detection on the color image sequence to obtain the initial face frame and facial key point information.

Specifically, the color image is transmitted to a pre-trained color image face detection model for facial key point detection. In one embodiment, the color image face detection model is constructed based on the SCRFD face detection algorithm, and includes two key parts: (1) Sample Redistribution (SR), which adds training samples at the most needed stage based on the statistics of the benchmark data set; (2) Computing Redistribution (CR), which redistributes the calculation between the trunk, neck and head of the model based on a carefully defined search method. Using these two points, samples and computing resources are allocated to where they are most needed, thereby achieving a balance between high efficiency and high accuracy. Specifically, the color image face detection model includes a feature extraction network, a path aggregation feature pyramid network, a head convolution layer, and an output layer; its steps include:

S20, the color image is transmitted to a feature extraction network with ResNet34 as the backbone, wherein the number of channels output by each layer is 75% of the original one, and the final output feature map has 48 channels;

S21, input the feature map extracted by the feature extraction network into the path aggregation feature pyramid network (PAFPN) to obtain an effective feature fusion layer, and the final output feature map has 24 channels;

S22, input the acquired effective feature fusion layer into the head convolution layer, the head convolution layer includes three 3×3 convolution layers, and the final output feature map has 96 channels;

S23, the output layer performs face prediction based on the feature map output by the head convolution layer to obtain the initial face frame and face key point information.

The output layer contains two prior frames, each of which represents a certain area on the color image. Face detection is performed on each prior frame. By setting the confidence threshold to 0.5, the probability of whether the prior frame contains a face is predicted and compared with the threshold. If the probability of the prior frame is greater than the threshold, the prior frame contains a face and is the initial face frame.

S3: Based on the initial face frame and face key point information obtained in step S2, face alignment and cropping are performed on the image collected based on step S1 to obtain a valid face image sequence, thereby reducing the influence of factors such as lighting, color, and complex background on face liveness detection, thereby further improving the accuracy of face liveness detection.

S4: Generate artificial modalities for the valid face image sequence obtained in step S3, including one or more of moiré modality, depth map modality, dense optical flow modality, and time pooling modality; in one embodiment, specifically including the above four, the following steps are included:

S41: Moiré pattern generation, using the pre-trained MoireDet network (Yang C, Yang Z, Ke Y, et al. Doing More With Moiré Pattern Detection in Digital Photos [J]. IEEE Transactions on Image Processing, 2023, 32: 694-708) to extract the corresponding moiré pattern from the valid face image sequence. The MoireDet network uses three encoders, namely the macro encoder (High-Level Encoder), the micro encoder (Low-Level Encoder) and the spatial encoder (Spatial Encoder) to capture high-level contextual features (i.e., moiré effects) and low-level structural features (i.e., fine-grained stripes, ripples, and curves) in the image. The macro encoder uses ResNet18 as the backbone network and is connected to two BiFPN layers to repeatedly encode moiré features. The BiFPN layer allows multi-scale feature fusion from top to bottom and bottom to top, so that the background features of high-level moiré can be effectively encoded. The micro encoder uses the early layers of ResNet18 (the first two blocks) and uses the attention of the BiFPN output to enhance the low-level features generated by stripes, ripples, and curves that contribute to the moiré pattern. Since the attention of BiFPN contains rich high-level contextual features, the early layers in ResNet18 can better locate and capture these necessary structural features. The spatial encoder contains a convolution kernel of size 5×5 with adaptive weights, where the weights are calculated by two 1×1 convolutions. This adaptive convolution kernel only calculates activations in specific areas. Finally, the feature maps from the three encoders are concatenated to estimate the moiré edge map.

S42: Depth map modality generation, using the pre-trained PRNet network (Feng Y, Wu F, Shao X, et al. Joint 3d face reconstruction and dense alignment with position map regression network [C] // Proceedings of the European conference on computer vision (ECCV). 2018: 534-551) to extract the corresponding depth map from the valid face image sequence. Specifically, the PRNet network adopts an encoder-decoder structure. The encoder part starts with a convolution layer, followed by 10 residual blocks, which reduce the 256×256×3 input image to 8×8×512 feature maps, and the decoder part contains 17 transposed convolution layers to generate a predicted 256×256×3 position map. Use kernel size 4 for all convolutional or transposed convolutional layers and use ReLU layers for activation.

S43: Dense optical flow modality generation, using the first frame and the last frame of the valid face image sequence to calculate the optical flow. Specifically, the pyramid-based optical flow algorithm is used to process the consecutive frame images to calculate the motion vector of each pixel between the two frames. By calculating the size and direction of the motion vector, the dense optical flow modality can be obtained.

S44: Temporal pooling modality generation. This modality uses the Rank Pooling method to encode the valid face image sequence and converts the valid face image sequence into a dynamic/time-series image, thereby capturing the process of frame-level features evolving over time and obtaining the temporal pooling modality.

S5: Input the valid face image sequence and several modalities into a pre-trained face liveness detection model to obtain the detection result; the face liveness detection model includes several artificial modality feature extraction modules, CBAM attention module, cropping module, fusion feature extraction module and full connection layer; wherein the several artificial modality feature extraction modules correspond to the valid face image sequence and several modalities one by one, and are respectively used to extract features from the valid face image sequence and several modalities; the CBAM attention module fuses the features output by the several artificial modality feature extraction modules; the cropping module is used to randomly horizontally flip and randomly rotate the fusion features output by the CBAM attention module and then crop them into multiple face blocks of fixed size; the fusion feature extraction module is used to extract features from multiple face blocks respectively, and the full connection layer is used to output the detection result based on the features output by the fusion feature extraction module; wherein the specific steps of the face liveness detection model detection include:

S51: Input the valid face image sequence, moiré, depth, dense optical flow and time pooling mode into the corresponding five artificial modality feature extraction modules respectively. In this embodiment, ResNet18 is used as the artificial modality feature extraction module for feature extraction. In addition, structures such as Light CNN, Xception or MobileNet can also be used.

S52: Use the CBAM attention module to fuse the five extracted feature maps. The fused feature maps are input to the cropping module for random horizontal flipping and random rotation, and then cropped into several face blocks of a fixed size (e.g., 160×160). Such processing can enhance the robustness of the model and make it more adaptable to various changes.

S53: The face blocks are respectively sent to a fusion feature extraction module (such as ResNet18) for feature extraction and normalization, and then input to a fully connected layer for liveness judgment and output the judgment result.

The face liveness detection model is trained based on the collected data set by minimizing the loss function, wherein each sample of the data set is subdivided into categories according to the provided materials and capture devices. Each sample contains a valid face image sequence and several modalities corresponding to the color image sequence collected by N patch types, as well as the corresponding true labels; the N patch types are composed of k real face classes and N-k deception classes; for example, for the CASIA-FASD data set, there are two different deception media and three different capture resolutions, so there are a total of nine fine-grained patches (three liveness types (corresponding to real faces obtained at three different capture resolutions without deception media) and six deception types).

For the loss function, an asymmetric margin classification loss is used to make the feature clustering distribution of each category more compact and have better generalization ability. In addition, since the distribution difference between deceptive samples is greater than that of live samples, asymmetric treatment is performed on live and deceptive samples: forcing the model to learn a more compact clustering in live samples while making deceptive samples more dispersed in the feature space. At the same time, a self-supervised similarity loss is used to apply the positive part of the contrast loss to the two transformed patch views of a single face image, thereby further normalizing the patch features. Such processing can enhance the robustness of the model and make it better adaptable to various changes.

Specifically, assume that there are N patch-type classes in the training set, which consists of k real face classes and Nk spoof classes. The true label of each input sample belongs to a fine-grained class y _i ∈ L ₁ , L ₂ , … L _k , S ₁ , S ₂ , … S _Nk . The set of live classes of real faces is L = {L ₁ , L ₂ , … L _k }, and the set of spoof classes is S = {S ₁ , S ₂ , … S _Nk };

The classification loss for asymmetric margins is:

Where, the subscript i represents the i-th sample, _ml and _ms are the angle boundaries of the live and spoof classes, which are adjustable parameters. is the input of the fully connected layer for classification, i.e., the patch features corresponding to the t-th face patch extracted by the fusion feature extraction module, t∈P, P is the number of face patches cropped by the cropping module, _Wj is the j-th column of the fully connected layer, indicating the predicted probability of the j-th class, and _yi is the true label of the ith sample. /> is the value of the corresponding column of _yi in the fully connected layer. /> is the logistic regression of the patch feature corresponding to the i-th sample, s is a hyperparameter used to scale the value, /> are any two different patch features from the same sample, t ₁ , t ₂ ∈P, and n represents the number of samples.

The self-supervised similarity loss is:

|*| ₂ indicates the L2 norm.

Face liveness detection can be performed based on the trained face liveness detection model; for a given test face image, the valid face image sequence and several modalities are input into a pre-trained face liveness detection model. After feature extraction and fusion, the cropping module evenly crops face patches of the same size as those in the training process from the entire fused feature image for network inference. Assuming that P face patches are cropped from a face image, the fusion feature extraction module extracts the corresponding P patch features (f ¹ ,f ² ,…,f ^p ), then the average liveness probability can be obtained by the sum of the predicted probabilities of the liveness categories in the last fully connected layer:

In one embodiment, the live probability threshold is pre-set to 0.6. If the probability detected by the face liveness detection model is greater than the threshold, the face in the face image is alive and the liveness judgment result is output; if it is less than the threshold, the operation is terminated. It should be understood that the pre-set liveness probability can be set according to actual conditions and is not limited here.

FIG2 shows a schematic diagram of a face liveness detection system based on artificial multimodality and fine-grained patches. The system 200 includes the following parts:

Acquisition camera 201: used to acquire a color image sequence of a target area.

Image detection module 202: includes a color face detection unit 2021 and an image preprocessing unit 2022. The color face detection unit 2021 is used to detect faces and obtain initial face frames and face key point information. The image preprocessing unit 2022 is used to perform face alignment on the initial face frames and face key point information to obtain an effective processed image, reduce the influence of factors such as lighting, color, and complex background on face liveness detection, and further improve the accuracy of face liveness detection.

Image cropping module 203: used to crop the color image corresponding to the aligned valid processed image to obtain different valid face image sequences.

Image artificial modality generation module 204: used to generate different artificial modalities from the original RGB modality, including a moiré modality generation unit 2041, a depth map modality generation unit 2042, a dense optical flow modality generation unit 2043, and a time pooling modality generation unit 2044.

The liveness judgment module 205 includes a pre-trained face liveness detection model, which is used to input a valid face image sequence and several modalities into the pre-trained face liveness detection model, perform liveness judgment and output a liveness judgment result.

The system improves the accuracy and robustness of face liveness detection by means of artificial multimodality and fine-grained patches. Obviously, the above embodiments are only examples for clear explanation and are not limitations on the implementation methods. For ordinary technicians in the relevant field, other different forms of changes or modifications can be made on the basis of the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived from this are still within the scope of protection of the present invention.

Claims (10)

1. A method for face liveness detection based on artificial multimodality and fine-grained patches, comprising: Collecting a sequence of color images of the target area; Perform facial key point detection on the color image sequence to obtain the initial face frame; Based on the acquired initial face frame and face key point information, the collected color image sequence is aligned and cropped to obtain a valid face image sequence; Extracting several modalities based on a valid face image sequence, including one or more of a moiré modality, a depth map modality, a dense optical flow modality, and a temporal pooling modality; Input a valid face image sequence and several modalities into a pre-trained face liveness detection model to obtain a detection result; the face liveness detection model includes several artificial modality feature extraction modules, a CBAM attention module, a cropping module, a fusion feature extraction module and a fully connected layer; wherein the several artificial modality feature extraction modules correspond to the valid face image sequence and several modalities one by one, and are respectively used to extract features from the valid face image sequence and several modalities; the CBAM attention module fuses the features output by the several artificial modality feature extraction modules; the cropping module is used to randomly horizontally flip and randomly rotate the fusion features output by the CBAM attention module and then crop them into multiple face blocks of fixed size; the fusion feature extraction module is used to extract features from multiple face blocks respectively, and the fully connected layer is used to output the detection result based on the features output by the fusion feature extraction module; The face liveness detection model is trained based on the collected data set by minimizing the loss function. Each sample of the data set contains a valid face image sequence and several modalities corresponding to the color image sequence collected by N patch types, as well as the corresponding real labels; the N patch types are composed of k real face classes and N-k deception classes; the loss function includes the classification loss of asymmetric edges and the self-supervised similarity loss. 2. The method according to claim 1 is characterized in that a pre-trained color image face detection model is used to detect face candidate frames and face key points on a color image sequence to obtain initial face frames and face key point information; the color image face detection model is constructed based on the SCRFD face detection algorithm. 3. The method according to claim 1 is characterized in that the moiré modality is extracted from a valid face image sequence using a pre-trained MoireDet network. 4. The method according to claim 1 is characterized in that the depth map modality is extracted from a valid face image sequence using a pre-trained PRNet network. 5. The method according to claim 1, characterized in that the method for extracting the dense optical flow modality is as follows: A pyramid-based optical flow algorithm is used to calculate the motion vector of each pixel between the first frame and the last frame of a valid face image sequence, and the dense optical flow modality is obtained by calculating the size and direction of the motion vector. 6. The method according to claim 1, characterized in that the method for extracting the temporal pooling modality is as follows: The Rank Pooling method is used to encode the valid face image sequence, and the valid face image sequence is converted into a dynamic/time-series image to obtain the time pooling mode. 7. The method according to claim 1 is characterized in that the artificial modality feature extraction module adopts ResNet18, Light CNN, Xception or MobileNet. 8. The method according to claim 1, characterized in that the classification loss of the asymmetric edge is: Wherein, subscript i denotes the i-th sample, _ml and _ms are the angle boundaries of the live and spoofing classes, _fit ^is the patch feature corresponding to the t-th face patch extracted by the fusion feature extraction module, t∈P, P is the number of face patches cropped by the cropping module, _Wj is the j-th column of the fully connected layer, _yi is the true label of the i-th sample, is the value of the corresponding column of _yi in the fully connected layer, s is a hyperparameter used to scale the value, /> are any two different patch features from the same sample, t ₁ ,t ₂ ∈P, n represents the number of samples; L is the set of k real face classes, S is the set of Nk deception classes; The self-supervised similarity loss is: |*| ₂ indicates the L2 norm. 9. A face liveness detection system based on artificial multimodality and fine-grained patches, used to implement a face liveness detection method based on artificial multimodality and fine-grained patches according to any one of claims 1 to 8, characterized in that it comprises: Acquisition camera (201): used to acquire a color image sequence of a target area. The image detection module (202) comprises a color face detection unit (2021) and an image preprocessing unit (2022); wherein the color face detection unit (2021) is used to detect a face and obtain an initial face frame and face key point information; and the image preprocessing unit (2022) is used to perform face alignment on a collected color image sequence based on the face key point information. Image cropping module (203): cropping the aligned color image sequence based on the acquired initial face frame to obtain a valid face image sequence; Image artificial modality generation module (204): extracting several modalities based on a valid face image sequence; The liveness determination module (205) comprises a pre-trained human face liveness detection model, which is used to input a valid human face image sequence and a plurality of modalities into the pre-trained human face liveness detection model to obtain a detection result. 10. The system of claim 9, wherein the image artificial modality generation module (204) comprises: A moiré pattern generation unit (2041), used for generating a moiré pattern based on a valid face image sequence; A depth map modality generating unit (2042), configured to generate a depth map modality based on a valid face image sequence; A dense optical flow modality generating unit (2043), used for generating a dense optical flow modality based on a valid face image sequence; The temporal pooling modality generating unit (2044) is used to generate a temporal pooling modality based on a valid face image sequence.