CN105243386A - Face liveness judgment method and system - Google Patents

Abstract

One aspect of the present invention provides a face living body judgment method for distinguishing a real face from a spoofed face, including: acquiring a certain number of face images; acquiring an eye image of each face image and the characteristics of the eye image; predicting the sight line according to the characteristics of the obtained eye images and the sight line estimation model to obtain the positions of the sight lines with corresponding quantity; quantizing the sight line positions of the corresponding number according to a sight line dictionary to establish a sight line histogram; and obtaining information entropy according to the sight line histogram and carrying out living body judgment, wherein if the entropy value is larger than 0, the real face is judged, and otherwise, the deceptive face is judged. The human face living body judgment method provided by the invention is a human face living body judgment method which does not need additional equipment or user cooperation, and can effectively distinguish a real human face from a deceptive human face due to the characteristics that the visual line behavior is uncertain and is difficult to be acquired by other people through a camera and the like.

Description

Face liveness judgment method and system

technical field

The invention relates to the technical field of identity authentication, in particular to a method and system for judging human face livingness.

Background technique

Face recognition technology is an effective identity authentication technology, but with the widespread use of face recognition technology, there have been various methods of posing as faces to pass identity authentication. The method is called "face attack". Common forms of facial attacks include deceptive identification with photos, video playback, or 3D models to pass identity verification. Among them, playing photos and videos is the most common attack method, and people can easily obtain relevant information of legitimate users from mobile devices or surveillance cameras. Moreover, with the development of modern technology, the synthesis of 3D faces and the acquisition of models are no longer difficult to achieve. For example, the service of ThatsMyFace.com can achieve 3D face recognition by uploading a frontal and a side photo. Reconstruction and customization of 3D models. Therefore, for the sake of information security, the function of adding liveness detection to the face recognition system has received more and more attention.

People analyze common face attack methods and think that compared with real faces, photo faces are flat, and there are problems such as quality loss and blurring caused by secondary acquisition; video faces have LCD reflections and other phenomena; 3D model face motion is rigid motion etc. These are the weaknesses of the current face attack methods. Therefore, corresponding to these weaknesses, the current liveness detection methods in face recognition are mainly divided into three categories: motion-based methods, texture-based methods, and fusion methods. .

The motion-based method is mainly to analyze the motion trend of the image frame sequence, such as estimating the motion trend of different areas of the face through the optical flow method to distinguish real faces from photo attacks, or using the optical flow method to judge by estimating the rigid body motion of the object Various methods such as whether it is alive or not. In addition to involuntary head movements, other biological behaviors of legitimate users are also used to judge whether they are alive, such as blinking behavior. The texture-based method is based on the micro-texture changes such as quality degradation and blurring caused by deceiving faces in the process of secondary acquisition, and performs liveness judgment by finding out the salient distinguishing features of single-frame real faces and deceptive faces.

However, although the above two methods can resist face attacks to a certain extent, there are still many problems. For example, motion-based methods can resist photo attacks, and are basically ineffective against distorted photos and video attacks. Texture-based methods can effectively learn discriminative models for real and spoofed faces, however, they lose the correlation information between image frames.

In addition, there are currently liveness detection methods based on multi-mode and multi-spectrum. Liveness detection based on multi-mode is to combine face and other biological features, such as posture, voice, fingerprint and hand shape, etc. for identity authentication. In general, biometric authentication systems based on multimodality are more resistant to attacks than authentication systems based on single features. The liveness detection method based on multi-spectrum distinguishes real faces and spoofed faces by using the difference in reflectivity in near-infrared. Although these two types of methods have achieved better performance in live detection, these methods require additional equipment, and even require the cooperation of users in a multi-mode live detection system, which increases the burden on users.

Contents of the invention

In view of the above problems, the present invention provides a human face liveness judgment method for distinguishing real faces and fake faces in face recognition, which requires neither additional equipment nor user cooperation. , which can effectively distinguish real faces from spoofed faces.

One aspect of the present invention provides a method for judging the living body of a human face, which is used to distinguish a real human face from a spoofed human face in human face recognition, which includes: the step of obtaining a certain number of human face images; obtaining the eyes of each human face image The steps of the features of the image and the eye image; the step of predicting the line of sight according to the features of the acquired eye image and the line of sight estimation model to obtain the corresponding number of line-of-sight positions; quantifying the corresponding number of line-of-sight positions according to the line-of-sight dictionary, and establishing the line-of-sight histogram Steps; and the step of obtaining information entropy according to the line-of-sight histogram and performing living body judgment, wherein if the entropy value is greater than 0, it is judged as a real face, otherwise it is judged as a deceptive face.

The human face liveness judgment method provided by the present invention is a face liveness judgment method that requires neither additional equipment nor user cooperation, and due to the uncertainty of line of sight behavior and the characteristics that it is difficult for others to obtain it through a camera, etc., It can effectively distinguish real faces from spoofed faces.

Another aspect of the present invention provides a living body judgment system, which includes: an image acquisition unit, which is used to collect a certain number of human face images; a feature extraction unit, which is used to obtain an eye image of each human face image and the eye image The feature of the line of sight estimation unit, which performs line of sight prediction according to the characteristics of the acquired eye image and the line of sight estimation model to obtain the corresponding number of line of sight positions; the line of sight statistical unit quantifies the corresponding number of line of sight positions according to the line of sight dictionary, and establishes a line of sight histogram , and a living body judging unit, which obtains information entropy according to the line-of-sight histogram and conducts living body, wherein if the entropy value is greater than 0, it is judged as a real face, otherwise it is judged as a deceptive face.

The living body judgment system provided by the present invention can effectively distinguish between real human faces and deceptive human faces by taking advantage of the uncertainty of line of sight behavior and the characteristics that it is difficult for others to obtain through a camera.

Description of drawings

Fig. 1 is a flowchart of a method for establishing a line of sight estimation model according to an embodiment of the present invention;

Fig. 2 is a data acquisition system for establishing a line of sight estimation model according to an embodiment of the present invention;

Fig. 3 is a schematic diagram of obtaining eye images and features according to an embodiment of the present invention;

Fig. 4 is a flow chart of a method for acquiring eye images and features according to an embodiment of the present invention;

Fig. 5 is a flow chart of a method for judging human face livingness according to an embodiment of the present invention;

Fig. 6 is a schematic diagram of constructing a sight dictionary according to an embodiment of the present invention;

Fig. 7 is a schematic structural diagram of a human face living body judging system according to an embodiment of the present invention;

Fig. 8 is a schematic diagram of the detailed structure of the feature extraction unit of the human face liveness judgment system shown in Fig. 7 .

detailed description

In order to enable those skilled in the art to better understand the technical solution of the present invention, the method and system for judging human face livingness provided by the present invention will be described in detail below in conjunction with the accompanying drawings and specific embodiments. In these drawings, the same reference numerals are assigned to the same or corresponding components. The following are only the best implementations of the method and system for judging human face life of the present invention, and the present invention is not limited to the following structures.

The human face liveness judgment method and system of the present invention are used for distinguishing real human faces and spoofed human faces in human face recognition. Due to the uncertainty of the attention transfer process, gaze behavior is a kind of biological information that is difficult to be obtained by others through surveillance cameras and other equipment. At the same time, compared with spoofed faces, the gaze movement of real faces has greater uncertainty. , so the present invention introduces the line of sight into the living body judgment problem, which can effectively distinguish real faces from spoofed faces. The basic idea of the method and system for judging human face livingness of the present invention is to obtain eye images and features, and estimate line of sight according to eye image features and line of sight estimation models, and then quantify and count the predicted line of sight, and then perform living body judgment, that is, judge Whether the recognized object is a real face or a spoofed face.

For this reason, the present invention first needs to establish a suitable line of sight estimation model, which can be used to detect line of sight changes well. The existing line of sight estimation models are mainly: appearance-based methods and feature-based methods. Feature-based methods need to extract local features of the eye (such as iris outline, pupil, and bright spots, etc.), and establish a mapping between features and viewpoints. relation. However, such methods generally require a high-definition camera or even multiple light sources or multiple cameras. The appearance-based method first locates the eye area, and then directly establishes the mapping relationship between the eye image and the coordinates of the viewpoint on the screen, making full use of the information of the eye sight. Considering that the human face liveness judgment method proposed in the present invention is carried out under natural light environment and does not rely on additional equipment or light sources (actually, there is only one USB camera with a resolution of 640×480 pixels), so this embodiment Choose an appearance-based approach to model gaze directions. In addition, considering the difference in microtexture between real faces and spoofed faces, the present invention first extracts local texture features from eye images, and then establishes a mapping relationship between local texture feature vectors and viewpoints through a regression model. In this embodiment, the line of sight estimation model is mainly established by letting the user look at nine viewpoints on the computer screen in turn, thereby obtaining user frontal image data in nine sight directions, and establishing the user's eye image features and computer screen through a regression model. Viewpoint mapping relationship. The establishment and solution of the line-of-sight estimation model of the present invention will be described below in conjunction with FIGS. 1 to 4 .

Fig. 1 is a flow chart of a method for establishing a line of sight estimation model according to an embodiment of the present invention; Fig. 2 is a data collection system for establishing a line of sight estimation model according to an embodiment of the present invention. As shown in FIG. 1 , the method for establishing the sight line estimation model includes: step S1, data collection; step S2, extracting eye images and features; step S3, model establishment and solution.

In order to establish the statistical model of line-of-sight estimation, at first carry out the acquisition of relevant data, the data acquisition system of the present embodiment is shown in (a) in Fig. Display and a USB camera with a resolution of 640×480 pixels. There are 9 fixed points on the liquid crystal display, which are called viewpoints. The specific positions of the viewpoints are shown in (b) in Figure 2. The coordinates of each viewpoint are in units of pixels. In Figure 2 (b), 1, 2, 3....9 represents the label of the viewpoint, and the value in the brackets at the lower right, upper right or lower left of the label is the position of the corresponding viewpoint on the computer screen, for example, the position of the first viewpoint is (88, 83).

In step S1, the acquisition subject sits about 50cm-60cm away from the computer screen, and keeps the head as fixed as possible. In the process of data collection, firstly, the subject is required to gaze at the first point of view (88, 83) under the premise of keeping the head fixed and maintain the line of sight direction, and save the frontal image of the subject captured by the camera during this period, After reaching 30 frames, stop saving the image; then guide the subject’s sight to the second viewpoint (552, 83), and also save 30 frames of the user’s frontal image; continue until 270 frames of images from 9 viewpoints are collected. In the same way, this paper collects images of 50 samples looking at 9 viewpoints, a total of 13,500 user frontal images, and manually removes images with closed eyes, leaving 12,698 valid images.

It should be noted that, in the data collection, as described above, 30 frames of images may be collected for each viewpoint, but other suitable number of images may also be used, and it is not limited to 30 frames. Because there is no clear scope for model building, the more samples there are, the more accurate the model will be, and the more complex the model will be. In this embodiment, 30 frames are selected comprehensively considering the accuracy and complexity of the model.

After step S1 is completed, proceed to step S2, in which eye images and features are extracted. The method for extracting eye images and features in this embodiment will be described below with reference to FIGS. 3 to 4 .

As shown in Figure 3, the extraction of eye images and features in this embodiment includes:

Step S20, face positioning and inner and outer eye corner detection. For example, the classic Viola and Jones method is used to locate the face and detect the inner and outer corners of the eyes. Taking the left eye as an example, it is shown in (a) in Figure 4.

Step S21, eye image alignment, that is, cropping and aligning the eye area according to the position of the corner of the eye. The specific alignment method is as follows: Firstly, through statistics of the collected face areas, define a standard template of eye images with a size of 64×32, and set the positions of the inner and outer corners of the eyes as (54, 20) and (9, 20), respectively. By transforming the scale and rotation of the face area according to the position of the outer corner of the eye, an aligned eye image is obtained, as shown in (b) in Figure 4.

Further, taking the left eye as an example, the specific steps of scale and rotation transformation are: 1) Connect the inner and outer corners of the eyes, and rotate the line segment so that the inner and outer corners of the eyes are on the same horizontal line, and at the same time, other pixels in the face area are also Perform the same rotation transformation; 2) Change the scale of the left and right eye corners at the same horizontal position to coincide with the inner and outer eye corner positions defined on the standard template, and at the same time do the same scale transformation for other pixels in the face area ; 3) Finally, according to the size of the defined eye image standard template, the rotated and scale-transformed face area is intercepted to obtain an aligned eye image.

It should be noted that the above-mentioned eye image standard template is suitable for everyone. As long as the inner and outer corners of the left eye are positioned, the detected inner and outer corners of the eye and the inner and outer corners defined on the standard template can be maintained through rotation and scale transformation. consistent, resulting in aligned eye images. Eye image alignment is an important preprocessing step for face recognition. In this embodiment, eye image alignment is performed before building a model to deal with tiny head movements that appear in the collected images.

Step S22, acquiring features of the eye image. In the feature extraction part, in order to make full use of the micro-texture difference between the real face and the deceptive face, the eye image is further divided into r×c sub-regions, which are 4×2 in this embodiment, as shown in (c) in Figure 4 , extract the dual histogram local binary pattern (DualHistogramLocalBinaryPattern, DH-LBP) feature for each sub-region (DH-LBP is an improvement of the local binary pattern (LBP, LocalBinaryPattern) histogram, which greatly reduces the feature dimension while retaining The good resolution ability of LBP), the DH-LBP features of all sub-regions are concatenated to form the features of the eye image, as shown in (d) in Figure 4, which is 128 dimensions.

It can be understood that, on the one hand, the eye image features can be used to indicate the user's line of sight; on the other hand, due to problems such as printing and display reflections, there are obvious micro-texture differences between real faces and fake faces, so local Texture features can well distinguish real faces from spoofed faces. .

After step S2 is completed, proceed to step S3, in which the model is established and solved. The establishment and solution process of the line-of-sight estimation model will be described in detail below.

Obtain such a dataset through steps S1 and S2 Among them, x _i ∈ R ⁿ (n=128 in this embodiment, which is related to the dimension of the eye image feature) is the eye image feature of the sample, y _i =(p _x,i ,p _y,i )∈R ² are the two-dimensional coordinates of the corresponding viewpoint. In this embodiment, the following two-dimensional regression model is used to establish the mapping relationship between eye image features and corresponding viewpoint coordinates:

y= ^wTx +b,

w=(w ₁ , w ₂ )∈R ^n×2 , b=(b ₁ , b ₂ )∈R ² (1)

Where w, b are the model parameters to be sought

And define the corresponding loss function as:

$\mathrm{<span\; class="notranslate">\; loss</span>}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; -</span>{\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; the\; y</span>}}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>$

in is the corresponding predicted value. Generally speaking, formula (1) can be obtained by least square method. However, combined with the actual situation of this problem, the least squares method has the following disadvantages: 1) The line of sight estimation problem under a non-HD camera is a complex nonlinear problem, especially for the 9 calibration points with a large span; 2) The least squares method is more sensitive to the points outside the fitting line; 3) The least squares method only minimizes the empirical risk, and the generalization ability is not strong. Based on the above points, this embodiment decides to use a support vector regression (SVR, Support Vector Regression) method with strong robustness to solve this problem. In order to apply SVR, formula (1) is simplified into two one-dimensional regression models:

$\left\{\begin{array}{c}{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; =</span>{{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}}^{\mathrm{<span\; class="notranslate">\; T</span>}}\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}\\ {\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}<span\; class="notranslate">\; =</span>{{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}}^{\mathrm{<span\; class="notranslate">\; T</span>}}\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}\end{array}\right.<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; )</span>$

That is, considering the x coordinate point and the y coordinate point separately, in order to make the fitting function as smooth as possible, formula (1) is further transformed into the solution of the following two optimization problems:

$\mathrm{<span\; class="notranslate">\; min</span>}\mathrm{<span\; class="notranslate">\; imize</span>}\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; 2</span>}}{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; )</span>$

subjectto|p _x,i -w ₁ ^T xb ₁ |≤ε ₁

and

$\mathrm{<span\; class="notranslate">\; min</span>}\mathrm{<span\; class="notranslate">\; imize</span>}\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; 2</span>}}{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 5</span>}<span\; class="notranslate">\; )</span>$

subjectto|p _y,i -w ₂ ^T xb ₂ |≤ε ₂

Where ε ₁ , ε ₂ are the allowable maximum deviations between the predicted value and the actual value. In this embodiment, the libsvm toolbox is used to solve (4) and (5) to obtain and That is, for the test sample z, the x and y coordinate values of its sight position are predicted as f _x (z), f _y (z):

$\left\{\begin{array}{c}{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; z</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{{\mathrm{<span\; class="notranslate">\; no</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}^{<span\; class="notranslate">\; *</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; z</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}^{<span\; class="notranslate">\; *</span>}\\ {\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; z</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{{\mathrm{<span\; class="notranslate">\; no</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}^{<span\; class="notranslate">\; *</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; z</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}^{<span\; class="notranslate">\; *</span>}\end{array}\right.<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 6</span>}<span\; class="notranslate">\; )</span>$

Among them, x _i , y _i are support vectors respectively, and n ₁ , n ₂ are the number of support vectors. Compared with linear regression, nonlinear regression can more effectively model data. The common method for converting linear to nonlinear is the kernel technique. By defining the kernel function, the solution of formula (1) is:

$\left\{\begin{array}{c}{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; z</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{{\mathrm{<span\; class="notranslate">\; no</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}^{<span\; class="notranslate">\; *</span>}\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; z</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}^{<span\; class="notranslate">\; *</span>}\\ {\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; z</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{{\mathrm{<span\; class="notranslate">\; no</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{<span\; class="notranslate">\; *</span>}\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; z</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{<span\; class="notranslate">\; *</span>}\end{array}\right.<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 7</span>}<span\; class="notranslate">\; )</span>$

In this embodiment, the radial basis kernel function is used to solve the problem, and for the test data z, the predicted line-of-sight position is obtained by formula (7) $\widehat{\mathrm{the\; y}}=({\hat{p}}_x,{\hat{p}}_{\mathrm{the\; y}})=(f_x\left(z\right),f_{\mathrm{the\; y}}\left(z\right)).$

The line-of-sight estimation model (7) that can be established according to the above steps S1-S3 focuses on predicting changes in line-of-sight rather than the accuracy of estimation, and the line-of-sight estimation model proposed in this embodiment only needs to be configured under natural light with a resolution of 640 The USB camera with ×480 pixels can detect the change of line of sight very well.

It can be understood that although the present embodiment adopts nonlinear regression to establish and solve the line-of-sight estimation model in step S3, in fact, other regression methods, such as partial least squares regression, can also be used.

Furthermore, the biggest difference between the line of sight estimation model in this embodiment and the existing line of sight estimation model is that the emphases are also different due to different application occasions. The existing gaze estimation models are mainly used in human-computer interaction, so they focus on the accuracy of gaze estimation. Existing line-of-sight estimation models either require multiple calibration points, or high-definition cameras, and some even require multiple cameras. However, according to the application occasion of the present invention---living body judgment, the line of sight estimation model proposed in this embodiment focuses on the change of line of sight, rather than the accuracy of line of sight estimation, and the line of sight estimation model proposed in this embodiment only needs to be configured under natural light A USB camera with a size of 640×480 pixels can detect the change of line of sight very well.

As mentioned above, after the above line-of-sight estimation model is established, the model can be used for living body judgment. The method for judging human face livingness of the present invention will be described below with reference to FIG. 5 and FIG. 6 .

Fig. 5 is a flow chart of a method for judging human face liveness according to an embodiment of the present invention; Fig. 6 is a schematic diagram of constructing a gaze dictionary according to an embodiment of the present invention. As shown in Figure 5, the method for judging human face livingness in the present embodiment includes the following steps:

Step S100, acquiring a face image. In step S100, a certain number of face images are acquired by an image acquisition unit such as a usb camera, for example, 100 frames of images are acquired for 10 seconds.

Step S200, acquiring an eye image and features of the eye image. In step 200, the eye image and the features of the eye image of each face image are obtained, for example, the eye image and the feature of the eye image of each of the aforementioned 100 frames of face images are obtained. The eye image and the feature acquisition method of the eye image are the same as those described above, and will not be repeated here.

Step S300, perform gaze prediction, that is, perform gaze prediction according to the features of the acquired eye image and the gaze estimation model to obtain a corresponding number of gaze positions. Specifically, in step S300, the user's line of sight position sequence within a period of time is predicted according to line of sight estimation model formula (7). For example, for a certain user, assuming that M frames of images are collected in the current time period, the predicted M sight positions can be obtained by acquiring eye images and features of the image sequence ${\{{\widehat{\mathrm{the\; y}}}_i=({\hat{p}}_{x,i},{\hat{p}}_{\mathrm{the\; y},i})\}}_{i=1}^m.$

Step S400, line of sight quantification and statistics, that is, quantify the corresponding number of sight line positions according to the line of sight dictionary, and establish a line of sight histogram.

The bag of words model (Bag of Words, BOW) is usually applied in the field of information retrieval, and the document is represented by counting the frequency of occurrence of each word to form a histogram. In this embodiment, the bag-of-words model is extended to the bag-of-words model by extending the vocabulary in the document to the ``visual vocabulary'' in the image. It is embodied as "line of sight direction", which is called "line of sight word bag model" in the present invention, and the entries in the line of sight dictionary are called "line of sight entries". Similar to the bag-of-words model, the construction of the bag-of-sight word model is mainly divided into two steps: the construction of the sight-sight dictionary (codebook) and the generation of the sight-sight histogram. In the present invention, the line-of-sight entry is obtained by dividing the user's line-of-sight range into a grid. Assume that the user's line-of-sight range is m×n in size, where m and n are in units of pixels, and are evenly divided into r×c sizes The grid, then the set of (r+1)×(c+1) grid points on the grid constitutes the line-of-sight dictionary. For example, as shown in Figure 6, when the method for judging human face life in this embodiment is applied to a window with a resolution of 640×480 pixels, first determine the range of the user’s line of sight (assumed to be a rectangle with a size of 600×400 pixels) area (the edge area is not considered)), and then divide 600×400 into 2×2 (here the values of r and c may not be 2) sub-regions, then 9 red points constitute the line-of-sight dictionary. Preferably, the structure of the line-of-sight dictionary in this embodiment is based on 9 viewpoints when establishing the line-of-sight estimation model as line-of-sight entries, namely {(88,83), (552,83), (1016,83), (88,440), (552,440), (1016,440), (88,797), (552,797), (1016,797)}.

After constructing the gaze dictionary, the predicted gaze position sequence Quantification based on sight dictionary. For example, find the location associated with each prediction The nearest line of sight entry is voted (this embodiment adopts Euclidean distance), and the number of votes of each position entry is counted to form a histogram and normalized to form a line of sight histogram.

Step 500, living body judgment, that is, obtaining information entropy according to the line-of-sight histogram and performing living body judgment.

Specifically, for the normalized line-of-sight histogram H={p ₁ ,...,p ₉ } (satisfying Calculate the entropy value according to the definition of entropy

$\mathrm{<span\; class="notranslate">\; entroy</span>}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; -</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; 9</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}\mathrm{<span\; class="notranslate">\; log</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 8</span>}<span\; class="notranslate">\; )</span>$

if $\&Exists;{\{p_{i_k}>0\}}_{k=1}^l,1<l\&le;9,$ then there is

$\mathrm{<span\; class="notranslate">\; entropy</span>}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; -</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; 9</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}\mathrm{<span\; class="notranslate">\; log</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span><span\; class="notranslate">\; -</span>\underset{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; l</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; p</span>}}_{{\mathrm{<span\; class="notranslate">\; i</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}}\mathrm{<span\; class="notranslate">\; log</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; p</span>}}_{{\mathrm{<span\; class="notranslate">\; i</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ></span>\mathrm{<span\; class="notranslate">\; 0</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 9</span>}<span\; class="notranslate">\; )</span>$

Indicates that the line of sight direction is quantified into l different line of sight entries, and the corresponding entropy value is greater than 0 at this time.

If only one then there is

$\mathrm{<span\; class="notranslate">\; entroy</span>}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; -</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; 9</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}\mathrm{<span\; class="notranslate">\; log</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; p</span>}}_{{\mathrm{<span\; class="notranslate">\; i</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}}\mathrm{<span\; class="notranslate">\; log</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; p</span>}}_{{\mathrm{<span\; class="notranslate">\; i</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 10</span>}<span\; class="notranslate">\; )</span>$

Indicates that the line of sight direction is only quantized into one line of sight entry, and the corresponding entropy value is equal to 0 at this time. That is, under quantized conditions, when the line of sight changes, the entropy value is greater than 0; when there is no change, the entropy value is equal to 0.

To sum up, in this embodiment, the threshold value is set to 0, and the living body judgment condition is set as follows: if the entropy value entropy>0 in formula (8) is obtained, it is judged as a real face; otherwise, it is a spoofed face.

In addition to providing a human face living body judging method, the present invention also provides a living body judging system for distinguishing real faces and spoofed faces in face recognition. The living body judging system of the present invention will be described below with reference to FIG. 7 and FIG. 8 .

FIG. 7 is a schematic structural diagram of a living body judging system according to an embodiment of the present invention; FIG. 8 is a detailed structural schematic diagram of a feature extraction unit of the living body judging system shown in FIG. 7 .

As shown in FIG. 7 , the living body judgment system of this embodiment includes an image acquisition unit 100 , a feature extraction unit 200 , a line of sight estimation unit 300 , a sight line statistics unit 400 and a living body judgment unit 500 .

Wherein, the image acquisition unit 100, such as a camera with 640×480 pixels, is used to acquire a certain number of face images.

The feature extraction unit 200 is used to acquire the eye image of each face image and the features of the eye image. The feature extraction unit 200 includes a location detection module 201 , an eye image alignment module 202 and a feature extraction module 203 . The positioning detection module 201 is used for face positioning and inner and outer corner detection, such as face positioning and inner and outer corner detection through the classic Viola and Jones method, as shown in (a) in FIG. 4 taking the left eye as an example. The eye image alignment module 202 is used to crop and align the eye area according to the position of the corner of the eye to obtain the aligned eye image. The position of the corner of the eye, and the scale and rotation transformation of the face area according to the position of the outer corner of the eye to obtain an aligned eye image, the detailed process is as described above, and will not be repeated here. The feature extraction module 203 acquires the features of the eye image according to the aligned eye image, and specifically performs the following operations: divide the eye image into r*c sub-regions, and extract a dual histogram local binary pattern for each sub-region feature, concatenating the dual histogram local binary pattern features of all sub-regions to form the feature of the eye image.

The line-of-sight estimation unit 300 is configured to perform line-of-sight prediction according to the features of the acquired eye image and the line-of-sight estimation model to obtain a corresponding number of line-of-sight positions. That is, according to the gaze estimation model formula (7), the user's gaze position sequence within a period of time is predicted. For example, for a certain user, assuming that M frames of images are collected in the current time period, the predicted M sight positions can be obtained by acquiring eye images and features of the image sequence

The gaze statistics unit 400 quantifies the corresponding number of gaze positions according to the gaze dictionary, and establishes a gaze histogram. The process of line-of-sight statistics and quantification is as described above, and will not be repeated here.

The living body judging unit 500 obtains information entropy according to the line-of-sight histogram and performs living body, wherein

If the entropy value is greater than 0, it is judged as a real face, otherwise it is judged as a spoofed face.

The above embodiments are only exemplary embodiments adopted to illustrate the principles of the present invention, but the present invention is not limited thereto. For those skilled in the art, various modifications and improvements can be made without departing from the spirit and essence of the present invention. These variations and improvements are also regarded as the protection zone of the present invention.

Claims (12)

1. a face live body determination methods, for distinguishing real human face and deception face in recognition of face, is characterized in that, comprise:

Obtain the step of the facial image of some;

Obtain the step of the eye image of each facial image and the feature of this eye image;

The step that sight line prediction obtains respective amount eye position is carried out according to the feature of eye image obtained and sight line estimation model;

According to sight line dictionary, described respective amount eye position is quantized, set up the histogrammic step of sight line; With

Obtain information entropy according to sight line histogram and carry out the step of live body judgement, wherein

If entropy is greater than 0, be then judged to be real human face, otherwise be judged to cheat face.

2. face live body determination methods as claimed in claim 1, it is characterized in that, the step of the eye image of each facial image of described acquisition and the feature of this eye image comprises:

Carry out the step of Face detection and the survey of interior tail of the eye spot check;

According to the cutting of position, canthus and the eye areas thus obtain of the aliging step of eye image of aliging; With

According to the step of the feature of the Ophthalmologic image-taking eye image of described alignment.

3. face live body determination methods as claimed in claim 2, is characterized in that, described according to the cutting of position, canthus and align eye areas thus the step obtaining the eye image alignd comprise:

By the facial image that statistics gatherer arrives, definition eye image standard form, and the step of inner eye corner and tail of the eye position is set; With

Described human face region is carried out the step of the eye image of the conversion acquisition alignment of yardstick and rotation according to the position of the tail of the eye.

4. face live body determination methods as claimed in claim 2, it is characterized in that, the step according to the feature of the Ophthalmologic image-taking eye image of described alignment is specially:

Described eye image is divided into r × c sub regions, and Double-histogram local binary patterns feature is extracted to every sub regions, the Double-histogram local binary patterns feature of all subregions is connected, forms the feature of described eye image.

5. the face live body determination methods as described in claim 1-4 any one, is characterized in that, also comprise:

Set up the step of sight line estimation model, this step comprises:

The step of image data, described data be user observe successively setting quantity viewpoint time facial image;

Obtain the step of the eye image of each facial image and the feature of this eye image; With

According to the feature of described eye image and the coordinate of corresponding viewpoint, set up sight line estimation model and solve.

6. face live body determination methods as claimed in claim 5, it is characterized in that, described sight line estimation model is:

$\left\{\begin{array}{c}{f}_x\left(z\right)=\underset{i=1}{\overset{n_x}{\mathrm{\&Sigma;}}}{w}_x^{*}k(z,x_i)+b_x^{*}\\ f_y\left(z\right)=\underset{i=1}{\overset{n_y}{\mathrm{\&Sigma;}}}{w}_y^{*}k(z,y_i)+b_y^{*}\end{array}\right.$

Wherein, described eye position x _i∈ R ⁿthe feature of eye image, y _i=(p _x,i, p _y,i) ∈ R ²it is corresponding viewpoint two-dimensional coordinate.

7. live body judges a system, for distinguishing real human face and deception face in recognition of face, it is characterized in that, comprise:

Image acquisition units, it is for gathering the facial image of some;

Feature extraction unit, it is for the feature of the eye image and this eye image that obtain each facial image;

Sight line estimation unit, it carries out sight line prediction according to the feature of the eye image obtained and sight line estimation model and obtains respective amount eye position;

Sight line statistic unit, it quantizes described respective amount eye position according to sight line dictionary, sets up sight line histogram, and

Live body judging unit, it obtains information entropy according to sight line histogram and carries out live body, wherein

If entropy is greater than 0, be then judged to be real human face, otherwise be judged to cheat face.

8. live body as claimed in claim 7 judges system, and it is characterized in that, described feature extraction unit comprises:

Detection and localization module, it is for carrying out Face detection and the survey of interior tail of the eye spot check;

Eye image alignment module, its for according to the cutting of position, canthus and align eye areas thus obtain align eye image; With

Characteristic extracting module, it is according to the feature of the Ophthalmologic image-taking eye image of described alignment.

9. live body as claimed in claim 8 judges system, and it is characterized in that, described eye image alignment module specifically performs following operation:

By the facial image that statistics gatherer arrives, definition eye image standard form, and inner eye corner and tail of the eye position are set; With

The conversion that described human face region carries out yardstick and rotation according to the position of the tail of the eye is obtained the eye image alignd.

10. live body as claimed in claim 8 judges system, and it is characterized in that, described characteristic extracting module specifically performs following operation:

Described eye image is divided into r × c sub regions, and Double-histogram local binary patterns feature is extracted to every sub regions, the Double-histogram local binary patterns feature of all subregions is connected, forms the feature of described eye image.

11. live bodies as claimed in claim 7 judge system, and it is characterized in that, described image acquisition units is the camera of 640 × 480 pixels.

12. live bodies as described in claim 7-11 judge system, and it is characterized in that, described sight line estimation model is:

$\left\{\begin{array}{c}{f}_x\left(z\right)=\underset{i=1}{\overset{n_x}{\mathrm{\&Sigma;}}}{w}_x^{*}k(z,x_i)+b_x^{*}\\ f_y\left(z\right)=\underset{i=1}{\overset{n_y}{\mathrm{\&Sigma;}}}{w}_y^{*}k(z,y_i)+b_y^{*}\end{array}\right.$

Wherein, described eye position x _i∈ R ⁿthe feature of eye image, y _i=(p _x,i, p _y,i) ∈ R ²it is corresponding viewpoint two-dimensional coordinate.