CN105550634B - Face posture recognition method based on Gabor characteristics and dictionary learning - Google Patents

Abstract

The invention discloses a face gesture recognition method based on Gabor characteristics and dictionary learning, which comprises the following steps of: firstly, discretizing the human face posture into different subspaces, and training a sub dictionary for each subspace by using K-SVD (K-singular value decomposition) to enable the sub dictionary to correspond to a category; then, all the sub-dictionaries are combined into an ultra-complete dictionary; and finally, carrying out attitude classification by adopting a method based on gabor characteristics and sparse representation. In order to improve the robustness of the algorithm, the invention reconstructs an occlusion face dictionary and solves the problem of face occlusion in face gesture recognition. The method can solve the problems of illumination, noise, shielding and the like in the face posture estimation, and quickly and robustly identify the front face, the head raising, the nodding, the left deflection, the left side face, the right deflection and the right side face. The method can be well applied to the fields of safe driving, man-machine interaction, face recognition and the like.

Description

Face pose recognition method based on Gabor feature and dictionary learning

technical field

The invention belongs to the technical fields of image processing, pattern recognition, computer vision and human-computer interaction, and relates to a face gesture recognition method, in particular to a face gesture recognition method based on dictionary learning and sparse representation.

Background technique

Face pose estimation has great application prospects in the fields of intelligent video surveillance, face recognition, human-computer interaction and virtual reality. For example, in the aspect of intelligent video surveillance, face pose estimation can be applied to driving monitoring systems, by monitoring the changes in the driver's face pose to identify whether the driver is paying attention to driving and avoid crashes. In addition, face pose estimation has a great impact on the accuracy of face recognition. Many face recognition algorithms can achieve a good recognition rate for frontal face images, but for multi-pose non-frontal face images, its The recognition accuracy will drop seriously, and face pose pre-estimation is an important way to solve the multi-face pose recognition.

At present, the existing face pose detection methods can be roughly divided into three categories: texture subspace methods, 3D methods, and other methods. The first category of methods implements pose detection and estimation through 2D face appearance-based learning methods. The typical ones are Principal Component Analysis (PCA) and Linear Discriminant (LDA). Since PCA is a linear dimensionality reduction method, the 3D rotation change of face pose is largely a nonlinear change. Therefore, scholars use Kernel Principal Component Analysis (KPCA) and manifold learning method to solve this nonlinear change problem. However, kernel methods and manifold learning methods have a flaw: it is difficult to separate identity and pose as face training samples increase. This means that when the face training library is large enough, the accuracy of pose estimation will vary from person to person. The biggest feature of the first type of method is that the processing speed is fast and easy to implement, but it needs to be trained through a large number of samples, and it is more sensitive to changes in face illumination and expression, especially for video face images with extremely poor illumination. obvious.

The second type of method considers that face pose detection itself is a 3D problem, and only 3D information can characterize the essential features of face pose. Therefore, such methods often represent different poses by extracting 3D features, or use multiple images from different perspectives to reconstruct 3D models of faces in three-dimensional space to detect poses. Such methods often require high image size and quality, and spend a lot of computation time. The second type of method can achieve high accuracy through 3D methods, but the real-time performance is not high, and the effect of ultra-low resolution and occluded face images in video surveillance is not very good.

The third type of methods are some non-mainstream methods, which can only solve some problems in face pose estimation or can only be applied to some specific occasions. For example, Rafael et al. proposed a multi-camera face pose estimation method. In order to correctly estimate the face pose, their method needs to use 6 images taken by 6 front, back, left and right cameras for fusion and discrimination. J.Nuevo et al. proposed a block clustering method for face pose estimation, and achieved good results, but the range of poses estimated by their method was limited (only 45-degree range of pose changes could be recognized). Chen Zhenxue of Shandong University and others proposed a triangular face pose estimation method, and obtained an accuracy rate of about 91%, but their method is only effective for the deflection of the face around the Y and Z axes, but for the rotation of the face around the X axis. Invalid, that is, it is invalid for the rotation of the face up and down.

At present, it is still difficult for computers to have the same attitude recognition ability as humans. The main reason is that changes in lighting, noise, occlusion, resolution, identity, expression and other factors will have a huge impact on the accuracy of attitude estimation. How to eliminate these The influence of factors is an urgent problem to be solved at present.

SUMMARY OF THE INVENTION

The purpose of the present invention is to overcome the deficiencies of the above methods, and propose a face gesture recognition method based on Gabor feature and dictionary learning, which solves the problems of illumination, noise and occlusion in face gesture recognition, and robustly recognizes frontal, Head up, nod, left deflection, left face, right deflection and right face.

The object of the present invention is achieved through the following technical solutions:

A face gesture recognition method based on Gabor feature and dictionary learning, comprising the following steps:

Extract the Gabor feature of the face pose image to be recognized online input, and construct the Gabor feature vector y;

The Gabor eigenvector y is represented by a linear combination using an attitude complete dictionary, a sparse representation model is established, and the coefficient vector is solved, where the Gabor eigenvector m is the dimension of Gabor feature vector;

The face pose classification and recognition is performed according to the coefficient vector of the linear combination obtained above.

Further, the facial posture is divided into 7 different posture categories, which are respectively defined as left deflection, left face, right deflection, right face, frontal, head-up and nodding, which correspond to different subspaces.

Further, before using the posture complete dictionary to express the linear combination of the Gabor feature vector y, the training step of the posture complete dictionary is also included, wherein the posture complete dictionary includes the first posture complete corresponding to the unobstructed face posture. The dictionary D and the second pose complete dictionary D _e corresponding to the occluded face pose, the training of the first pose complete dictionary D and the second pose complete dictionary D _e are completed independently.

Further, the training process of the first posture complete dictionary D is as follows:

Collect face pose image samples of each pose category respectively, and perform Gabor filtering processing and feature extraction on the face pose image samples and quantify the face pose Gabor feature training set of each pose category;

Use K-SVD to perform training optimization on the face pose Gabor feature training set of each pose category to obtain the best sub-dictionary D _i , i=1, 2, ..., 7;

All kinds of optimal sub-dictionaries D _i are formed into a first pose complete dictionary D=[D ₁ , D ₂ , . . . , D ₇ ].

Further, the Gabor feature extraction of the face pose image is:

in, is the modulo of the Gabor filter coefficients The column vector obtained by sampling ρ times, μ, v are the direction and scale of the Gabor filter, and the Gabor filter with constant direction and scale is used to describe the characteristics of the face pose image, is the convolution of the face pose image and the Gabor kernel ψ _{μ, v} , and the Gabor kernel is defined as:

Among them, z(x, y) represents the pixel; is the wavelet term, k _v =km _max /f ^v , φ _μ =πμ/8, σ=1.5π controls the ratio of the width of the Gaussian window to the wavelength.

Further, when the face pose image to be recognized inputted online is an unoccluded face pose image, extract the Gabor feature vector of the face pose image to be estimated m is the dimension of Gabor feature vector, and y is regarded as the complete dictionary of the first pose The linear combination of : where N is the total number of dictionary atoms, and the sparse representation model is

Further, the sparse representation model

Solve by least squares with sparsity constraints:

Among them, λ is a balance factor, which plays a role in balancing reconstruction error and sparsity.

Further, when the face pose image to be recognized inputted online is an occluded face pose image, extract the Gabor feature vector of the face pose image to be estimated m is the dimension of the Gabor feature vector, and y is regarded as the first complete dictionary D and the second attitude complete dictionary The linear combination of :

in The unoccluded image y ₀ and the occlusion error image e ₀ can be obtained from the first pose complete dictionary D and the second pose complete dictionary respectively Sparse representation, _De is an orthogonal unit matrix, and the sparse representation model is

Further, the sparse representation model The occluded face pose image recognition problem is transformed into the following optimization problem:

This problem can be solved by standard linear norm methods.

Further, the described performing face pose classification and recognition according to the coefficient vector of the linear combination of the above-mentioned solution is specifically by effectively accumulating the coefficient vector of the linear combination of the solution, and taking the maximum accumulated value as the basis for judging the classification, that is,

where f( ) is a special function used to set the negative factor of the representation coefficient of the sparse representation model to 0, is the selection function used to select only the representation coefficients of the sparse representation model The representation coefficients corresponding to the i-th subspace matrix in , and other representation coefficients are set to 0, p _i (y) is the subspace corresponding to the i-th pose category in the training sample set and the representation coefficients of its corresponding sparse representation model Effectiveness cumulative factor.

Compared with the prior art, the present invention has the following advantages and effects:

1) At present, most face gesture recognition methods only classify the left and right deflection of the human face. The present invention can not only classify the left and right deflection of the human face, but also can classify the up and down posture. The invention can identify the roughly left and right up and down deflection states of the face image in about 0.3 seconds, the recognition accuracy is above 95%, and can be well applied to the fields of safe driving and human-computer interaction.

2) Since the present invention is based on the principle of dictionary learning and sparse representation, it can simultaneously solve problems such as illumination, noise, expression and occlusion in face gesture recognition;

3) Since the Gabor filter can extract the local features of the image from multi-scale and multi-direction. Therefore, the present invention uses the Gabor feature of the face image as the atomic vector of the overcomplete dictionary. This method improves the stability of face gesture recognition.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present invention more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only for the present invention. In some embodiments, for those of ordinary skill in the art, other drawings can also be obtained according to these drawings without any creative effort.

Fig. 1 is a schematic diagram of the classification of face pose images in the present invention;

2 is a schematic diagram of the general flow of the face gesture recognition method based on Gabor feature and dictionary learning disclosed by the present invention;

Fig. 3 is a schematic diagram of Gabor feature extraction of face pose image in the present invention;

FIG. 4 is a schematic diagram of face pose classification based on sparse representation in the face pose recognition method disclosed in the present invention.

Detailed ways

In order to make the technical means, creative features, achieved objects and effects of the present invention easy to understand and understand, the present invention will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention.

The terms "first", "second", "third" and "fourth" in the description and claims of the present invention and the above drawings are used to distinguish different objects, rather than to describe a specific order. Furthermore, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes unlisted steps or units, or optionally also includes For other steps or units inherent to these processes, methods, products or devices.

The following detailed descriptions are given respectively according to the examples.

Example

The face gesture recognition method based on Gabor feature and dictionary learning disclosed in the embodiment of the present invention mainly performs face gesture classification according to the idea of dictionary learning and sparse representation.

The face pose is divided in advance, and the face pose is discretized into different subspaces, and each word space corresponds to a face pose category. In the embodiment of the present invention, the division of 7 different gesture categories and the definitions of their corresponding subspaces are used to discretize the face gestures into 7 different subspaces, namely left side 1, left side 2, right side 1 and right side 2, There are 7 gesture categories including frontal, head-up, and nodding, as shown in Figure 1, which are defined as left deflection, left face, right deflection, right face, front, head up, and nod.

The face gesture recognition method based on Gabor feature and dictionary learning disclosed in the embodiment of the present invention includes two parts: gesture dictionary training and gesture classification and recognition. The pose dictionary training part includes three steps: Gabor feature extraction, K-SVD dictionary optimization and construction of pose complete dictionary; pose classification and recognition part includes Gabor feature extraction, sparse representation model matching solution and pose classification and recognition. Each step is described in detail below. The process of the face gesture recognition method based on Gabor features and dictionary learning is shown in Figure 2:

Step S1: pose dictionary training

This part is done offline. First, collect face pose image samples of each pose category respectively, perform Gabor filtering processing and feature extraction on the face pose image samples, and quantify the face pose Gabor feature training set of each pose category. The face pose is divided into 7 different subspaces, namely left 1, left 2, right 1 and right 2, frontal, head-up, nodding and other 7 pose categories, so 7 different people are vectorized respectively. Face pose Gabor feature training set. Then, the K-SVD method is used to train and optimize various sample sets to obtain various sub-dictionaries. Finally, various sub-dictionaries are fused to form a complete pose dictionary.

The posture complete dictionary includes a first posture complete dictionary D and a second posture complete dictionary D _e , wherein the first posture complete dictionary D corresponds to the unoccluded face posture, and the second posture complete dictionary D _e corresponds to the occluded face posture, Also known as the occlusion erosion dictionary _De . The training of the first posture complete dictionary D and the training of the second posture complete dictionary D _e is completed independently.

Specific steps and implementations include:

Step S1a: Gabor Feature Extraction

The Gabor feature extraction of the face pose image is:

in, is the modulo of the Gabor filter coefficients The column vector obtained by sampling ρ times, μ, v are the direction and scale of the Gabor filter, and the Gabor filter with constant direction and scale is used to describe the features of the face pose image. is the convolution of the face pose image with the Gabor kernel ψ _{μ, v} . The Gabor kernel is defined as:

Among them, z(x, y) represents the pixel; is the wavelet term, k _v =km _max /f ^v , φ _μ =πμ/8, σ=1.5π controls the ratio of the width of the Gaussian window to the wavelength. The value of each parameter of the present invention is k _max =π/2, ρ≈40, μ={0,...,7}, v={0,...,4}. An example of Gabor feature extraction is shown in Figure 3.

Step S1b: K-SVD optimization training

First, use the previous method to extract Gabor features from all training samples to form a training set, and then use the K-SVD method to optimize the training set to obtain a complete pose dictionary D. K-SVD is a classic dictionary training algorithm. According to the principle of minimum error, SVD decomposition is performed on the error term, and the decomposition term with the smallest error is selected as the updated dictionary atom and the corresponding atomic coefficient. After continuous iteration, the optimized algorithm is obtained. untie.

The specific process is: first, use K-SVD to perform training optimization on each type of pose sample set to obtain the best sub-dictionary _Di ; then, form the first _posture complete dictionary D=[D ₁ , D ₂ , …, _D7 ].

The present invention adopts K-SVD to perform training and optimization to obtain the best sub-dictionary D _i . K-SVD is a process that crosses iterative sparse representation and dictionary update. For a training set with n _i samples, the objective function of K-SVD is:

in, is the sparse coefficient set of the sample, is the sparse representation coefficient vector of the _i -th training sample yi, and k is the number of atoms in the dictionary D _i . The K-SVD algorithm consists of two processes. In the first process, the dictionary D _i is fixed, so the objective function above becomes an optimization problem for solving sparse representation coefficients, and there are many solving methods at present. The second process is to update the dictionary D _i using the sparse coefficients solved by the first process. This process works by incrementally updating each column _dk of D _i and the i-th row of X accomplish. and d _k , The solution is achieved by singular value decomposition (SVD).

Step S1c: Construct a pose complete dictionary

Various optimized sub-dictionaries D _i are obtained through step 1(b), and all sub-dictionaries are fused to form an over-complete dictionary. Because the present invention divides the human face postures into 7 types, the complete dictionary of human face postures is: Among them, m is the sample feature dimension, and n is the total number of dictionary atoms.

Step 2: Pose classification and recognition

This section is an online process. First, extract the Gabor feature vector of the online input face pose image. Then, the Gabor feature vector of the online input face pose image is represented by linear combination using the constructed pose complete dictionary, and a sparse representation model is established, and the Lars-Lasso algorithm is used to solve the model matching. Finally, pose classification is performed using the solution coefficients. Specific steps and implementations include:

Step S2a: Gabor feature extraction

This part is the same as step S1a, extracting the Gabor feature vector y of the face pose image input online.

Step S2b: sparse representation model matching and solution

After extracting the Gabor feature vector y of the face pose image input online, the constructed pose complete dictionary is used for linear combination representation, and vectorized to form a face pose Gabor feature training set.

The posture complete dictionary includes a first posture complete dictionary D and a second posture complete dictionary D _e , wherein the first posture complete dictionary D corresponds to the face posture without occlusion, and the second posture complete dictionary D _e corresponds to the face posture with occlusion , aka the occlusion erosion dictionary _De . Extract the Gabor feature vector of the face pose image to be estimated m is the dimension of Gabor feature vector, and y is regarded as the complete dictionary of the first pose (N is the total number of dictionary atoms) linear combination representation:

in is a very sparse linear combination coefficient vector. Ideally, if the test sample y belongs to the i-th type of pose, then x is 0 except for its corresponding i-th type which is not 0. According to the coefficient vector x, the pose category of the test image y can be obtained. Therefore, the face pose recognition problem is transformed into a problem of solving a system of linear equations. If m>N, the system of equations is overdetermined, and x has a unique solution or no solution. If m<N, x has multiple solutions. In the application of face gesture recognition, m<N is often needed, that is, the problem of underdetermined equations needs to be solved. According to the research results of atomic tracking, compressed sensing and sparse representation methods, it is shown that if the solutions of the above underdetermined equations are sparse enough, it can be Depend on The norm regularization minimization problem is solved:

This problem can be solved by standard linear norm methods.

Sparse representation model matching is to solve the following system of linear equations:

(unblocked) or (with cover)

In the above equation system, only the linearly combined coefficient vector x (take no occlusion as an example) or ω (take occlusion as an example) is unknown, so it can be solved by the least square method with sparse constraints:

Among them, λ is a balance factor, which plays the role of balancing reconstruction error and sparsity, and λ=0.01 in the present invention. The present invention adopts the Lars-Lasso algorithm of least angle regression (Least Angle Regression, LAR) to solve the above-mentioned minimum optimization model (B.Efron, T.Hastie, I.Johnstone and R.Tibshirani, "Least AngleRegression", Annals of Statistics, 32, 407-499, 2004). The solution is similar for occlusion cases.

Step S2c: face gesture recognition

The face pose classification and recognition is performed according to the coefficient vector x of the linear combination of the solution. theoretically It should only be closely related to the test sample of a certain type of pose in the training sample, and its corresponding characterization coefficient is non-zero. Therefore, the to-be-measured pose can be clearly classified. However, due to noise and modeling errors, some non-correlated class characterization coefficients of the samples to be tested will appear non-zero elements with small values, which will affect the correct classification. Considering that the sparse representation of the samples to be tested on the training sample set is based on the overall training set, it is considered that the representation coefficients of the samples to be tested on the training sample set are effectively accumulated, and the maximum accumulated value is used as the basis for judging the classification.

The pose category of the face pose image y to be estimated can be obtained according to the coefficient vector x solved in step S2b. As shown in Figure 4, the test face pose belongs to the fourth category, so its non-zero coefficients are mainly concentrated in x ₄ , and most of the coefficients of other categories are 0.

In order to reduce the influence of noise and modeling errors; at the same time, considering that the sparse representation of the samples to be tested on the training sample set is based on the overall training set, it is considered to accumulate the validity of the representation coefficients of the samples to be tested on the training sample set to obtain The maximum accumulated value is used as the basis for judging the classification. So the poses based on sparse representation are classified as:

Among them, f( ) is a special function, which means that the negative factor of the sparse representation coefficient is set to 0; is a selection function that selects only sparse representation coefficients The representation coefficient corresponding to the i-th subspace matrix in , and set the others to 0; p _i (y) is the validity accumulation factor of the i-th subspace and its corresponding sparse representation coefficient in the training sample set.

Corresponding to occlusion face pose processing, the problem of face pose image occlusion is solved by adding occlusion pose face dictionary. Because, the above face gesture recognition method cleverly utilizes the principle of sparse representation classification method. Therefore, it is robust to face illumination, noise, expression and resolution changes. In order to solve the problem of face pose occlusion: first, establish an occlusion erosion dictionary De according to step _S1 . Then, extract the Gabor feature vector of the face pose image to be estimated according to step S2a and step S2b m is the dimension of the Gabor feature vector, and y is regarded as a combination of the non-occlusion dictionary D and the occlusion erosion dictionary The common linear combination representation (corresponding to the dictionary learning model in step S2b) is:

in, The unoccluded image y ₀ and the occlusion error image e ₀ can be obtained from the dictionary D and the occlusion dictionary respectively Sparse representation. _De is an orthogonal identity matrix. Finally, the occlusion face pose recognition problem is transformed into the following optimization problem:

This problem can be solved by standard linear norm methods. Sparse representation model matching is to solve the following system of linear equations:

(with cover)

In the above equation system, only the coefficient vector ω of the linear combination (take occlusion as an example) is unknown.

So far, the face _occlusion problem is successfully solved by adding the occlusion dictionary De.

The above-mentioned embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited by the above-mentioned embodiments, and any other changes, modifications, substitutions, combinations, The simplification should be equivalent replacement manners, which are all included in the protection scope of the present invention.

Claims (5)

1. A face posture recognition method based on Gabor characteristics and dictionary learning is characterized by comprising the following steps:

performing Gabor feature extraction on the online input face posture image to be recognized to construct a Gabor feature vector y;

performing linear combination representation on the Gabor characteristic vector y by using an attitude complete dictionary, establishing a sparse representation model and solving a coefficient vector, wherein the Gabor characteristic vectorm is a Gabor eigenvector dimension;

carrying out face posture classification and identification according to the solved coefficient vector of the linear combination;

before the Gabor feature vector y is subjected to linear combination representation by using a pose complete dictionary, the method also comprises a training step of the pose complete dictionary, wherein the pose complete dictionary comprises a first pose complete dictionary D corresponding to the non-shielded human face pose and a second pose complete dictionary D corresponding to the shielded human face pose_eSaid first well-posed complete dictionary D and said second well-posed complete dictionary D_eThe training is respectively and independently finished;

when the online input face gesture image to be recognized is a shielded face gesture image, extracting a Gabor characteristic vector of the face gesture image to be recognizedm is the dimension of Gabor eigenvector, and y is considered as the complete dictionary from the first postureAnd a second gesture complete dictionaryCommon linear combinations represent:

whereinNon-occluded image y₀And occlusion error image e₀From a first posture-completed dictionary D and a second posture-completed dictionary D respectivelySparse representation, D_eIs an orthogonal identity matrix, and the sparse representation model is

The sparse representation modelThe problem of recognizing the human face posture image with the shielding is converted into the following optimization problem:

(l¹):ω＝argmin||ω||₁s.t.Bω＝y，

the problem is solved by standard linear normative methods.

2. The face pose recognition method based on Gabor feature and dictionary learning of claim 1,

the human face pose is divided into 7 different pose categories, which are respectively defined as left deflection, left side face, right deflection, right side face, front, head raising and nodding, and each of the pose categories corresponds to different subspaces.

3. The face pose recognition method based on Gabor feature and dictionary learning of claim 1, wherein the training process of the first pose complete dictionary D is as follows:

respectively collecting face attitude image samples of all attitude categories, carrying out Gabor filtering processing and feature extraction on the face attitude image samples, and vectorizing to form a face attitude Gabor feature training set of all attitude categories;

training and optimizing a face posture Gabor feature training set of each posture category by using K-SVD to respectively obtain an optimal sub-dictionary D_i，i＝1，2，…，7；

All kinds of optimal sub-dictionaries D_iForm a first pose complete dictionary D ═ D₁,D₂,…,D₇]。

4. The face pose recognition method based on Gabor features and dictionary learning of claim 3, wherein the Gabor features of the face pose image are:

wherein,by modulo of Gabor filter coefficientsColumn vectors obtained by sampling rho times, mu and v are directions and scales of the Gabor filter,for human face pose image and Gabor kernel psi_μ,vThe Gabor kernel is defined as:

wherein z (x, y) represents a pixel;is a wavelet term, k_v＝k_max/f^v，φ_μPi μ/8, and 1.5 pi control the ratio of gaussian window width to wavelength.

5. The method of claim 1, wherein the face pose recognition based on Gabor feature and dictionary learning is performed by performing validity accumulation on the coefficient vectors of the linear combination of the solution, and taking the maximum accumulated value as a basis for determining classification, that is, performing face pose classification recognition based on the coefficient vectors of the linear combination of the solution

Where f (-) is a special function for setting the negative factor of the representation coefficient of the sparse representation model to 0,is a selection function for selecting only the representation coefficients of the sparse representation modelThe representing coefficient corresponding to the i-th type subspace matrix, and setting other representing coefficients to be 0, p_iAnd (y) is an effectiveness accumulation factor of the subspace corresponding to the ith posture category in the training sample set and the corresponding expression coefficient of the sparse expression model.