CN1137662C - Main unit component analysis based multimode human face identification method - Google Patents

Abstract

The present invention belongs to the technical fields of image processing, computer vision and pattern recognition. The present invention comprises: human face images are positioned; five kinds of human face components, namely naked faces, eyebrows and eyes, eyes, nasal tips and mouths, are extracted from whole human faces; five kinds of human face components from each human face image, each known human face image and each human face image to be recognized in a training set are extracted; five kind character components, and the projection character values of the human face components of the known human faces are respectively formed by a character face method in a principal component analysis method, a database which comprises the projection character valves of the human face components of the known human faces, the compressing images of the known human faces and personal identity files of the known human is established, and the character valves of the human face components of the human faces to be recognized are established; the human faces to be recognized in the database of the known human faces are processed by multimode overall human face recognition and part human face recognition by computing similarity and by a similarity ordering method. The present invention achieves high recognition rate.

Description

Multimodal Face Recognition Method Based on Component Principal Component Analysis

technical field

The invention belongs to the technical fields of image processing, computer vision and pattern recognition, and in particular relates to a face recognition method.

Face recognition involves many disciplines, including image processing, computer vision, pattern recognition, and neural networks. It is also closely related to the research results of neurophysiology and neurobiology on the structure of the human brain. The difficulty of face recognition lies in:

(1) Face plastic deformation caused by expression

(2) Face diversity caused by posture

(3) Face changes caused by age

(4) The multiplicity of face patterns caused by factors such as hairstyles, beards, glasses, and makeup

(5) Differences in face images caused by factors such as illumination angle, intensity, and sensor characteristics

Many factors make portrait recognition a thorny and challenging subject, and thus become a hot research topic in recent years.

At present, the existing face recognition methods all recognize the whole face, and in many recognition methods, mainly use methods such as principal component analysis (PCA-Principal Component Analysis), elastic matching, neural network, geometric features, etc. For example, the PCA method currently used is to calculate the eigenface for the entire face and calculate the projected eigenvalue of the entire face. Due to the difficulties in the above five aspects, it is difficult to achieve a high recognition rate for the recognition of the entire face.

Contents of the invention

The purpose of the present invention is to overcome the deficiencies of the prior art, and propose a multi-mode face recognition method based on component principal component analysis, carry out component extraction to the human face, and then carry out principal component analysis and detection of the human face components Multi-pattern recognition to achieve high recognition rate.

A kind of multi-mode face recognition method based on component principal component analysis that the present invention proposes, is characterized in that: comprise the following steps:

1) Use template matching and projection histogram methods to locate the face image, and determine the basic positions of the rough positioning area of the face, left and right eyeballs, nose tip, mouth, and mandible vertices;

2) Extract five face parts from the whole face: naked face, eyebrows+eyes, eyes, nose tip, and mouth;

3) Use steps 1) and 2) to extract each person's face image of each person in the training set, including naked face, eyebrows+eyes, eyes, nose tip, and mouth. The naked face, eyebrows+eyes, eyes, nose tip, and mouth extracted from the face are used to form the characteristic naked face, characteristic (eyes+eyebrows), and characteristic eyes respectively by using the eigenface method in the principal component analysis method. , characteristic nose, characteristic mouth;

4) For each known face image of each person, use steps 1) and 2) to extract the naked face, eyebrows+eyes, eyes, nose tip, and mouth five face parts, and extract from known faces The naked face, eyebrows+eyes, eyes, nose tip, and mouth are five face parts, and the projection eigenvalue analysis method in the principal component analysis method is used to extract the naked face, eyes+eyebrows, eyes, nose, and mouth of known faces. Projection eigenvalues of five face parts and establishment of projected eigenvalues including known face naked face, eyes+eyebrows, eyes, nose, mouth, compressed images of known faces and known personal database of identity files;

5) The face image of each person to be recognized adopts the naked face, eyebrows+eyes, eyes, nose tip, and mouth five face parts extracted in steps 1) and 2), and extracts from the face to be recognized The bare face, eyebrows+eyes, eyes, nose tip, and mouth are five face parts, and the feature projection value analysis method in the principal component analysis method is used to extract the naked face, eyes+eyebrows, eyes, nose, The projected eigenvalues of the mouth;

6) In the known face database, the faces to be recognized are calculated and sorted by similarity for multi-mode global face recognition and local face recognition.

Said multi-mode global face recognition method may include a comprehensive recognition method of eigenface, eigen[eyes+eyebrows], eigeneye, eigennose, and eigenmouth.

The partial face recognition method of said multi-mode can comprise the recognition of single eigenface, feature [eye+eyebrow], feature eye, feature nose, feature mouth, or feature face, feature [eye+eyebrow], feature eye, feature Combination recognition between nose and characteristic mouth.

Features and effects of the present invention

The invention extracts the components of the face, and then performs principal component analysis and multi-pattern recognition on the components of the face to achieve a high recognition rate.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a schematic diagram of a face image positioning result according to an embodiment of the present invention. Fig. 2 is a schematic diagram of a face template used for matching in the present invention. Figure 3: Division of the area to be matchedFigure 4: Gradient directional integral projectionFigure 5: Valley point searchFigure 6: Gradient integral projection on the X-axis of the two nostrilsFigure 7: Grayscale integral projection of the lower part of the eyeballFigure 8: From the whole Five parts of naked face, eyebrows+eyes, eyes, nose tip, and mouth are extracted from the human face. Eyes + eyebrows, eyes) recognition query Figure 12: Global face recognition query

Detailed ways

A kind of multi-mode face recognition method embodiment based on component PCA that the present invention proposes is described in detail as follows in conjunction with each accompanying drawing;

This embodiment includes the following steps: 1) Positioning the face image. The selected standard face image size is 330×480 (width×height), and the positioning process is divided into two steps: rough positioning and fine positioning. The process of rough positioning is the process of finding the rough positioning area of the face in the face map, and the process of fine positioning is the process of determining the position of the eyes of the person on the basis of rough positioning. Then determine the positions of the tip of the nose, mouth, and the top of the mandible, and the face image positioning results are shown in Figure 1. In Figure 1, the white line frame is the position of the rough positioning area of the face determined by the rough positioning, the white dots in the eyeballs are the positions of the left and right eyeballs determined by the fine positioning, and the small white circles at the tip of the nose and the top of the mandible are the fine positioning The determined position of the tip of the nose, the top of the mandible, and the small white circle in the mouth is the position of the center of the mouth. Among them, the rough positioning adopts the method of template matching, and Fig. 2 is a face template used for matching, which is a template with a size of 48×33 pixels. FIG. 3 divides a single matching area in the face image to be located into 9 sub-areas, and the size of each sub-area is 16×11. In template matching, gradient statistics are performed on each sub-area of the area to be matched first, and the statistical results are Mg1~Mg9. According to the gradient distribution of the face, it is considered that the matching area does not perform matching operations as long as it meets one of the following conditions:

   When Mg1<Mg4 or Mg3<Mg6 or Mg1<Mg2 or Mg3<Mg2 is used for matching operations, the matching function of formula (1) is used.

S=S _sym +S ₀ +S _hist (1) where S _sym represents the symmetry of the left and right sides of the region to be matched, S ₀ represents the result of template matching, and Sh _hist is the matching of the X-axis integral projection histograms of regions 1 and 4 Condition. In practice, the calculation of S ₀ is performed after adjusting the mean values of the left and right regions to the same value according to S _sym . S ₀ is to calculate the correlation between the template and the region to be matched. In the case where the mean and variance of the region to be matched and the template are adjusted to the same value, subtract and then take the absolute value for matching.

The precise positioning of the face is realized through the positioning of the eyeballs, and areas 1 and 3 should be the areas where the eyeballs may appear. The positioning of the eyeball is achieved by finding the valley point. A bottom-up search method is adopted here, as shown in Figure 4, the integral projection in the figure is the integral projection of the gradient map. When the peak value of point A in Figure 4 is found, it is considered that the approximate horizontal position Ay of the eye has been found, and the region of interest y belongs to (Ay-delta, Ay+delta), let x change in a large range, and look for The valley point (lowest point) in this area. Adopt the model shown in Fig. 5 to carry out the ratio calculation shown in (2) formula, thereby determine the position of valley point, in Fig. 5, sub-area 5 represents the area where eyeball is located. $S=\frac{\underset{\mathrm{no}=1}{\overset{9}{\mathrm{\&Sigma;}}}\mathrm{Gray}\left(\mathrm{no}\right)}{\mathrm{Gray}\left(5\right)}---\left(2\right)$

(2) represents the gray scale ratio inside and outside the sub-area 5, where Gray(n) represents the gray scale sum of the n area.

The extraction of the centerline between the two nostrils mainly depends on the peak information of the gradient integral projection on the X-axis of the two nostrils, as shown in Figure 6.

In Fig. 6, L1 is the x-coordinate of the left nostril, L2 is the x-coordinate of the right nostril, and x0 is the x-coordinate of the center line between the two nostrils. The part below the eyeball of the matching face is to be gray-scale integral projection, as shown in Figure 7. In Figure 7, y1 is the Y coordinate of the apex of the mandible, y2 is the Y coordinate of the center of the mouth, y3 is the Y coordinate of the tip of the nose, and their X coordinates are all x0.

In this way, the positioning of the left and right eyeballs, the tip of the nose, the center of the mouth, and the top of the mandible is completed.

2) Carry out facial component extraction. According to the specific positions of the left and right eyeballs, the tip of the nose, the center of the mouth, and the top of the mandible determined by the positioning of the face image, five parts of the naked face, eyebrows+eyes, eyes, nose tip, and mouth are extracted from the entire face. The components are shown in Figure 8. The image size of the bare face is 90×120, the image size of the eyebrows+eyes is 182×70, the image size of the eyes is 160×40, the image size of the tip of the nose is 78×32, and the image size of the mouth is 90×68.

3) Use the PCA (Principal Component Analysis) method to analyze the naked face, eyebrows + eyes, eyes, nose tip, and mouth extracted from the training set, and form the characteristic naked face, feature (eyes + eyebrows), feature eyes, and feature nose respectively. , Feature mouth, respectively extract the projected feature values of the naked face, eyes + eyebrows, eyes, nose, and mouth of the known face and the projected feature values of the naked face, eyes + eyebrows, eyes, nose, and mouth of the face to be recognized.

Select more than 1,000 face images, and each face image is processed by the above steps 1) and 2) to form a naked face training set, (eyes+eyebrows) training set, eye training set, nose training set, and mouth training set PCA analysis is performed on these training sets to form eigenfaces, features (eyes+eyebrows), feature eyes, feature noses, and feature mouths. Figure 9 shows the process of forming eigenfaces from the face images of the training set through PCA analysis. Process, where X is the number of pixels of the naked face, N is the number of face images in the training set, and D is the number of reserved eigenfaces.

The formation process of features (eyes+eyebrows), feature eyes, feature nose, and feature mouth is similar to that of feature faces.

On the basis of the obtained eigenface, feature (eye+eyebrow), eigeneye, eigennose, and eigenmouth, the projected eigenvalues and Projection eigenvalues of naked face, eyes+eyebrows, eyes, nose, and mouth of the face to be recognized. The specific method is:

Represent N human face vectors with n * N matrix, n is the lattice number of human face image, N is the lattice number of training human face figure item, then: $C=\frac{1}{N}{\mathrm{XX}}^T---x=(x_1,x_2,...,x_N)---\left(3\right)$

(3) where X _k = x _1k , x _2k , ..., x _nk k = (1, 2, ..., N)

In calculating the eigenvectors and eigenvalues of C, due to the large dimension of XX ^T (n ² dimensions), the singular value decomposition is used to calculate X ^T X, so that the eigenvectors and eigenvalues of C can be obtained indirectly , and after calculating X ^T X, it is reduced to N ² dimensions, the eigenvalues of XX ^T and X ^T X are the same, and the relationship between their eigenvectors satisfies the following formula: $u_k=\frac{1}{{\mathrm{\&lambda;}}_k}\&times;{\mathrm{\&phi;}}_k---\left(4\right)$

(4) u _k is the eigenvector of XX ^T , and φ _k is the eigenvector of X ^T X, λ _k is not only the eigenvalue of XX ^T , but also the eigenvalue of X ^T X.

For the matrix R, there exists a Φ matrix such that the following formula holds:

R×Φ＝Φ×Λ (5)

In the formula (5), Λ contains the eigenvalues of the matrix R, A=Dig{λ ₁ , λ ₂ ,...,λ _N }, and the formula (5) is expressed as N equations:

R × φ _k = λ _k × φ _k k = 1, 2, ..., N (6)

The eigenvalue λ _k in formula (6) can be obtained by the following formula:

|R-λ _k ×I|＝0 (7)

Sort the obtained values of λ _k from large to small, take out the first D largest eigenvalues and keep the corresponding D eigenvectors φ _k .

The eigenvector u _k of matrix C is calculated from formula (4).

Matrix C is the naked face, eyes+eyebrows, eyes, nose, and mouth separated from the faces of the training set, respectively, through the operations of formulas (3), (4), (5), (6), and (7) , get the eigenface, feature (eyes + eyebrows), feature eyes, feature nose, feature mouth, this is the eigenface method in the PCA (Principal Component Analysis, principal component analysis) method.

4) Extract the projected feature values of the naked face, eyes+eyebrows, eyes, nose, and mouth of known faces and establish a database of known faces.

Use the methods described in steps 1) and 2) above to separate the naked face, eyes+eyebrows, eyes, nose, and mouth from the image of a known face to form face features. Using the operation of the following formula (8) (i.e. the projection eigenvalue analysis method in the PCA (Principal Component Analysis) method), respectively form the projection features of the naked face, eyes + eyebrows, eyes, nose, and mouth of the known human face value. $B=u_k^T\&times;q---k=\mathrm{1,2},...,\mathrm{D.}---\left(8\right)$

(8) where q are the naked face, eyes+eyebrows, eyes, nose, and mouth of known faces, and u _k ^T are the eigenfaces, features (eyes+eyebrows), and eigeneyes obtained from the faces in the training set, respectively. , characteristic nose, characteristic mouth, D=60.

The characteristic projection values of naked face, eyes+eyebrows, eyes, nose, and mouth of a known face are formed into a projection feature value string of a known face in the order of naked face, eyes+eyebrows, eyes, nose, and mouth. Based on this, a database for face recognition is established, which contains known face images (compressed by JPEG method), projection feature value strings of known faces and personal identity files.

5) Extract the feature projection values of the naked face, eyes + eyebrows, eyes, nose, and mouth of the face to be recognized. Use the methods described in 1) and 2) above to separate the naked face, eyes + eyebrows, eyes, nose, and mouth from the image of the face to be recognized to form facial features. Use formula (9) to calculate (that is, the projection eigenvalue analysis method in the PCA (Principal Component Analysis, principal component analysis) method) to form the projection features of the naked face, eyes+eyebrows, eyes, nose, and mouth of the face to be recognized value. $A=u_k^T\&times;q---k=\mathrm{1,2},...,\mathrm{D.}---\left(9\right)$

(9) where q is the naked face, eyes + eyebrows, eyes, nose, and mouth of the face to be recognized, and u _k ^T are the eigenface, feature (eye + eyebrow), and eigeneye obtained from the faces in the training set, respectively. , characteristic nose, characteristic mouth.

6) The method of global face recognition and partial face recognition is used for face recognition. The process of face recognition is: compare the features of the face to be recognized with the features of the face stored in the database, calculate the similarity, and then compare the faces in the database according to the similarity with the face to be recognized. To the small order, and in this order, display the photo of the person being found, the personal identity file and the similarity with the person to be identified, so as to find out the identity of the person to be identified or similar in appearance to the person to be identified person's identity. Formula (10) is used to calculate the similarity between the face to be recognized and the known face. $R=\sqrt{1-\frac{\left|\right|A-B\left|\right|}{\left|\right|A\left|\right|+\left|\right|B\left|\right|}}---\left(10\right)$

(10) where A is the projected eigenvalue string of the face to be recognized, and B is the normalized projected eigenvalue string of the known face in the database.

When the global face recognition method is used, the projected feature values of the naked face, eyes+eyebrows, eyes, nose, and mouth of the known face are weighted in a ratio of 5:6:4:3:2, and the eigenvalues of the face to be recognized are The projected eigenvalues of naked face, eyes+eyebrows, eyes, nose, and mouth are weighted according to the ratio of 5:6:4:3:2, and then the similarity is calculated according to (10).

When using the partial face recognition method, use the method of human-computer interaction to select any combination of naked face, eyes+eyebrows, eyes, nose, and mouth, and the number of combinations is 5! A total of 120 types, that is, a total of 120 face recognition modes. The projected feature values of bare face, eyes+brows, eyes, nose, and mouth are still weighted in a ratio of 5:6:4:3:2.

The structure of a face recognition system adopting the method of this embodiment is shown in FIG. 10 . Including the 6 steps of 1 to 6 steps in the method, plus the input of the image, using VC++ to compile the program to form each module in Figure 10, which is respectively installed on 5 PC microcomputers (4 as a server, and one as a server) In the client computer), it also includes a camera for image input, a microtek E6 scanner.

The face recognition system constituted by this method adopts the client/server mode, and the matching algorithm of face recognition is embedded in the server. A database of known portraits can be established through the registration module. The client sends the query request and the features of the face to be recognized to the server through the recognition query module, and accepts the recognition query result sent back from the server. In order to improve the identification and query speed, multiple servers are connected by cluster computer to realize parallel identification and query. The face input module accepts the face image input by the scanner or the camera, and after passing the quality identification module and the feature extraction module, the identification query module realizes the identification query of the portrait.

The database for face recognition query in the system includes three parts: personal identity file, face image (compressed by JPEG method) and face projection feature value. The four servers connected by cluster computers include a master server and three slave servers. In order to reduce the communication traffic of the network, it is necessary to carry out database sub-database work in advance, that is, divide a large database into four small databases and store them in of the four servers.

The main database server of the system uses Sugon PC server, configured as: dual CPU-PIII 500, 512MB memory, dual SCISI hard disk-9G×2. Install Windows NT 4.0 and Oracle8.

This system uses three database slave servers, and the configuration of each slave server is: CPU-PIII733MHz, 512MB memory, 15G IDE hard disk, Windows NT 4.0 and Oracle8 database installed.

There are 92,000 known persons entered in this embodiment.

The configuration of the client computer is: CPU-PII 266, 128MB memory, Windows 98 is installed. The entire system is connected by a network, and the network hub adopts a 100Bb switch.

The face recognition system composed of the above methods has passed the actual running test, and the effect is very obvious. The examples are as follows:

Example 1: As shown in Figure 11, using this method for partial (naked face, eyes+eyebrows, eyes) recognition query, the suspect's simulated face (as shown in the left part of the figure) and the real suspect's face The matching degree is 87.88%, and the real suspect is ranked 182 out of 92,000 face images. If the global face recognition is used, the real suspect in the query ranking ranks 16617 among the 92000 face images (not shown in the figure).

Example 2: As shown in Figure 12, the global face recognition query using this method, the queried face is shown in the left part of the figure, and the recognition query result is: in the 92,000 face image database, the photos of the queried person in different periods in the top three.

Claims (3)

1, a kind of multi-mode face identification method based on the parts principal component analysis is characterized in that: may further comprise the steps:

1) adopts the method for template matching and projection histogram that facial image is located, determine the home position on people's face coarse positioning district, left and right sides eyeball, nose, mouth, lower jaw summit;

2) from whole people's face, extract naked face, eyebrow+eyes, eyes, nose, five kinds of face components of mouth;

3) everyone facial image to training set adopts step 1), 2) extract everyone five kinds of face components of naked face, eyebrow+eyes, eyes, nose, mouth, to the five kinds of face components of naked face, eyebrow+eyes, eyes, nose, mouth that from training set people face, extract, utilize the eigenface method in the principal component method, form the naked face of feature, feature eyes+feature eyebrow, feature eyes, feature nose, feature face respectively;

4) known everyone facial image is adopted step 1), 2) the naked face that extracts, eyebrow+eyes, eyes, nose, five kinds of face components of mouth, to the naked face that from the known person face, extracts, eyebrow+eyes, eyes, nose, five kinds of face components of mouth, utilize the projection properties value analytical method in the principal component method, extract the naked face of known person face, eyes+eyebrow, eyes, nose, the projection properties value of five kinds of face components of mouth and foundation comprise the naked face of known person face, eyes+eyebrow, eyes, nose, the data base of the personal identification archives of the projection properties value of five kinds of face components of mouth and the compressed image of known person face and known person;

5) facial image to be identified everyone adopts step 1), 2) five kinds of face components of naked face, eyebrow+eyes, eyes, nose, mouth of extracting, to the five kinds of face components of naked face, eyebrow+eyes, eyes, nose, mouth that from people's face to be identified, extract, utilize the feature projection value analytical method in the principal component method, extract naked face, eyes+eyebrow, eyes, the nose of people's face to be identified, the projection properties value of face;

6) in known face database, people's face to be identified is adopted the calculating similarity and carries out multimodal overall recognition of face and local recognition of face by the method for sequencing of similarity.

2, the multi-mode face identification method based on the parts principal component analysis as claimed in claim 1, it is characterized in that, said multimodal overall face identification method comprises eigenface, feature eyes+feature eyebrow, feature eyes, feature nose, the comprehensive recognition methods of feature face.

3, the multi-mode face identification method based on the parts principal component analysis as claimed in claim 1, it is characterized in that, said multimodal local face identification method comprises the identification of single eigenface, feature eyes+feature eyebrow, feature eyes, feature nose, feature mouth, or the combination identification each other of eigenface, feature eyes+feature eyebrow, feature eyes, feature nose, feature mouth.