CN102419819B - Face image recognition method and system - Google Patents

Abstract

The present invention relates to a human face image recognition method and system. In order to solve the problems of slow recognition speed and low recognition rate in the prior art, the method is to first establish a user template library and then perform recognition; the user template library is when registering a user, the user changes N postures, three frames of images are collected in each posture and feature templates are extracted, and the template sorting principle is: A. The selected template represents the unselected template as much as possible, that is, it has the greatest similarity with other candidate templates; B. The selected template is as far away from the selected template as possible, that is, it maintains the minimum similarity with the selected template. The system uses an embedded microprocessor as the central processing unit of the system, a CMOS sensor is used to collect human face images, and is supplemented by an infrared lighting device for the human face, and a narrow-band infrared filter is used to filter visible light. It can effectively solve the problems of low recognition rate and slow recognition speed, and the speed and recognition rate can reach the level of commercial application. The system has the advantages of being able to be used under any light conditions, being able to eliminate the interference of ambient light, having good reliability, and reaching a satisfactory level of recognition rate and recognition speed.

Description

Face image recognition method and system

technical field

The invention designs a biological feature recognition method, in particular relates to a face image recognition method and system.

Background technique

At present, there are different face image recognition methods for personal identification, but these methods generally have the problems of low recognition rate, slow image processing and recognition speed. This makes it difficult to apply face recognition technology to embedded systems.

Contents of the invention

The object of the present invention is to overcome the defective of prior art, propose and realize a kind of face image recognition method that can keep better recognition rate while greatly improving processing speed; facial image recognition system.

In order to achieve the above object, the face image recognition method of the present invention is to first set up the user template library, and then identify according to the following steps: 1) collect the face image to be recognized, 2) extract the template to be recognized, 3) user face recognition; To realize a complete face recognition system, it is necessary to provide two main functions of user face registration and face recognition.

Described user's facial recognition is: compare the template to be identified with the first template of each user in the user template library, obtain all similarity scores greater than the user list A1 of the first lower limit threshold value, according to the similarity score from large to Small arrangement; if A1 is empty, the recognition fails. If the similarity score of the first user in A1 is greater than the upper threshold, the recognition succeeds and returns the corresponding user. If not, proceed to the next step: combine the template to be recognized with each user in A1 Comparing the 2nd to 5th templates, get a list A2 of all users whose scores are greater than the second lower limit threshold, and arrange them in descending order of similarity scores; if A2 is empty, the recognition fails, if the first user in A2 has a similarity If the score is greater than the upper limit threshold, the recognition is successful and the corresponding user is returned. If not, proceed to the next step: compare the template to be recognized with the 6th-15th template of each user in A2, and obtain a list of all users whose similarity scores are greater than the recognition threshold A3 , if A3 is empty, then the recognition fails; according to the similarity score in descending order, the recognition function returns the corresponding user;

The user template library is that when the user is registered, the user changes five postures, collects three frames of images under each posture and extracts the feature templates, obtains a total of 15 feature templates, and selects the templates sequentially from the candidate template set The sorting method is based on the following two principles: A. The selected template represents the unselected template as much as possible, that is, it has the greatest similarity with other candidate templates; B. The selected template is as far away from the selected template as possible the resulting template, i.e. it retains a minimum similarity to the selected template;

The upper threshold is greater than the identification threshold, the identification threshold is greater than the second lower threshold, and the second lower threshold is greater than the first lower threshold. It has the advantage of being able to greatly increase the processing speed while maintaining a good recognition rate.

As an optimization, after the recognition is successful, if the similarity score between the template to be recognized and the user is greater than a set learning threshold that is greater than the upper threshold, the template collected on-site this time will be used as a template to be learned to learn according to the following steps: The user template library imports the templates to be learned, and combines the templates to be learned and the templates registered by the user to form a new template to be sorted. Sorting: if the last template is a template to be learned, then the learning fails and returns to the corresponding user; if not, remove the last one The template is stored in the database, and the corresponding user is returned after successful learning.

As an optimization, the set learning threshold is 100.

As an optimization, the first lower threshold is 43, the upper threshold is 90, the second lower threshold is 60, and the identification threshold is 80.

As an optimization, the sorting method of the 15 feature templates is as follows: select the first template so that the average similarity score between it and other templates is the largest, then move it to the selected template, and then select the second template so that its The average similarity score with other templates is the largest, then move it to the selected template, and then select the third template to make the average similarity score with other templates the largest, then move it to the selected template, and so on until there are no candidate templates.

As an optimization, the similarity score between the selected template and other templates is calculated according to the following formula:

C=a1×c1-a2×c2;

In the formula: c1 represents the average similarity score between this template and other candidate templates, c2 represents the average similarity score between this template and the selected template, and a1 and a2 are two parameters.

As an optimization, the value of parameter a1 is 5/9, and the value of parameter a2 is 4/9.

As an optimization, the user template library is built through the following steps: 1) collecting user face images, 2) extracting user feature templates, 3) registering users;

The 1) collection of the face image to be identified and 1) the collection of the user's face image are: using an infrared LED lighting source to irradiate the collected face, and filtering the visible light during the collection process;

Described 2) extracting template to be identified and 2) extracting the steps of user feature template are successively: 1) detecting human face; 2) positioning eye position; 3) normalizing human face image; 4) evaluating human face image quality; 5) Gabor feature extraction.

As an optimization, the 1) face detection is: detect the harr feature on the integral map, and then use the AdaBoost algorithm to detect the face; the 2) positioning the eye position is: through the symmetry of the face and the gray of the eye position 3) normalized face image is: first, rotate the image according to the positions of the left and right eyes so that the left and right eyes are on the same horizontal line, and then zoom For the entire image, the positions of the left and right eyes are normalized to a fixed width; further, according to the position of the eyes, the image of the entire face is intercepted according to the fixed width and height; finally, the image is locally enhanced to eliminate the image caused by light The brightness difference that inequalities produce; Obtain the image of key part of people's face at last; Said 4) evaluate the quality of people's face image is: in the processing process that image is carried out to normalization above, calculate the sharpness, contrast of image simultaneously and gray level, and these three indicators that describe the image quality at the same time only proceed to the next step if they exceed the respective thresholds set in advance; the 5) Gabor feature extraction is: we use N frequency bands and M directions A set of two-dimensional Gabor wavelet bases filter the normalized image to obtain the response energy of the image under this set of wavelet bases. After normalizing the response energy, the final face feature template is obtained.

As an optimization, the image of the key part of the human face obtained is as follows: take the distance between the left and right eyes as 75 pixels, start from the position of moving up 35 pixels and moving left 38 pixels, and take an image with a width of 151 and a height of 151;

Said N≥8, M≥16.

The facial image recognition system that is used to realize the method for the present invention is to use embedded microprocessor as the central processing unit of system, central processing unit connects the CMOS sensor of gathering facial image, and is supplemented with the infrared illuminating device of human face to realize any The use function under light conditions, using a narrow-band infrared filter to filter visible light to eliminate the interference of ambient light. It is a complete and reliable face recognition system, its recognition rate and recognition speed have reached a satisfactory level, and can be applied to attendance, access control and other systems. In terms of collecting user's face images, a good system should collect images as clear as possible and eliminate interference and noise as much as possible, because this is very important for the accuracy of recognition. Considering the use conditions of a real application system, that is, the system may be installed in various lighting environments, even used in the dark, so we cannot rely on the environment to provide suitable and uniform lighting. Based on this reason, the infrared LED lighting source is specially designed in the present invention to solve this problem. At the same time, the infrared rays will not cause harm to the user, and because of its invisibility, it will not cause interference to the user.

After adopting the above technical solution, the face image recognition method of the present invention performs user registration and recognition, which can effectively solve the problems of low recognition rate and slow recognition speed, and can greatly improve the processing speed while its speed and recognition rate can reach level of commercial application.

The face image recognition system of the present invention has the advantages of being able to be used under any light conditions, can eliminate the interference of ambient light, has good reliability, and has the advantages of satisfactory recognition rate and recognition speed.

Description of drawings

Fig. 1 is the principle schematic diagram of face image recognition system of the present invention;

Fig. 2 is the main flow schematic diagram of face image recognition system of the present invention;

Fig. 3 is the schematic flow chart of template extraction of face image recognition method of the present invention;

Fig. 4 is a schematic diagram of the user registration process of the face image recognition method of the present invention;

Fig. 5 is a schematic diagram of the template sorting flow chart of the face image recognition method of the present invention;

Fig. 6 is a schematic diagram of the user identification process of the face image identification method of the present invention;

Fig. 7 is a schematic diagram of the face template self-learning process of the face image recognition method of the present invention.

Detailed ways

As shown in Figures 2-7, the face image recognition method of the present invention is to first establish a user template library, and then identify according to the following steps: 1) collect the face image to be recognized, 2) extract the template to be recognized, 3) user face identification;

Described user's face recognition is: the template to be identified is compared with the first template of each user in the user template storehouse, obtains all user lists A1 that the similarity score is greater than 43, arranges by similarity score from large to small; If A1 is empty, the recognition fails. If the similarity score of the first user in A1 is greater than the upper limit threshold of 90, the recognition succeeds and returns the corresponding user. If not, proceed to the next step: combine the template to be recognized with the first user in A1 Compare 2-5 templates to get a list A2 of all users whose scores are greater than 60, and arrange them in descending order of similarity scores; if A2 is empty, the recognition fails, and if the first user in A2 has a similarity score greater than the upper limit threshold of 90 , then the recognition is successful and the corresponding user is returned. If not, go to the next step: compare the template to be recognized with the 6th-15th template of each user in A2, and obtain a list A3 of all users whose similarity scores are greater than the threshold 80. If A3 is If it is empty, the recognition fails, and the similarity scores are arranged in descending order, and the recognition function returns the corresponding user;

The user template library is that when the user is registered, the user changes five postures, collects three frames of images under each posture and extracts the feature templates, obtains a total of 15 feature templates, and selects the templates sequentially from the candidate template set The sorting method is based on the following two principles: A. The selected template represents the unselected template as much as possible, that is, it has the greatest similarity with other candidate templates; B. The selected template is as far away from the selected template as possible The resulting template, i.e. it retains a minimum similarity to the selected template.

After the identification is successful, if the similarity score between the template to be identified and the user is greater than a set threshold 100 greater than the upper limit threshold 90, then the template collected on-site this time is used as a template to be learned to learn according to the following steps: The user template library imports the templates to be learned, and combines the templates to be learned and the templates registered by the user to form a new template to be sorted. Sorting: if the last template is a template to be learned, then the learning fails and returns to the corresponding user; if not, remove the last one The template is stored in the database, and the corresponding user is returned after successful learning.

The sorting method of the 15 feature templates is: select the first template so that the average similarity score between it and other templates is the largest, then move it to the selected template, and then select the second template so that it has the highest similarity score with other templates. The average similarity score of the template is the largest, then move it to the selected template, and then select the third template to make the average similarity score between it and other templates the largest, then move it to the selected template, and so on until there are no candidates template.

The similarity score between the selected template and other templates is calculated according to the following formula:

C=a1×c1-a2×c2;

In the formula: c1 represents the average similarity score between the template and other candidate templates, c2 represents the average similarity score between the template and the selected template, a1 and a2 are two parameters; the value of parameter a1 is 5/9, parameter The value of a2 is 4/9.

Described user template storehouse is built through the following steps: 1) collect user's face image, 2) extract user feature template, 3) register user;

The 1) collection of the face image to be identified and 1) the collection of the user's face image are: using an infrared LED lighting source to irradiate the collected face, and filtering the visible light during the collection process;

Described 2) extracting template to be identified and 2) extracting the steps of user feature template are successively: 1) detecting human face; 2) positioning eye position; 3) normalizing human face image; 4) evaluating human face image quality; 5) Gabor feature extraction.

The 1) detecting the face is: detecting the harr feature on the integral image, and then using the AdaBoost algorithm to detect the face; the 2) locating the eye position is: the symmetry of the face and the darker gray level of the eye position feature, locate the positions of the left and right eyes on the face of the picture; the 3) normalized face image is: first rotate the image according to the positions of the left and right eyes so that the left and right eyes are on the same horizontal line, and then zoom the entire image, Normalize the positions of the left and right eyes to a fixed width; further, according to the position of the eyes, the image of the entire face is intercepted according to the fixed width and height; finally, the image is locally enhanced to eliminate the uneven illumination in the image The difference in brightness; finally obtain the image of the key parts of the face; the 4) evaluation of the image quality of the face is: in the above process of normalizing the image, the sharpness, contrast and grayscale of the image are calculated at the same time level, and these three indicators that describe the image quality at the same time only proceed to the next step if they exceed the respective thresholds set in advance; the 5) Gabor feature extraction is: we use a group of N frequency bands and M directions The two-dimensional Gabor wavelet base filters the normalized image to obtain the response energy of the image under this group of wavelet bases. After normalizing the response energy, the final face feature template is obtained. The image of the key part of the described acquisition of people's face is: take the distance between the left and right eyes to be 75 pixels, start from the position of moving up 35 pixels and moving left 38 pixels, get the image with a width of 151 and a height of 151; ≥8, M≥16.

More specifically: In order to realize a complete face recognition system, it is necessary to provide two main functions of user face registration and face recognition.

The registration process is to extract the feature template from the collected face image (in biometrics, the term "template" template refers to the extracted biological feature) and save it in the database; while the comparison process is to save the collected face image to the database After the feature template is extracted from the face image, it is compared with the templates stored in the database during registration until a matching template is found.

1. Collect face images

The key to these two functions is to collect the user's face image and extract the features of the face from the image. In terms of collecting user's face images, a good system should collect images as clear as possible and eliminate interference and noise as much as possible, because this is very important for the accuracy of recognition. Considering the use conditions of a real application system, that is, the system may be installed in various lighting environments, even used in the dark, so we cannot rely on the environment to provide suitable and uniform lighting. Based on this reason, the infrared LED lighting source is specially designed in the present invention to solve this problem. At the same time, the infrared rays will not cause harm to the user, and because of its invisibility, it will not cause interference to the user.

2. Extract feature template

The process of extracting feature templates from original images is usually the most critical part of biometric technology, as is face recognition technology. In the present invention, we implement the following feature extraction process, which can quickly and effectively extract the feature template of the human face.

1) Face detection: detect harr features on the integral map, and then use the AdaBoost algorithm for face detection;

2) Locating eye position: through the symmetry of the face and the darker gray level of the human eye position, the position of the left and right eyes on the face of the person in the picture is located;

3) Normalize the face image: We first rotate the image according to the positions of the left and right eyes so that the left and right eyes are on the same horizontal line, and then scale the entire image to normalize the positions of the left and right eyes to a fixed width; further, we According to the position of the eyes, the image of the entire face is intercepted according to a fixed width and height; finally, we locally enhance the image to eliminate the brightness difference in the image due to uneven illumination. We take the distance between the left and right eyes as 75 pixels, start from the position of 35 pixels up and 38 pixels left, and take an image with a width of 151 and a height of 151, which happens to be an image of a key part of a face.

4) Evaluate the quality of face image: We found that the quality of face image has a great influence on the recognition accuracy. Features extracted from poor image quality are not reliable. Therefore, in order to improve the performance of the system, we put forward appropriate requirements for the image quality. In the process of normalizing the image above, the sharpness, contrast and gray level of the image are calculated at the same time, and these three indicators simultaneously describe the quality of the image. We set corresponding thresholds, and only proceed to the next step if these thresholds are exceeded.

5) Gabor feature extraction: We use a set of two-dimensional Gabor wavelet bases with 8 frequency bands and 16 directions to filter the normalized image, so as to obtain the response energy of the image under this set of wavelet bases. After normalizing the response energy, we get the final face feature template.

3. Registered users

Since the face is three-dimensional, the images collected by the face in different poses will be very different, because the image of a pose can only describe the two-dimensional features of the face in this pose, and it is difficult to fully express the three-dimensional features of the face Come. So if we only register a user based on a frame of face images, we will inevitably get a very low recognition rate. Therefore, we need to collect multiple images of the face in multiple poses in order to obtain as many 3D features of the face as possible. Only in this way can an acceptable recognition rate be obtained in practical applications.

For further processing, the usual face recognition technology is to fuse images of multiple poses, analyze the geometric expression of the three-dimensional features of the face, and build a feature template based on this, which is the so-called 3D face recognition technology. However, this method brings a large computational overhead and is not very accurate, making it impossible to implement in embedded systems. By analyzing these face pictures of different poses, we found that the similarity of images of the same face with similar poses is very high, while the similarity of images of different faces with different poses is very low. Based on this, the present invention designs a method for user registration and identification, which can effectively solve the problems of low recognition rate and slow recognition speed.

When we register the user, we require the user to change five postures, and the system will collect three frames of images in each posture and extract the feature template. This way we get a total of 15 feature templates. During the collection process, the user is prompted to change posture through voice, and the system automatically collects and calculates at the same time, which makes the process not too difficult to use. Since in actual use, the user often does not strictly follow the time of the voice prompt to change the posture, and the user's posture may not fully meet our requirements, so we cannot ensure that the last 15 templates collected are accurate. There are 3 templates for each pose. This prevents us from simply extracting a template of each pose as a representative for subsequent processing. The present invention simply stores these 15 templates in the database, but sorts them first.

We sort by sequentially selecting templates from the candidate template set, based on two principles:

A. The selected template represents the unselected template as much as possible, that is, it has the greatest similarity with other candidate templates;

B. The selected template is as far away from the selected template as possible, that is, it maintains the minimum similarity with the selected template; these two principles ensure that the first 5 templates can basically represent the facial features of 5 poses. The next 10 templates are basically repetitions of the previous templates, with some additions. Through experiments, we obtained the best values of the parameters a1 and a2 in the process above as a1=5/9, a2=4/9.

5. Face recognition

In the recognition process, the process of collecting face images and extracting feature templates is the same as when registering faces. Of course, at this time, we can only recognize immediately after collecting a frame of image.

The result of matching two face feature templates is a similarity score ranging from 0 to 120. A score of 120 indicates that the two faces match exactly; a score of zero indicates no match at all. In order to improve the recognition rate, we will not use 120 as the basis for judging the same face, but select a threshold. When the similarity score is greater than this threshold, it is concluded that the two face templates come from the same face. According to the test results in a large-scale face database, when the threshold is 80, our system can get a false recognition rate of about 1/100000. We adopt this threshold in common applications.

The matching process we designed is divided into three stages according to our storage order of user fingerprint templates: the first one is only identified in the first template of each user, but takes a lower threshold (the first lower threshold) , such as 43, so that we can filter out some users for the second stage of comparison. After passing the first stage of matching, about 70% of the users will be filtered out, so only about 30% of the users participating in the second stage of comparison are left. In the second stage of comparison, we choose a higher threshold (the second lower limit threshold), such as 60, so that 95% of users will be filtered out, so only 5 users are left to participate in the third stage of comparison % or so. In the third stage, we compare the remaining 10 templates of the last 5% of users selected one by one to obtain the final result. It can be seen from this step that the complexity of template matching calculation in this process is:

Template matching calculation times = C+C*30%*4+C*30%*4*5%*10＝2.8*C

Where C is the number of people in the database. Taking 1000 people as an example, we need to perform template matching calculations up to 2800 times. In fact, our tests show that the vast majority of templates that cannot be unmatched in the end will usually not get any possible matching results in the second stage of matching, and thus will not enter the third stage of matching. That is to say, the computational complexity in the actual process will be much lower than that calculated by the above formula.

We also set an upper threshold, such as 90, in each stage of the comparison, once a template is compared with our on-site template that exceeds this upper threshold, it can be considered immediately that a matching template has been found without continuing Alignment in the next step. In this way, matching results can be obtained quickly.

Therefore, although the present invention saves 15 templates in the database for each user, through the above process, all 15 templates can be brought into play, greatly improving the recognition rate, and because there is not much increase in template comparison The number of times, the whole system can eventually identify quickly.

6. Learning of face templates

The characteristics of human faces will also change over time, such as fat and thin changes. Therefore, our system is best able to reflect these changes to the user's registration template during use. In this way, as time goes by, the user's registration template will also be kept updated, that is to say, it has an automatic learning function, so that the system will not eventually fail to recognize the user's face changes to a certain extent.

We set a strategy. In the process of user use, if the recognition is successful, if the matching score is greater than a given upper threshold, such as 100 points, then we are sure that the recognition result is absolutely correct, so we put The template collected on-site this time is used as a template to be learned.

In this process, we use the same method as when registering templates to sort the 15 templates registered by the user and the templates to be learned together, and then remove the last template.

After learning successfully, just update the database and write the 15 templates reordered by the user.

The algorithm flow and technical solutions described in this invention have been applied to various devices of ours. These devices adopt Marvel's embedded application processor PXA310, whose speed and recognition rate have reached the commercial application level.

As shown in Figure 1-2, the face image recognition system that the present invention is used to realize the above-mentioned method of the present invention is to use embedded microprocessor as the central processing unit of system or microprocessor core board 1, central processing unit or microprocessor The core board 1 is connected to the CMOS sensor 2 for collecting face images, supplemented by an infrared lighting device for the face, an infrared LED light source 3 to realize the use function under any light conditions, and a narrow-band infrared filter 4 is used to filter visible light to eliminate the interference of ambient light . In Figure 1: the user is 5, the infrared light irradiated by the light source is 61, and the infrared light reflected by the face is 62. Among them: the central processing unit or microprocessor core board adopts Marvel's embedded application processor PXA310.

More specifically: the present invention uses an embedded microprocessor as the central processing unit of the system, connects a CMOS sensor to collect face images, and supplements infrared lighting to realize nighttime use, and filters visible light to eliminate the interference of ambient light. Therefore, the present invention realizes a complete and reliable face recognition system, whose recognition rate and recognition speed reach a satisfactory level, and can be applied to systems such as time attendance and access control.

Claims (4)

1. a face image recognition method, it is characterized in that set up user's template storehouse earlier, then identify according to the following steps: 1) gather the face image to be identified, 2) extract template to be identified, 3) user's face recognition; Described user's facial recognition is: compare the template to be identified with the first template of each user in the user template library, obtain all similarity scores greater than the user list A1 of the first lower limit threshold value, according to the similarity score from large to Small arrangement; if A1 is empty, the recognition fails. If the similarity score of the first user in A1 is greater than the upper threshold, the recognition succeeds and returns the corresponding user. If not, proceed to the next step: combine the template to be recognized with each user in A1 Comparing the 2nd to 5th templates, get a list A2 of all users whose scores are greater than the second lower limit threshold, and arrange them in descending order of similarity scores; if A2 is empty, the recognition fails, if the first user in A2 has a similarity If the score is greater than the upper limit threshold, the recognition is successful and the corresponding user is returned. If not, proceed to the next step: compare the template to be recognized with the 6th-15th template of each user in A2, and obtain a list of all users whose similarity scores are greater than the recognition threshold A3 , if A3 is empty, then the recognition fails; according to the similarity score in descending order, the recognition function returns the corresponding user; The user template library is that when the user is registered, the user changes five postures, collects three frames of images under each posture and extracts the feature templates, obtains a total of 15 feature templates, and selects the templates sequentially from the candidate template set The sorting method is based on the following two principles: A. The selected template represents the unselected template, that is, it has the greatest similarity with other candidate templates; B. The selected template is far away from the selected template, i.e. it maintains a minimum similarity to the selected template; The upper threshold is greater than the identification threshold, the identification threshold is greater than the second lower threshold, and the second lower threshold is greater than the first lower threshold. 2. according to the said face image recognition method of claim 1, it is characterized in that after the recognition is successful, if the template to be recognized and the user similarity score are greater than a set learning threshold greater than the upper limit threshold, then the on-site collection As a template to be learned, learn according to the following steps: import the template to be learned to the user template library, combine the template to be learned and the template registered by the user to form a new template to be sorted, sort: if the last template is the template to be learned, If the learning fails, the corresponding user will be returned. If not, the last template will be removed and stored in the database. If the learning is successful, the corresponding user will be returned. 3. The face image recognition method according to claim 2, characterized in that the set learning threshold is 100. 4. The face image recognition method according to claim 1, wherein the first lower threshold is 43, the upper threshold is 90, the second lower threshold is 60, and the recognition threshold is 80.