CN107424141B - Face image quality evaluation method based on probability block - Google Patents

Abstract

The invention discloses a face image quality evaluation method based on probability blocks, which comprises the following steps: carrying out normalization processing on the image by adopting logarithmic transformation, amplifying low-intensity pixels and compressing high-intensity pixels, and reducing the intensity difference between skin colors; normalizing each block in the transformed image to enable the transformed image to have a zero mean value and a unit variance, and extracting a two-dimensional DCT (discrete cosine transformation) feature vector; and calculating the probability of the corresponding feature vector by adopting a position probability model, and generating an overall quality score by integrating the local probability to reflect the image quality. The invention avoids the influence of alignment errors caused by cast shadow, ambiguity and automatic face positioning on face recognition, and can be applied to solving the problems of face selection and video-based face recognition.

Description

A Probabilistic Block-Based Facial Image Quality Assessment Method

technical field

The invention relates to the field of facial image quality assessment, in particular to a probability block-based facial image quality assessment method.

Background technique

With the rapid development of monitoring systems and the reduction of application costs, more and more monitoring systems are applied in all aspects of people's lives. However, due to the existence of problems such as low resolution, blurred images, large posture changes and low contrast, the Conditional video-based identity inference is challenging ^[1] .

In recent years, many methods have been proposed to deal with the problem of face recognition in inferior images ^[2] . One approach is to assume that images are outliers in the sequence, however, when the majority of images in the sequence are of poor quality, these approaches classify good quality images as outliers; another approach is to explicitly Set selection, where facial quality assessment is performed automatically for each image, selecting a subset consisting of high-quality images. Improved recognition performance and reduced overall computational load, but it was difficult to find a good definition for "face quality".

ISO/IEC 19794-5 and ICAO 9303 are facial image standards for facial quality assessment, and based on the above standards, many methods have been proposed to analyze various facial and image attributes.

Since facial recognition performance is simultaneously affected by multiple factors, being able to detect one or two qualities is not sufficient for robust subset selection. Nasrollahi and Moeslund ^[3] proposed a weighted quality fusion method to combine out-of-plane rotation, sharpness, brightness and image resolution quality; Rua et al. ^[4] proposed a similar quality assessment method using asymmetric analysis and two sharpness measures; Hsu et al. ^[5] proposed to learn fusion parameters on multiple quality scores to achieve maximum correlation with matching scores between face image pairs; but due to the various properties are measured separately and have different effects on facial quality, and it is difficult for the above methods to combine them to output a single quality score for image selection; Luo ^[6] proposed a learning-based method in which a quality model is trained to match the manually labeled quality scores. However, given the subjective nature of human labeling, this approach may not yield the best quality models for facial recognition.

The main challenges facing the facial image quality assessment problem are: due to the existence of various problems such as alignment errors, pose changes, image shadows and low image clarity, the selection of the facial image with the best overall performance is greatly restricted; cast shadows The existence of problems such as automatic face positioning and other problems brings great difficulties to face recognition in videos.

SUMMARY OF THE INVENTION

The present invention provides a method for evaluating the quality of facial images based on probability blocks. The present invention avoids the influence of alignment errors on face recognition caused by projected shadows, ambiguity and automatic facial positioning, and improves the accuracy of recognition. See description below:

A face image quality assessment method based on probability block, the method comprises the following steps:

The image is normalized by logarithmic transformation, amplifying low-intensity pixels and compressing high-intensity pixels, reducing the intensity difference between skin tones;

Normalize each block in the transformed image to have zero mean and unit variance, and extract a two-dimensional DCT feature vector;

The localization probability model is used to calculate the probability of the corresponding feature vector, and the overall quality score is generated by integrating the local probability, reflecting the image quality.

The steps of normalizing each block in the transformed image to make it have zero mean and unit variance, and extracting the two-dimensional DCT feature vector are as follows:

To accommodate for contrast changes between face images, each block in the transformed image is normalized to have zero mean and unit variance; and from each block, a 2-dimensional DCT feature vector is extracted and excluded without normalization The 0th DCT component of the information, retains the first d low frequency components containing the generic facial texture.

Wherein, the positional probability model is used to calculate the probability of the corresponding feature vector, and the overall quality score is generated by integrating the local probability, and the steps of reflecting the image quality are as follows:

The model at each location is trained using frontal face images with frontal lighting and natural expressions, all face images to be trained are first scaled and aligned to a fixed size with each eye at a fixed position;

The probability of the corresponding feature vector of each block is calculated using a positional probability model, and the overall quality score is generated by integrating the local probabilities, assuming that the models at each position are independent.

The beneficial effects of the technical scheme provided by the present invention are:

1. The method has the best overall performance, being able to identify the most frontal, well-aligned, illuminated and sharp images;

2. For each given set of face images, the proposed method is used to rank the images according to their quality, significantly improving the recognition accuracy by selecting a subset containing only top-quality images.

Description of drawings

Figure 1 shows a method for evaluating the quality of facial images based on probability blocks.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention clearer, the embodiments of the present invention are further described in detail below.

Example 1

In order to solve the above problems, it is necessary to be able to perform face selection and face recognition comprehensively, automatically and accurately. Research has shown that patch-based local analysis can output a single score for each image by quantifying the similarity of a given face to a probabilistic face model, without the need to locate facial features and without resorting to fusion, with scores reflecting alignment Degree of error, pose changes, shadows, and image sharpness. An embodiment of the present invention proposes a probability block-based facial image quality assessment method, see FIG. 1 , and details are described below:

101: Use logarithmic transformation to normalize the image, amplify low-intensity pixels and compress high-intensity pixels, and reduce the intensity difference between skin colors;

102: Normalize each block in the transformed image so that it has zero mean and unit variance, and extract a two-dimensional DCT (Discrete Cosine Transform, discrete cosine transform) feature vector;

103: Calculate the probability of the corresponding feature vector by using the localization probability model, and generate an overall quality score by integrating the local probability, reflecting the image quality.

Wherein, in step 102, each block in the transformed image is normalized so that it has zero mean and unit variance, and the steps of extracting the two-dimensional DCT feature vector are as follows:

To accommodate for contrast changes between face images, each block in the transformed image is normalized to have zero mean and unit variance; and from each block, a 2-dimensional DCT feature vector is extracted and excluded without normalization The 0th DCT component of the information, retains the first d low frequency components containing the generic facial texture.

Further, in step 103, the positional probability model is used to calculate the probability of the corresponding feature vector, and the overall quality score is generated by integrating the local probabilities, and the steps of reflecting the image quality are as follows:

The model at each location is trained using frontal face images with frontal lighting and natural expressions, all face images to be trained are first scaled and aligned to a fixed size with each eye at a fixed position;

The probability of the corresponding feature vector of each block is calculated using a positional probability model, and the overall quality score is generated by integrating the local probabilities, assuming that the models at each position are independent.

To sum up, the embodiment of the present invention avoids the influence of alignment errors caused by projected shadows, ambiguity and automatic face positioning on face recognition, and improves the recognition accuracy.

Example 2

The scheme in Embodiment 1 is further introduced below in conjunction with the specific calculation formula and Fig. 1, and is described in detail below:

201: Normalize image I by logarithmic transformation, amplify low-intensity pixels and compress high-intensity pixels, and reduce the intensity difference between skin colors;

For a given image I, non-linear preprocessing (logarithmic transformation) is performed to reduce the dynamic range of the data, and the normalized image I _log is calculated using:

I _log (r,c)=ln[I(r,c)+1]

where I(r,c) is the pixel intensity at position (r,c). Logarithmic normalization amplifies low-intensity pixels and compresses high-intensity pixels, helping to reduce intensity differences between skin tones.

202: Normalize each block in the transformed image I _log so that it has zero mean and unit variance, and extract a two-dimensional DCT feature vector;

where the transformed image I _log is divided into N overlapping blocks, each block bi has the size of _n × n pixels and overlaps with adjacent blocks by t pixels, in order to accommodate the contrast changes between face images, each Slices are normalized to have zero mean and unit variance.

From each block, a 2-dimensional DCT feature vector is extracted, the 0th DCT component is excluded, the first d low-frequency components are preserved, and the low-frequency components retain the general facial texture while largely omitting individual-specific information, while, projecting Shadows and changes in pose and alignment can alter local textures.

203 : Calculate the probability of the corresponding feature vector _xi by using the localization probability model, and generate an overall quality score by integrating the local probabilities, reflecting the image quality.

The model at each location is trained using frontal face images with frontal lighting and natural expressions, all face images to be trained are first scaled and aligned to a fixed size with each eye at a fixed position.

For each block's position i, compute the probability of the feature vector x _i using a positional probability model:

Among them, x _i is the eigenvector, μ _i and Σ _i are the mean value and covariance matrix of the normal distribution, T represents the transposition, and d represents the number of low-frequency components.

By assuming the models at each location are independent, the overall probability quality score Q(I) for an image I consisting of N patches is calculated using:

The resulting quality score represents the probabilistic similarity of a given face image to an "ideal" face image, where the "ideal" face image is represented by a set of training images.

The higher the quality score, the better the image quality, and vice versa, the worse the image quality.

To sum up, the embodiment of the present invention avoids the influence of alignment errors caused by projected shadows, ambiguity and automatic face positioning on face recognition, and improves the recognition accuracy.

Example 3

Below in conjunction with concrete example, calculation formula, table 1, table 2, table 3, feasibility verification is carried out to the scheme in embodiment 1 and 2, see below for details:

The FERET and PIE databases were used to verify the performance of the method in correctly selecting images with desired properties. The Choke Point database is used to verify the effectiveness of our method in selecting a subset of video-based images for facial recognition.

FERET (The Facial Recognition Technology Database) database is one of the most widely used face databases in the field of face recognition. Among them, the 'fb' subset has blurred images and images with alignment errors, and the alignment errors are determined by horizontal and vertical displacement (shift 0, ±2, ±4, ±6, ±8 pixels), in-plane rotation (0 °, ±10°, ±20°, ±30°), scaling (using 0.7, 0.8, 0.9, 1.0, 1.1, 1.2, 1.3 as scaling factors), etc. This subset can be used to simulate blurred images as well as four alignment error images.

The PIE (Pose Illumination Expression) database was created by Carnegie Mellon University in the United States and contains facial images of multiple poses, lighting and expressions. A subset of its illumination is used to evaluate performance under various projection conditions. In an embodiment, the front view image is divided into six subsets based on light source angle, with subset 1 having the most frontal light sources and subset 6 having the largest light source angle (54°-67°). This subset can be used to verify the performance of the proposed method in the presence of cast shadows.

The Choke Point database is a video dataset designed for experiments on face recognition or verification under real-world surveillance conditions, the dataset consists of 25 subjects (19 males and 6 females) in Entrance 1 and Entrance 2 consists of 29 subjects (23 males and 6 females) in 48 video sequences and 64,204 facial images.

This method is compared with the following three methods to evaluate how the proposed quality assessment method determines the best quality face image when both good and poor quality face images are provided:

Asym_shrp ^[4] : a pixel-based asymmetry analysis and a fractional fusion method for two sharpness analyses;

Gabor_asym ^[7] : an asymmetric analysis method based on Gabor features;

DFFS ^[8] : (Distance From Face Space) A classic distance face space method.

In order to prove the effectiveness of this method for quality assessment of various facial recognition systems, without loss of generality, we use Multi-Region Histograms (MRH) and Local Binary Patterns (LBP) respectively. ) extract features from each face image; use the Mutual Subspace Method (MSM for short) to classify the set of faces.

Table 1

The results for shifts, rotations, and scale changes shown in Table 1 show that the method consistently achieves optimal or near-optimal performance in most variations; Gabor_asym is useful for detecting images with various sharpness changes. The performance is also poor; Asym_shrp addresses this by combining asymmetry analysis and two image sharpness measures, but its overall performance is still poor; DFFS performs worse than the proposed method and fails to detect Image with the best sharpness.

Table 2

The results on the 6 PIE lighting subsets in Table 2 show that this method can achieve good results even in the presence of cast shadows; Asym_shrp achieves the best performance (fronts are marked as having high-quality front-illuminated surfaces); By contrast, Gabor_asym confuses between subsets 1 and 4, while DFFS incorrectly labels most face images (containing significant shadows) in subset 4 as having the highest quality.

table 3

The results of face recognition performance verification based on video database in Table 3 show that the number of subsets varies from 4 to 16, regardless of which facial feature extraction algorithm is used, the proposed quality measurement method consistently obtains better results than the other three methods. face verification performance. The experimental results verify the feasibility and superiority of this method.

references:

[1] Dong Haibo. Surveillance video image quality assessment [D]. Shanghai Jiaotong University, 2013.

[2] Yang Yu. Research on key technologies of face recognition in video images [D]. Hebei University of Engineering, 2014.

[3] K.Nasrollahi and T.B.Moeslund.Face quality assessment system in video sequences.In BIOID, Lecture Notes in Computer Science (LNCS), volume 5372, pages 10–18, 2008..

[4] E.A.Rúa, J.L.A.Castro, and C.G.Mateo. Quality-based scorenormalization and frame selection for video-based person authentication. InBIOID, Lecture Notes in Computer Science (LNCS), pages 1–9, 2008.

[5] R.-L.V.Hsu, J.Shah, and B.Martin.Quality assessment of facialimages.In Biometrics Symposium, 2006.

[6] H. Luo. A training-based no-reference image quality assessment algorithm. In International Conference on Image Processing (ICIP), pages 2973–2976, 2004.

[7]J.Sang,Z.Lei,and S.Z.Li.Face image quality evaluation for ISO/IECstandards 19794-5 and 29794-5.In ICB,Lecture Notes in Computer Science(LNCS),volume 5558,pages 229–238 , 2009.

[8] H. Bae and S. Kim. Real-time face detection and recognition using hybrid-information extracted from face space and facial features. Image and Vision Computing, 23(13):1181–1191, 2005.

Those skilled in the art can understand that the accompanying drawing is only a schematic diagram of a preferred embodiment, and the above-mentioned serial numbers of the embodiments of the present invention are only for description, and do not represent the advantages or disadvantages of the embodiments.

The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection of the present invention. within the range.

Claims (2)

1. A method for probability block-based facial image quality assessment, the method comprising the steps of:

carrying out normalization processing on the image by adopting logarithmic transformation, amplifying low-intensity pixels and compressing high-intensity pixels, and reducing the intensity difference between skin colors; for a given image I, a logarithmic transformation is performed to reduce the dynamic range of the data, and the normalized image I is calculated using the following equation_log：

I_10g(r,c)＝In[I(r,c)+1]Wherein I (r, c) is the pixel intensity at the (r, c) position;

normalizing each block in the transformed image to enable the transformed image to have a zero mean value and a unit variance, and extracting a two-dimensional DCT (discrete cosine transformation) feature vector;

calculating the probability of the corresponding feature vector by adopting a position probability model, generating an overall quality score by integrating local probabilities, and reflecting the image quality;

the model for each position is trained using frontal facial images with frontal illumination and natural expression, all facial images to be trained are first scaled and aligned to a fixed size, with each eye in a fixed position; for each block position i, a feature vector x is calculated using a localized probability model_iProbability of (c):

wherein x is_iIs a feature vector, u_i、Σ_iIs the average value and covariance matrix of normal distribution, T represents transposition, d represents the number of low-frequency components;

the overall probabilistic mass fraction q (I) of an image I composed of N blocks is calculated using the following equation, by assuming that the model for each location is independent:

2. the method for evaluating the quality of a face image based on probability blocks according to claim 1, wherein the step of normalizing each block in the transformed image to have zero mean and unit variance and extracting the two-dimensional DCT feature vector comprises the steps of:

to accommodate contrast variations between facial images, each block in the transformed image is normalized to have zero mean and unit variance; and extracting 2-dimensional DCT feature vectors from each block, and excluding the 0 th DCT component without normalization information, and reserving the first d low-frequency components containing the general facial texture.

Images