CN105095880B - A kind of multi-modal Feature fusion of finger based on LGBP coding - Google Patents

Abstract

A multimodal feature fusion method for fingers based on LGBP coding. It includes using Gabor filter to perform Gabor filtering on the three-modal ROI image of the finger to obtain an amplitude characteristic image; encoding the image to form a characteristic encoding image; dividing the image; taking the pixels of the divided image as feature points Extract its grayscale features to form a grayscale feature vector; superimpose the grayscale feature vectors to form a grayscale feature histogram, and then concatenate the grayscale feature histograms to form the grayscale feature histogram of the block image; The grayscale feature histograms are fused in series to form the finger trimodal grayscale feature histogram; the grayscale feature histogram intersection coefficients of the two finger trimodal ROI images to be matched are calculated to determine whether the two match. The invention effectively solves the problem that the finger posture is volatile in the process of finger image collection, and has high operation speed and high recognition rate for finger multi-modal recognition.

Description

A Finger Multimodal Feature Fusion Method Based on LGBP Coding

technical field

The invention belongs to the technical field of image recognition, and in particular relates to a finger multimodal feature fusion method based on LGBP coding.

Background technique

At present, there are certain limitations in the application of single-modal biometric identification, so it cannot meet people's needs for high-precision identification. In order to effectively fuse the three-modal features of the finger, robust feature analysis has become a key issue in the research. Due to the problem of variable finger posture in the process of collecting three-modal ROI (region of interest) images of fingers, and most of the finger robust feature extraction methods are limited by rotation invariance, they cannot be effective. solve this problem.

SUMMARY OF THE INVENTION

In order to solve the above problems, the purpose of the present invention is to provide a multi-modal feature fusion method of fingers based on LGBP coding.

In order to achieve the above-mentioned purpose, the finger multimodal feature fusion method based on LGBP coding provided by the present invention comprises the following steps in order:

1) Use Gabor filters with different scale parameters to perform Gabor filtering on the three-modal ROI images of fingers with different postures, and obtain 8 directions respectively, namely 0°, 22.5°, 45°, 67.5°, 90°, 112.5°, 135° Amplitude characteristic images of fingerprints, finger veins and knuckle prints at ° and 157.5°;

2) Utilize LBP to encode the three-modal amplitude characteristic images of the finger in the above-mentioned eight directions respectively, thereby forming the three-modal LGBP characteristic encoding images of the finger in eight directions;

3) dividing the three-modal LGBP feature encoding images of the fingers in the above 8 directions;

4) The pixel points of each block image are regarded as feature points and their grayscale features are extracted, thereby forming a grayscale feature vector. The process is as follows:

Step 1: Grayscale grouping; first, sort the grayscale values of each pixel of each block image from small to large to form a sequence of pixels; then, divide the sequence equally according to the total number of pixels For k gray-scale groups, form k groups of gray-scale grouped images; then use the method of rounding to determine the boundary point of each gray-scale group, and obtain the gray value of the boundary point;

Step 2: Calculate the grayscale feature vector of each pixel: take each pixel in each grayscale grouped image as the center, and compare the grayscale values of its symmetrical neighbors. If the grayscale of a pixel is If the value is greater than the gray value of its symmetrical neighbor, it is 1; otherwise, it is 0, thus forming a grayscale feature vector of 4-bit binary code, and then converting the 4-bit binary code vector into a 16-bit binary code grayscale feature vector;

5) Superimpose the grayscale feature vectors of each pixel in the above-mentioned each grayscale grouped image to form a grayscale feature histogram of each grayscale grouped image, and then combine the grayscale feature histogram of each grayscale grouped image. The grayscale feature histogram of the block image is formed in series;

6) First, generate the coefficients with the same number of blocks as the LGBP feature-encoded image through the two-dimensional Gaussian model, then weight the gray feature histogram of each block image above, and then use the gray scale of the weighted block image. The degree feature histograms are concatenated to obtain a finger single-modal gray feature histogram, and finally, the above-mentioned finger single-modal gray feature histograms are concatenated and fused to form a finger three-modal gray feature histogram;

7) Determine whether the two finger ROI images match by calculating the intersection coefficient of the grayscale feature histogram of the two finger tri-modal ROI images to be matched.

In step 1), the expression of the described Gabor filter is:

Among them, δ represents the scale of the Gabor filter, δ=4, 5, 6; θ _k represents the angle value of the k-th direction.

In step 2), the method for encoding the three-modal amplitude characteristic images of the finger in 8 directions is: first, define a 3-mode image with a certain pixel in a certain finger amplitude characteristic image as the center pixel. ×3 window, take the gray value of the center pixel as the threshold, and binarize the remaining 8 neighbor pixels in the window; if the gray value of a neighbor pixel is less than the gray value of the center pixel degree value, it is encoded as 0; otherwise, it is encoded as 1, forming an 8-bit binary value; then the binary value is shifted to the right by b times, and the weighted summation is performed on the binary value shifted by one bit to the right to obtain the pixel value. 8 LBP values; finally, take the smallest LBP value as the LBP value of the pixel;

The formula for the minimum LBP value is:

Among them, the function ROR(x,b) means to shift the binary value x to the right b times, represents the LBP value of the i-th center pixel, The definition is shown in formula (3):

In the formula: B(I _i -I _c ) represents the binarization function, namely I _i represents the gray value of the central pixel point i, I _c represents the gray value of the neighboring pixel point, and a represents the a-th bit of the binarization function, where P=8.

In step 4), the described gray value formula of the obtained boundary point is:

in, represents the boundary point of each group, t _i represents the boundary value of the i-th grayscale group, and _Imin and _Imax represent the minimum grayscale value and the maximum grayscale value of the image pixel, respectively.

In step 4), the described grayscale feature vector of the 4-bit binary code is converted into the formula of the grayscale feature vector of the 16-bit binary code:

Among them, i represents the ith pixel, and m represents the logarithm of the nearest neighbor of the pixel.

In step 6), the formula of the two-dimensional Gaussian model is:

Among them, σ represents the mean square error of the two-dimensional Gaussian model, m and n are the number of image blocks in each row and each column, respectively, mid(i) and mid(j) represent the block image at the center of the image. row and column j.

In step 7), the method for judging whether these two finger ROI images match by calculating the intersection coefficients of the grayscale feature histograms of the two finger trimodal ROI images to be matched is: first use the following The intersection coefficient expression calculates the intersection coefficient of the three-modal gray feature histogram of the finger in the two finger ROI images to be matched. If the calculated intersection coefficient > the similarity decision threshold T, it means that the two finger ROI images are similar , which means that the two finger ROI images match; if the intersection coefficient is less than or equal to T, it is determined that the two finger ROI images do not match; the similarity decision threshold T is that the false rejection rate in the finger ROI image matching result is 0, and the error is allowed The threshold point corresponding to the lowest rate;

The expression for the intersection coefficient is:

where m ₁ and m ₂ represent two finger ROI images to be matched respectively, H _m1 (i) and H _m2 (i) respectively represent the grayscale feature histograms of the two finger tri-modal ROI images to be matched, L represents the dimension of the grayscale feature histogram of the finger trimodal image.

The finger multimodal feature fusion method based on LGBP coding provided by the present invention effectively solves the problem of variable finger posture during the finger image acquisition process, and has high operation speed and high recognition rate for finger multimodal recognition.

Description of drawings

Figure 1 is a three-modal amplitude feature map of a finger in 8 directions, wherein (a) fingerprint (b) finger vein (c) knuckle pattern;

Figure 2 is a three-modal LGBP feature encoding diagram of a finger in 8 directions, wherein (a) fingerprint (b) finger vein (c) knuckle pattern;

Figure 3 is a block diagram of three-modal LGBP feature coding of fingers in 8 directions, wherein (a) fingerprints (b) finger veins (c) knuckle prints;

FIG. 4 is a schematic diagram of 8 nearest neighbors of a pixel.

Fig. 5 is the grayscale feature histogram of the LGBP feature encoding block image, wherein (a) fingerprint (b) finger vein (c) knuckle print;

6 is a schematic diagram of a weighted series process based on a two-dimensional Gaussian model;

Figure 7 is a comparison of the recognition performance of different gray-scale groups of 8×8 block images;

Fig. 8 is the recognition performance comparison of different block images;

Figure 9 is a comparison of the recognition performance of the two-dimensional Gaussian model with different σ values;

Figure 10 shows four finger vein ROI images with different postures;

Figure 11 shows the comparison of the recognition performance of the three feature extraction methods.

Detailed ways

The LGBP coding-based finger multimodal feature fusion method provided by the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

The finger multimodal feature fusion method based on LGBP coding provided by the present invention comprises the following steps in order:

1) Since the textures of the three-modal images of the fingers are different, the present invention uses Gabor filters with different scale parameters (δ=4, 5, 6) to perform Gabor filtering on the three-modal ROI images of the fingers with different postures. The expression is shown in Equation 1, and the amplitude features of fingerprints, finger veins and knuckle prints in 8 directions (0°, 22.5°, 45°, 67.5°, 90°, 112.5°, 135° and 157.5°) are obtained respectively. image, as shown in Figure 1;

Among them, δ represents the scale of the Gabor filter, and θ _k represents the angle value of the k-th direction.

2) Use LBP (local binary pattern) to encode the three-modal amplitude characteristic images of the finger in the above 8 directions respectively. The encoding method is: first, define a certain pixel in a certain finger amplitude characteristic image as the center pixel. A 3×3 window of points, with the gray value of the central pixel as the threshold, the remaining 8 neighboring pixels in the window are binarized. For example: if the gray value of a neighborhood pixel is less than the gray value of the center pixel, the code is 0; otherwise, the code is 1, forming an 8-bit binary value; then the binary value is shifted to the right b times , weighted and summed the binary values shifted by one bit to the right to obtain 8 LBP values of the pixel; finally, take the smallest LBP value as the LBP value of the pixel, as shown in formula (2), thus forming The three-modal LGBP feature-encoded images of fingers in 8 directions are shown in Figure 2.

Among them, the function ROR(x,b) means to shift the binary value x to the right b times, represents the LBP value of the i-th center pixel, The definition is shown in formula (3):

In the formula: B(I _i -I _c ) represents the binarization function, namely I _i represents the gray value of the central pixel point i, I _c represents the gray value of the neighboring pixel point, and a represents the a-th bit of the binarization function, where P=8.

3) Since MRRID (Multi-Support Region Rotation Invariance Feature) is only suitable for describing local images, this step divides the three-modal LGBP feature encoding images of the finger in the above 8 directions. The three-modal LGBP feature encoding image of the finger is divided into 8×8 blocks respectively, and the block diagram is shown in Figure 3.

4) Since the three-modal LGBP feature coding segmented images of the fingers in the above 8 directions are small, detailed information will be lost if the feature points are searched. The points are regarded as feature points and their grayscale features are extracted to form grayscale feature vectors. The process is as follows:

The first step: grayscale grouping. First, sort the gray value of each pixel of each block image from small to large to form a sequence of pixels; then, divide the sequence into k gray-scale groups according to the total number of pixels to form K groups of gray-scale grouped images; then use the method of rounding to determine the boundary point of each gray-scale group, and obtain the gray value of the boundary point, as shown in formula (4):

in, represents the boundary point of each group, t _i represents the boundary value of the i-th grayscale group, and _Imin and _Imax represent the minimum grayscale value and the maximum grayscale value of the image pixel, respectively.

Step 2: Calculate the grayscale feature vector of each pixel. Since each pixel has 8 nearest neighbors, the present invention takes each pixel in each grayscale grouped image as the center, and compares the grayscale values of its symmetrical neighbors, for example: and is a pair of symmetrical neighbors of pixel i, where, is the nearest neighbor point labeled 1 of pixel i, is the nearest neighbor point labeled 5 of pixel i, as shown in Figure 4, if The gray value of the point is greater than The gray value of the point is 1; otherwise, it is 0, thus forming a grayscale feature vector of 4-bit binary code, and then using formula (5) to convert the 4-bit binary code vector into a 16-bit binary code grayscale feature vector.

Among them, i represents the ith pixel, and m represents the logarithm of the nearest neighbor of the pixel.

5) Superimpose the grayscale feature vectors of each pixel in the above-mentioned each grayscale grouped image to form a grayscale feature histogram of each grayscale grouped image, and then combine the grayscale feature histogram of each grayscale grouped image. The grayscale feature histogram of the block image is formed in series, and the LGIGF feature of the block image is represented by the histogram. Assuming that the number of segments of the three-modal image of the finger is N=8×8, and the number of gray-level groups is k=5, then the gray-scale feature histograms of the segmented images in the first row and the first column corresponding to the three single-modalities of the fingers The diagram is shown in Figure 5.

6) Since the LGIGF feature of the central part of the three-modal image of the finger is more stable than the LGIGF feature of the edge part, first, the coefficients equal to the number of blocks of the LGBP feature encoding image are generated by the two-dimensional Gaussian model, and then each of the above The grayscale feature histogram of the block image is weighted, and then the grayscale feature histogram of the above-mentioned weighted block image is connected in series to obtain the finger single-modal grayscale feature histogram. The two-dimensional Gaussian model is shown in formula (6):

Among them, σ represents the mean square error of the two-dimensional Gaussian model, m and n are the number of image blocks in each row and each column, respectively, mid(i) and mid(j) represent the block image at the center of the image. row and column j. The schematic diagram of the weighted concatenation of the LGIGF feature histogram of the block image is shown in Figure 6.

Finally, the above single-modal grayscale feature histograms of the finger are fused in series to form a finger three-modal grayscale feature histogram.

In addition, according to formula (4), the value of the number of gray-scale grouped images k is related to the size of the divided image, that is, the number of pixels contained in the divided image is different, and the optimal value of k is also different. Therefore, we use the ROC (acceptance characteristic curve) curve to determine the optimal number k of gray-scale grouped images and the optimal number of blocks N, so that the LGIGF feature histogram matching accuracy is the best. First, we assume that N=8×8, then it can be seen from Figure 7 that when k=4, the LGIGF feature histogram matching accuracy is the highest; according to the previous result, assuming k=4, it can be seen from Figure 8 that when N=8× 8, the LGIGF feature histogram matching accuracy is the highest. According to the above results, when N=8×8, k=4, the recognition performance of the LGIGF feature fusion method proposed by the present invention is the best. According to formula (6), it can be seen that the stability of the single-modal image feature of the finger is related to the shape of the two-dimensional Gaussian model. Since the shape of the two-dimensional Gaussian model depends on the value of the mean square error σ, the single-modal image feature of the finger is The stability of σ depends on the mean square error σ. It can be seen from Figure 9 that when σ = 0.15, the matching accuracy of the LGIGF feature histogram is the highest.

7) Using formula (7), determine whether the two finger ROI images match by calculating the intersection coefficient of the grayscale feature histogram of the two finger trimodal ROI images to be matched, and the larger the intersection coefficient of the histogram, more likely to match.

where m ₁ and m ₂ represent two finger ROI images to be matched respectively, H _m1 (i) and H _m2 (i) respectively represent the grayscale feature histograms of the two finger tri-modal ROI images to be matched, L represents the dimension of the grayscale feature histogram of the finger trimodal image.

In the above-mentioned image matching process, the intersection coefficient of the three-modal grayscale feature histogram of the finger in the two finger ROI images to be matched is calculated first. If the calculated intersection coefficient > T (similarity decision threshold), it means that the two finger ROI images are similar, which means that the two finger ROI images match; if the intersection coefficient is less than or equal to T, then the two finger ROI images are judged Images do not match. The similarity decision threshold T is the corresponding threshold point when the false rejection rate is 0 and the false allowable rate is the lowest in the finger ROI image matching result.

The inventors conducted two groups of experiments based on the above method. In these two sets of experiments, self-made databases were used. This database contains 100 different individuals, and each individual contains 10 fingerprint ROI images, 10 finger vein ROI images, and 10 knuckle print ROI images. A total of 3000 finger trimodal ROI images. And the gestures of the single-modal images of the fingers of each individual are different. Since the resolutions of the single-modal images of the fingers in the database will be different, the resolutions of the fingerprints, finger veins, and knuckle prints in the self-made database are adjusted to 152*152, 88*200, and 88*200, respectively. The experimental environment is a PC, completed in the Matlab R2010a environment.

In the first set of experiments, four finger vein ROI images were selected from a self-made database, as shown in Figure 10. The four pictures belong to the same person, and the poses are different.

In this experiment, we use the following three feature fusion methods represented by histograms to verify the rotation invariance of the LGIGF feature fusion method proposed by the present invention.

1. LGBP feature coding: First, according to the description of steps 1 and 2, four LGBP feature coding images of finger veins are formed; then, according to the description of step 3, the 4 LGBP feature coding images of finger veins are divided into 8× 8 blocks, and then describe each LGBP feature-encoded block image through the traditional grayscale histogram representation method. The traditional histogram representation method is: count the number of pixels in the LGBP feature-encoded image from grayscale values 0 to 255, A straight line is superimposed at each gray value to form the grayscale histogram of the LGBP feature-encoded block images of the four finger veins; finally, the grayscale histograms of the LGBP feature-encoded block images of each of the four finger veins are connected in series Histograms of LGBP feature-encoded images of 4 finger veins were formed.

2. Improved MRRID features: First, according to the description of step 3, divide the four finger vein ROI images into 8×8 blocks; then, according to the description of step 4, describe each finger vein ROI block image by the improved MRRID, The improved MRRID feature histogram of each finger vein ROI segmented image is formed; finally, the improved MRRID feature histogram of each finger vein ROI segmented image is concatenated to form the improved MRRID feature histogram of 4 finger vein ROI images.

3. LGIGF feature: First, according to the description of step 1 and step 2, 4 LGBP feature coding images of finger veins are formed; then, according to the description of step 3, the 4 LGBP feature coding images of finger veins are divided into 8 × 8 block, and then describe each LGBP feature encoding block image through the improved MRRID described in step 4 to form an improved MRRID feature histogram of each LGBP feature encoding block image; finally, the two-dimensional Gaussian model described in step 5 will The improved MRRID feature histogram of each LGBP feature encoded block image is weighted and concatenated to form an improved MRRID feature histogram of the LGBP feature encoded image, namely the LGIGF feature histogram proposed in the present invention.

According to the above-mentioned feature histogram formation process, the feature histograms of the four finger vein ROI images were matched respectively, and their intersection coefficients were compared, as shown in Table 1. From the data in Table 1, it can be seen that the LGIGF feature fusion method has good rotation invariance characteristics, and this method solves the problem of changing finger poses to a certain extent.

Table 1 Histogram similarity coefficient

In the second set of experiments, we compared the recognition performance of the LGIGF feature fusion method, the LGBP feature encoding method described in steps 1 and 2, and the improved MRRID feature fusion method described in step 4. The ROC curve is shown in Figure 11. shown. Table 2 shows the matching time of these three feature fusion methods and the corresponding EER (the recognition rate where the false allowable rate is equal to the false rejection rate). From the matching time of different feature fusion methods in Table 2, it can be seen that the LGIGF feature fusion method has a shorter matching time. Combined with the experimental results in Figure 11 and Table 2, it can be seen that the LGIGF feature fusion method not only solves the problem of changing finger poses to a certain extent, but also has a good matching effect and improves the matching efficiency, which is feasible to a certain extent.

Table 2 Recognition performance of different descriptors

Claims (5)

1. A finger multi-modal feature fusion method based on LGBP coding is characterized in that: the LGBP coding-based finger multi-modal feature fusion method comprises the following steps which are carried out in sequence:

1) gabor filtering is carried out on finger three-mode ROI images of different postures by utilizing Gabor filters with different scale parameters, and amplitude characteristic images of fingerprints, finger veins and finger joint prints in 8 directions, namely 0 degree, 22.5 degrees, 45 degrees, 67.5 degrees, 90 degrees, 112.5 degrees, 135 degrees and 157.5 degrees are obtained respectively;

2) respectively encoding the finger three-mode amplitude characteristic images in the 8 directions by using LBP (local binary pattern) so as to form finger three-mode LGBP characteristic encoded images in the 8 directions;

3) partitioning the finger three-mode LGBP feature coding image in the 8 directions;

4) the pixel points of each block image are regarded as feature points to extract the gray features of the feature points, so that gray feature vectors are formed, and the process is as follows:

the first step is as follows: gray grouping; firstly, sequencing the gray value of each pixel point of each block image from small to large to form a pixel point sequence; then, dividing the sequence into k gray groups according to the total number of the pixel points to form k groups of gray group images; then determining the boundary point of each gray grouping by a rounding method, and acquiring the gray value of the boundary point;

the second step is that: calculating the gray characteristic vector of each pixel: taking each pixel point in each gray grouping image as a center, comparing the gray values of the symmetrical adjacent points, and if the gray value of a certain pixel point is greater than the gray value of the symmetrical adjacent point, determining that the gray value is 1; otherwise, the gray level feature vector is 0, so that a gray level feature vector of the 4-bit binary code is formed, and then the 4-bit binary code vector is converted into a gray level feature vector of the 16-bit binary code;

5) superposing the gray characteristic vectors of each pixel point in each gray grouping image to form a gray characteristic histogram of each gray grouping image, and then connecting the gray characteristic histograms of each gray grouping image in series to form a gray characteristic histogram of a block image;

6) firstly, generating coefficients with the same number as the number of blocks of an LGBP feature coding image through a two-dimensional Gaussian model, then weighting a gray feature histogram of each block image, then connecting the weighted gray feature histograms of the block images in series to obtain a finger single-mode gray feature histogram, and finally connecting the finger single-mode gray feature histograms in series and fusing to form a finger three-mode gray feature histogram;

7) and judging whether the two finger ROI images are matched or not by a method of calculating the intersection coefficient of the gray feature histograms of the two finger three-mode ROI images to be matched.

2. The LGBP coding-based finger multimodal feature fusion method according to claim 1, wherein: in step 2), the method for encoding the finger three-mode amplitude feature images in the 8 directions respectively comprises the following steps: firstly, defining a 3 x 3 window with a certain pixel point in a certain finger amplitude characteristic image as a central pixel point, and carrying out binarization on the rest 8 neighborhood pixel points in the window by taking the gray value of the central pixel point as a threshold value; if the gray value of a certain neighborhood pixel point is smaller than the gray value of the central pixel point, the code is 0; otherwise, the code is 1, and an 8-bit binary value is formed; then shifting the binary value to the right for b times, and performing weighted summation on the binary value shifted by one bit to the right to obtain 8 LBP values of the pixel point; finally, taking the minimum LBP value as the LBP value of the pixel point;

the formula for the minimum LBP value is:

wherein the function ROR (x, b) represents shifting the binary value x b times to the right,indicating the LBP value of the ith center pixel,the definition is shown in formula (3):

in the formula: b (I)_i-I_c) Representing a binarization function, i.e.I_iRepresenting the gray value of the central pixel I, I_cThe gray value of the neighborhood pixel point is represented, a represents the a-th bit of the binarization function, and P is 8.

3. The LGBP coding-based finger multimodal feature fusion method according to claim 1, wherein: in step 4), the gray value formula for obtaining the boundary point is as follows:

wherein, representing the boundary points, t, of each group_iRepresenting the boundary value, I, of the ith gray grouping_minAnd I_maxRespectively representing the minimum gray value and the maximum gray value of the image pixel points.

4. The LGBP coding-based finger multimodal feature fusion method according to claim 1, wherein: in step 4), the formula for converting the gray level feature vector of the 4-bit binary code into the gray level feature vector of the 16-bit binary code is as follows:

wherein i represents the ith pixel point, and m represents the logarithm of the nearest neighbor point of the pixel point.

5. The LGBP coding-based finger multimodal feature fusion method according to claim 1, wherein: in step 7), the method for determining whether two finger ROI images match by calculating the intersection coefficient of the grayscale feature histograms of the two finger three-modality ROI images to be matched is: firstly, calculating the intersection coefficient of finger three-mode gray feature histograms in two finger ROI images to be matched by using the following expression of the intersection coefficient, and if the calculated intersection coefficient is greater than a similarity decision threshold T, indicating that the two finger ROI images are similar, namely indicating that the two finger ROI images are matched; if the intersection coefficient is less than or equal to T, judging that the two finger ROI images are not matched; the similarity decision threshold T is a threshold point corresponding to the finger ROI image matching result with the error rejection rate of 0 and the lowest error allowance rate;

the expression of the intersection coefficient is:

in the formula: m is₁And m₂Respectively representing two images of the finger ROI to be matched,andand the L represents the dimension of the gray feature histogram of the finger three-mode image.