CN105138974B - A kind of multi-modal Feature fusion of finger based on Gabor coding - Google Patents

Abstract

A finger multimodal feature fusion method based on Gabor coding, which includes using Gabor filter to perform Gabor filtering on ROI images of fingerprints, finger veins and knuckle prints to obtain Gabor direction feature images; Arrange and encode them to form a Gabor direction feature encoding image; divide the above image into blocks; extract the grayscale features of the pixels in the divided image, thereby forming grayscale feature vectors; superimpose the grayscale feature vectors A finger trimodal grayscale feature histogram is formed; whether the two finger ROI images match is determined by calculating the intersection coefficient of the trimodal grayscale feature histogram of the two finger ROI images to be matched. The method of the invention effectively solves the problem that the finger posture is volatile in the process of finger image acquisition, and has high operation speed and high recognition rate for finger multi-modal recognition.

Description

A Finger Multimodal Feature Fusion Method Based on Gabor Coding

technical field

The invention belongs to the technical field of image detection, and in particular relates to a method for fusing multi-modal features of fingers based on Gabor coding.

Background technique

At present, due to certain limitations in the application of single-modal biometric identification, it cannot meet people's needs for high-precision identification. In order to effectively integrate the three-modal features of fingers, robust feature analysis has become a research topic key issues in. However, because most research methods of finger robustness feature extraction rely on the position information and orientation information of feature points, and are limited by rotation invariance, they cannot effectively solve the problem that the finger pose is easily changed during the image acquisition process.

SUMMARY OF THE INVENTION

In order to solve the above problems, the purpose of the present invention is to provide a method for fusing multi-modal features of fingers based on Gabor coding.

In order to achieve the above object, the method for fusing finger multimodal features based on Gabor coding provided by the present invention comprises the following steps in order:

1) Use Gabor filters with different scale parameters to perform Gabor filtering on the ROI images of fingerprints, finger veins and knuckle prints with different postures, and obtain 8 directions, namely 0°, 22.5°, 45°, 67.5°, 90° , 112.5°, 135° and 157.5° finger trimodal Gabor orientation feature images;

2) arranging the angle values of the three-modal Gabor direction feature images of the fingers in the above-mentioned 8 directions respectively from small to large, and encoding them, thereby forming the three-modal Gabor direction feature encoding images of the fingers in 8 directions;

3) The three-modal Gabor direction feature encoding image of the finger in the above-mentioned 8 directions is divided into blocks to form a block image;

4) The pixel points in the above-mentioned 8 directions of the finger tri-modal Gabor direction feature encoding block image are regarded as feature points and their grayscale features are extracted, thereby forming a grayscale feature vector, and the process is as follows:

Step 1: Grayscale grouping: First, sort the grayscale values of each pixel in each segmented image from small to large to form a sequence of pixels; then, divide the sequence equally according to the total number of pixels For k gray-scale groups, form k groups of gray-scale grouped images; then use the method of rounding to determine the boundary point of each gray-scale group, and obtain the gray value of the boundary point;

Step 2: Calculate the grayscale feature vector of each pixel: take each pixel in each grayscale grouped image as the center, and compare the grayscale values of its symmetrical neighbors. If the grayscale of a pixel is If the value is greater than the gray value of its symmetrical neighbor, it is 1; otherwise, it is 0, thus forming a grayscale feature vector of 4-bit binary code, and then converting the 4-bit binary code vector into a 16-bit binary code grayscale feature vector;

5) Superimpose the grayscale feature vectors of each pixel in the above-mentioned each grayscale grouped image to form a grayscale feature histogram of each grayscale grouped image, and then combine the grayscale feature histogram of each grayscale grouped image. The grayscale feature histograms of the segmented images are formed in series, and then the grayscale feature histograms of all the segmented images of the three single modalities of the finger are fused in series to form three single-modal grayscale feature histograms. , and finally the grayscale feature histograms of the three single-modal images are fused in series to form a finger three-modal grayscale feature histogram;

6) Determine whether the two finger ROI images match by calculating the intersection coefficient of the three-modal grayscale feature histogram of the two finger ROI images to be matched.

In step 1), the described Gabor filter expression is:

Among them, σ represents the scale of the Gabor filter, σ=4, 5, 6; θ _k represents the angle value of the k-th direction.

In step 2), the angle values of the three-modal Gabor orientation feature images of the fingers in the eight directions are arranged from small to large, and then encoded, thereby forming the three-modal Gabor of the fingers in eight directions. The method of encoding the image by the direction feature is: first, arrange the angle values of the three-modal Gabor direction feature images of the finger in the above 8 directions from small to large, and then compare the gray values of the pixels at the same position in the above image respectively. , respectively take the direction of the three-modal Gabor direction feature image of the finger corresponding to the maximum gray value as the direction feature of the pixel, and encode it according to the following encoding methods: 0° is encoded as 0, 22.5° is encoded as 1, and 45° is encoded The encoding is 2, 67.5° is encoded as 3, 90° is encoded as 4, 112.5° is encoded as 5, 135° is encoded as 6, and 157.5° is encoded as 7; thus forming a three-modal Gabor orientation feature encoding image of the finger in 8 directions .

In step 4), the described formula for obtaining the gray value of the boundary point is:

in, represents the boundary point of each group, t _i represents the boundary value of the i-th grayscale group, and _Imin and _Imax represent the minimum grayscale value and the maximum grayscale value of the image pixel, respectively.

In step 4), the described grayscale feature vector of the 4-bit binary code is converted into the formula of the grayscale feature vector of the 16-bit binary code:

Among them, i represents the ith pixel, and m represents the logarithm of the nearest neighbor of the pixel.

In step 6), the method for judging whether the two finger ROI images match by calculating the intersection coefficient of the three-modal grayscale feature histogram of the two finger ROI images to be matched is: first use the following The intersection coefficient expression calculates the intersection coefficient of the three-modal grayscale feature histograms of the two finger ROI images to be matched. If the calculated intersection coefficient is greater than the similarity decision threshold T, it means that the two finger ROI images are similar, That is, it means that the two finger ROI images match; if the intersection coefficient is less than or equal to T, it is determined that the two finger ROI images do not match. The similarity decision threshold T is the corresponding threshold point when the false rejection rate is 0 and the false allowable rate is the lowest in the finger ROI image matching result.

The expression for the intersection coefficient is:

where m ₁ and m ₂ represent two finger ROI images to be matched respectively, H _m1 (i) and H _m2 (i) respectively represent the grayscale feature histograms of the two finger tri-modal ROI images to be matched, L represents the dimension of the finger trimodal image histogram.

The finger multi-modal feature fusion method based on Gabor coding provided by the present invention effectively solves the problem of variable finger posture in the process of finger image acquisition, and has high operation speed and high recognition rate for finger multi-modal recognition.

Description of drawings

Fig. 1 is a three-mode Gabor direction feature map of a finger in 8 directions; (a) is a fingerprint; (b) is a finger vein; (c) is a knuckle print;

Fig. 2 is a three-modal Gabor direction feature encoding diagram of a finger in 8 directions; wherein (a) is a fingerprint; (b) is a finger vein; (c) is a knuckle print;

Fig. 3 is the block image of the three-modal Gabor direction feature encoding map of the finger in 8 directions; wherein (a) is a fingerprint; (b) is a finger vein; (c) is a knuckle print;

FIG. 4 is a schematic diagram of 8 nearest neighbors of a certain pixel.

Fig. 5 is the grayscale feature histogram of the three-modal Gabor direction feature coding block image of the finger in 8 directions; wherein (a) is a fingerprint, (b) is a finger vein, and (c) is a knuckle print;

Figure 6 is a comparison of the recognition performance of different gray-scale groups of 8*8 block images;

Figure 7 is a comparison of the recognition performance of different segmented images.

Fig. 8 is the ROI image of the finger vein in different postures.

Figure 9. Comparison of recognition performance of three feature extraction methods.

Detailed ways

The method for fusing finger multimodal features based on Gabor coding provided by the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

The finger multimodal feature fusion method based on Gabor coding provided by the present invention comprises the following steps in order:

1) Using Gabor filters with different scale parameters to perform Gabor filtering on the ROI images of fingerprints, finger veins and knuckle prints with different postures, and obtain three-modal Gabor orientation feature images of fingers in 8 directions respectively;

Since fingerprint, finger vein and knuckle print images have ridge line structure, pipeline structure and trace line structure respectively, the texture information is relatively rich, so this step adopts the Gabor filtering method to extract the direction features of the three-modal texture of the finger. According to the different characteristics of the three-modal image texture of the finger, Gabor filters with different scale parameters (σ=4, 5, 6) are used to analyze the ROI (region of interest) of fingerprints, finger veins and knuckle prints of different postures. The image is subjected to Gabor filtering. The expression of the Gabor filter is shown in formula (1), and the 8 directions (0°, 22.5°, 45°, 67.5°, 90°, 112.5°, 135° and 157.5°) are obtained respectively. The three-modal Gabor orientation feature image of the finger is shown in Figure 1.

Among them, σ represents the scale of the Gabor filter, and θ _k represents the angle value of the k-th direction.

2) arranging the angle values of the three-modal Gabor direction feature images of the fingers in the above-mentioned 8 directions respectively from small to large, and encoding them, thereby forming the three-modal Gabor direction feature encoding images of the fingers in 8 directions;

First, the angle values of the three-modal Gabor direction feature images of the fingers in the above 8 directions are arranged from small to large, and then the gray values of the pixels at the same position in the above images are compared respectively, and the finger corresponding to the largest gray value is compared. The orientation of the three-modal Gabor orientation feature image is used as the orientation feature of the pixel, and is encoded according to the following encoding methods: 0° is encoded as 0, 22.5° is encoded as 1, 45° is encoded as 2, and 67.5° is encoded as 3. 90° is coded as 4, 112.5° is coded as 5, 135° is coded as 6, and 157.5° is coded as 7; for example: if a pixel at the same position on a single-modal orientation feature image of a certain 8 fingers is in the 45° direction The gray value on the feature image is the largest, then the pixel is encoded as 2 in the single-modal Gabor direction feature encoding image of the finger, thus forming the three-modal Gabor direction feature encoding image of the finger in 8 directions, as shown in Figure 2. Show.

3) The three-modal Gabor direction feature encoding image of the finger in the above-mentioned 8 directions is divided into blocks to form a block image;

Since MRRID (Multi-Support Region Rotation Invariance Feature) is only suitable for describing local images, this step divides the above-mentioned 8 directions of the finger tri-modal Gabor direction feature encoding image into blocks. In the present invention, the above 8 directions are The three-modal Gabor direction feature encoding image of the finger is divided into 8 × 8 blocks respectively, and the block diagram is shown in Figure 3.

4) The pixel points in the above-mentioned 8 directions of finger three-modal Gabor direction feature encoding block images are regarded as feature points and their grayscale features are extracted, thereby forming grayscale feature vectors;

Since the three-modal Gabor direction feature encoding of the finger in the above 8 directions is small, the detailed information will be lost if the feature points are searched. Therefore, this step improves the MRRID, and the The pixels are regarded as feature points and their grayscale features are extracted, thereby forming a grayscale feature vector. The process is as follows:

The first step: grayscale grouping. First, sort the gray value of each pixel in each block image from small to large to form a sequence of pixels; then, divide the sequence into k gray-scale groups according to the total number of pixels to form K groups of gray-scale grouped images; then use the method of rounding to determine the boundary point of each gray-scale group, and obtain the gray value of the boundary point, as shown in formula (2):

in, represents the boundary point of each group, t _i represents the boundary value of the i-th grayscale group, and _Imin and _Imax represent the minimum grayscale value and the maximum grayscale value of the image pixel, respectively.

Step 2: Calculate the grayscale feature vector of each pixel. Since each pixel has 8 nearest neighbors, the present invention takes each pixel in each grayscale grouped image as the center, and compares the grayscale values of its symmetrical neighbors, for example: and is a pair of symmetrical neighbors of pixel i, where, is the nearest neighbor point labeled 1 of pixel i, is the nearest neighbor point labeled 5 of pixel i, as shown in Figure 4, if The gray value of the point is greater than The gray value of the point is 1; otherwise, it is 0, thus forming a grayscale feature vector of a 4-bit binary code, and then using the formula (3) to convert the grayscale feature vector of the 4-bit binary code into a 16-bit binary code. The grayscale feature vector of .

Among them, i represents the ith pixel, and m represents the logarithm of the nearest neighbor of the pixel.

5) Superimpose the grayscale feature vectors of each pixel in the above-mentioned each grayscale grouped image to form a grayscale feature histogram of each grayscale grouped image, and then combine the grayscale feature histogram of each grayscale grouped image. The grayscale feature histograms of the segmented images are formed in series, and then the grayscale feature histograms of all the segmented images of the three single modalities of the finger are fused in series to form three single-modal grayscale feature histograms. , using this histogram to represent the GLGF feature of the single-modal image, if the number of blocks of the single-modal Gabor direction feature encoding image of the finger in each 8 directions is N, then the single-modal gray feature histogram dimension is N*k*16. Here, assuming that N=8×8 and k=5, the grayscale feature histograms of the segmented images in the first row and the first column corresponding to the three single modalities are shown in FIG. 5 . Finally, the grayscale feature histograms of the three single-modality images are fused in series to form a finger three-modal grayscale feature histogram.

In addition, according to formula (2), it can be known that the value of the number k of gray-scale groups is related to the size of the segmented image, that is, the number of pixels contained in the segmented image is different, and the optimal value of k is also different. Therefore, the present invention determines the optimal number of grayscale groups k and the optimal number of blocks N through the ROC (acceptance characteristic curve) curve, so that the matching accuracy of the three-modal grayscale feature histogram of the finger is the highest. First, we assume that N=8×8, then it can be seen from Figure 6 that when k=7, the matching accuracy of the three-modal grayscale feature histogram of the finger is the highest; When N=8×8, the matching accuracy of finger three-modal grayscale feature histogram is the highest. According to the above results, when N=8×8, k=7, the recognition performance of the method of the present invention is the best.

6) Judging whether these two finger ROI images match by calculating the intersection coefficient of the three-modal grayscale feature histogram of the two finger ROI images to be matched;

Using formula (4), it is determined whether the two finger ROI images match by calculating the intersection coefficient of the three-modal grayscale feature histograms of the two finger ROI images to be matched. more likely.

where m ₁ and m ₂ represent the two finger ROI images to be matched, respectively, and H _m1 (i) and H _m2 (i) represent the three-modal grayscale feature histograms of the two finger ROI images to be matched, respectively , L represents the dimension of the three-modal gray feature histogram of the finger.

In the above image matching process, firstly, the intersection coefficient of the three-modal grayscale feature histograms of the two finger ROI images to be matched are calculated. If the calculated intersection coefficient > T (similarity decision threshold), it means that the two finger ROI images are similar, which means that the two finger ROI images match; if the intersection coefficient is less than or equal to T, then the two fingers are judged ROI images do not match. The similarity decision threshold T is the corresponding threshold point when the false rejection rate is 0 and the false allowable rate is the lowest in the finger ROI image matching result.

The inventors conducted two groups of experiments based on the above method. In these two sets of experiments, the database of the three modalities of the finger was collected by the self-made system. This database contains 300 different individuals, and each individual contains 10 fingerprint ROI images, 10 finger vein ROI images and 10 knuckle print ROI images. A total of 9000 finger trimodal ROI images. And the gestures of the single-modal images of the fingers of each individual are different. Since the resolutions of the single-modal images of the fingers in the database will be different, the resolutions of the fingerprints, finger veins, and knuckle prints in the self-made database are adjusted to 152*152, 88*200, and 88*200, respectively. The experimental environment is a PC, completed in the Matlab R2010a environment.

In the first set of experiments, four finger vein ROI images were selected from a self-made database, as shown in Figure 8. The four finger vein ROI images belong to the same person with different postures.

In this experiment, we use the following three feature extraction methods represented by histograms to verify the rotation invariance of the GLGF feature extraction method proposed by the present invention.

1. Gabor direction feature encoding: First, through the description of step 1 and step 2, four images of Gabor direction feature encoding of finger veins are formed; then, through the description of step 3, the four images of Gabor direction feature encoding of finger veins are formed. Divide it into 8×8 blocks, and then describe each block image of Gabor direction feature encoding by the traditional gray histogram representation method. The traditional histogram representation method is: count the pixels of the Gabor direction feature encoding image from gray value 0 to 255 The number of points is superimposed at each gray value to form a straight line, which constitutes the grayscale histogram of the Gabor direction feature coding block images of 4 finger veins; finally, the Gabor direction feature coding of each 4 finger veins is divided into blocks The grayscale histograms of the images are concatenated to form the histograms of the four images of the Gabor orientation feature encoding of finger veins.

2. Improved MRRID features: First, according to the description of step 3, the 4 finger vein ROI images are divided into 8 × 8 blocks; then, according to the description of step 4, each finger vein ROI segmented image is described by the improved MRRID, The improved MRRID feature histogram of each finger vein ROI segmented image is formed; finally, the improved MRRID feature histogram of each finger vein ROI segmented image is concatenated to form the improved MRRID feature histogram of the finger vein ROI image.

3. GLGF features: First, through the description of steps 1 and 2, four images of Gabor orientation feature encoding of finger veins are formed; then, through the description of step 3, the four images of Gabor orientation feature encoding of finger veins are divided into 8 ×8 blocks, and then describe each Gabor direction feature-encoded block image through the improved MRRID described in step 4 to form an improved MRRID feature histogram of each Gabor direction feature-encoded block image; The improved MRRID feature histogram of the encoded block image is concatenated to form the improved MRRID feature histogram of the Gabor direction feature encoded image, that is, the GLGF feature histogram proposed by the present invention.

According to the above-mentioned feature histogram formation process, the feature histograms of the four finger vein ROI images were matched respectively, and their intersection coefficients were compared, as shown in Table 1. From the data in Table 1, it can be seen that the GLGF feature extraction method has good rotation invariance characteristics, and this method solves the problem of changing finger poses to a certain extent.

Table 1 Histogram similarity coefficient

In the second set of experiments, we compared the recognition performance of the GLGF feature extraction method, the Gabor orientation feature encoding method described in steps 1 and 2, and the improved MRRID feature extraction method described in step 4, and the ROC (acceptance characteristic Curve) The curve is shown in Figure 9. Table 2 shows the matching time of these three feature extraction methods and the corresponding EER (the recognition rate where the false allowable rate and the false rejection rate are equal, referred to as equal error rate in this paper). From the matching time of different feature extraction methods in Table 2, it can be seen that the matching time of the GLGF feature extraction method is shorter. Combined with the experimental results in Figure 9 and Table 2, it can be seen that the GLGF feature extraction method not only solves the problem of changing finger poses to a certain extent, but also has a good matching effect, and improves the matching efficiency, which is feasible to a certain extent.

Table 2. Recognition performance of different descriptors

Claims (4)

1. a finger multimodal feature fusion method based on Gabor coding, is characterized in that: the described finger multimodal feature fusion method based on Gabor coding comprises the following steps of carrying out in order: 1) Use Gabor filters with different scale parameters to perform Gabor filtering on the ROI images of fingerprints, finger veins and knuckle prints with different postures, and obtain 8 directions, namely 0°, 22.5°, 45°, 67.5°, 90° , 112.5°, 135° and 157.5° finger trimodal Gabor orientation feature images; 2) arranging the angle values of the three-modal Gabor direction feature images of the fingers in the above-mentioned 8 directions respectively from small to large, and encoding them, thereby forming the three-modal Gabor direction feature encoding images of the fingers in 8 directions; 3) The three-modal Gabor direction feature encoding image of the finger in the above-mentioned 8 directions is divided into blocks to form a block image; 4) The pixel points in the above-mentioned 8 directions of the finger tri-modal Gabor direction feature encoding block image are regarded as feature points and their grayscale features are extracted, thereby forming a grayscale feature vector, and the process is as follows: Step 1: Grayscale grouping: First, sort the grayscale values of each pixel in each segmented image from small to large to form a sequence of pixels; then, divide the sequence equally according to the total number of pixels For k gray-scale groups, form k groups of gray-scale grouped images; then use the method of rounding to determine the boundary point of each gray-scale group, and obtain the gray value of the boundary point; Step 2: Calculate the grayscale feature vector of each pixel: take each pixel in each grayscale grouped image as the center, and compare the grayscale values of its symmetrical neighbors. If the grayscale of a pixel is If the value is greater than the gray value of its symmetrical neighbor, it is 1; otherwise, it is 0, thus forming a grayscale feature vector of 4-bit binary code, and then converting the 4-bit binary code vector into a 16-bit binary code grayscale feature vector; 5) Superimpose the grayscale feature vectors of each pixel in the above-mentioned each grayscale grouped image to form a grayscale feature histogram of each grayscale grouped image, and then combine the grayscale feature histogram of each grayscale grouped image. The grayscale feature histograms of the segmented images are formed in series, and then the grayscale feature histograms of all the segmented images of the three single modalities of the finger are fused in series to form three single-modal grayscale feature histograms. , and finally the grayscale feature histograms of the three single-modal images are fused in series to form a finger three-modal grayscale feature histogram; 6) Determine whether the two finger ROI images match by calculating the intersection coefficient of the three-modal grayscale feature histogram of the two finger ROI images to be matched. 2. the finger multi-modal feature fusion method based on Gabor coding according to claim 1, is characterized in that: in step 2) in, described the angle of the finger three-modal Gabor direction feature image of above-mentioned 8 directions Arrange the values from small to large and encode them, thereby forming the three-modal Gabor orientation feature encoding image of the finger in 8 directions. The angle values are arranged from small to large, and then the gray values of the pixels at the same position in the above image are compared respectively, and the direction of the three-modal Gabor orientation feature image of the finger corresponding to the maximum gray value is taken as the direction of the pixel. feature, and encode it according to the following encoding method: 0° is encoded as 0, 22.5° is encoded as 1, 45° is encoded as 2, 67.5° is encoded as 3, 90° is encoded as 4, 112.5° is encoded as 5, and 135° is encoded as 5. is 6, and 157.5° is coded as 7; thus, the three-modal Gabor orientation feature coding image of the finger in 8 directions is formed. 3. the finger multimodal feature fusion method based on Gabor coding according to claim 1, is characterized in that: in step 4) in, the described grayscale feature vector of 4 binary codes is converted into 16 binary codes The formula for the grayscale feature vector of is: Among them, i represents the ith pixel, and m represents the logarithm of the nearest neighbor of the pixel. 4. the finger multi-modal feature fusion method based on Gabor coding according to claim 1, is characterized in that: in step 6), described by calculating the three-modal gray scale of two finger ROI images to be matched The method of the feature histogram intersection coefficient to judge whether the two finger ROI images match is: first, use the following intersection coefficient expression to calculate the intersection coefficient of the three-modal grayscale feature histograms of the two finger ROI images to be matched , if the calculated intersection coefficient > the similarity decision threshold T, it means that the two finger ROI images are similar, that is, this means that the two finger ROI images match; if the intersection coefficient is less than or equal to T, then the two finger ROI images are determined The images do not match; the similarity decision threshold T is the corresponding threshold point when the false rejection rate is 0 and the false allowable rate is the lowest in the finger ROI image matching result; The expression for the intersection coefficient is: In the formula: m ₁ and m ₂ respectively represent the two finger ROI images to be matched, and respectively represent the grayscale feature histograms of the two finger trimodal ROI images to be matched, and L represents the dimension of the finger trimodal image histogram.