CN101493891A - Characteristic extracting and describing method with mirror plate overturning invariability based on SIFT - Google Patents

Abstract

The invention belongs to the technical field of image processing, and relates to a SIFT-based feature extraction and description method with mirror flip invariance, comprising the following steps: (1) performing Gaussian kernel convolution processing on an input image; (2) continuing to process the image Perform Gaussian difference processing to detect its extreme points; (3) filter feature points; (4) accurately locate the position of feature points; (5) determine the direction parameters of each feature point; (6) pair the main direction as the dividing line (7) Organize the pixel units in the Gaussian weight window to perform encoding and normalization operations to generate image description data. The invention increases the robustness of the feature extraction and description method to the mirror imaging problem, and expands the application field of computer vision.

Description

Feature extraction and description method with mirror flip invariance based on SIFT

technical field

The invention belongs to the technical field of image processing and relates to an image feature extraction method.

Background technique

Today's computer technology is developing rapidly, and the application fields of computer vision and image retrieval are becoming more and more extensive, which also highlights their importance. Popular 3D reconstruction, object recognition, camera calibration, robot binocular navigation, etc. are all based on computer vision, so reasonably and effectively solving the problems existing in computer vision or improving imperfections can give the computer industry even A huge boost from the scientific community. Computer vision is based on the idea that the computer can simulate human (mammalian) vision to achieve a certain degree of intelligence. Together with image retrieval, it is necessary to extract and analyze the features of the image, so the definition and extraction scheme of image features has a pivotal role. Of course, there are many solutions today, and the common ones are gradient-based feature extraction and description methods, including Harriscorner detector [1], SIFT [2], SURF [3], HOG [4], GLOH [5]. In addition to gradient-based feature extraction methods, there are other methods based on contour extraction. Since the MIFT proposed by the present invention is a gradient-based method, other methods will not be described too much. Among them, the Harris corner detector (Harris corner detector) can extract feature points that are invariant to rotation and illumination on the scale of the image itself. In fact, it is not just extracting corner points as the name says, but all feature points with significant gradients in multiple directions. However, the limitations of the Harris corner detector are relatively large because it is very sensitive to the scaling of the image scale. In order to remove or weaken the influence of image scale changes, Lowe proposed SIFT (Scale Invariant Feature Transform) to solve the problems caused by scale scaling. Of course, it also ensures rotation invariance and even tolerates illumination and affine transformation to a certain extent. and covering effects. SURF, simply speaking, is an accelerated version of SIFT. Both it and SIFT adopt an auxiliary region strategy, that is to say, an auxiliary region is designated with the feature point as the center, and the pixels in this region jointly determine the description of the feature point. The difference is that SIFT uses a strategy to assign weights to different pixels in the region according to their contribution to feature points, while SURF only uses an equal weight strategy based on integral images. HOG combined with SVM (Support Vector Machine) provides a human detection method based on gradient information. GLOH is another variant of SIFT. It uses a circular strategy to organize the auxiliary area. The purpose is to enhance the robustness and prominence of features. The initial GLOH descriptor has 272 dimensions, but after PCA (Principal Component Analysis) operation, Reduce the 272 dimensions to the same 128 dimensions as SIFT, ensuring that the efficiency of the matching operation is improved without losing key information. Although all the above methods can well solve the deformation of images such as rotation, scale change and even illumination change and affine, almost all methods do not take into account the situation of mirror imaging, which is very difficult in real life. It is very common, such as water surface reflection, mirror imaging, observation of symmetrical objects from different angles, etc.

references:

【1】C.Harris and M.J.Stephens, "Corner and Edge Detector", Alvey Vision Conference, vol.20, pp.147-152, 1988.

【2】D.G.Lowe, "Unique Image Features Selected from Scale Invariant Key Points", International Journal of Computer Vision, vol.60, pp.91-110, 2004.

[3] H.Bay, T.Tuytelaars, and L.Van Gool., "Surf: Accelerated Extraction of Robust Features", European Conference on Computer Vision, pp.404-417, 2006.

【4】N.Dalal and B.Triggs, "Human Detection Based on Histogram of Oriented Gradients", International Conference on Computer Vision and Pattern Recognition, vol.1, pp.886-893, 2005.

【5】K.Mikolajczyk and C.Schmid, "Performance Estimation of Local Feature Descriptors", IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.27, pp.1651-1630, 2004.

Contents of the invention

The purpose of the present invention is to provide a method that can solve the problem of failure of the SIFT feature extraction method caused by the mirror flip phenomenon, while maintaining all the advantages and performances of SIFT. In other words, a feature extraction and description method that maintains the same description form before and after flipping is provided, that is, a feature extraction and description method that is invariant to mirror flipping. For this reason, the present invention adopts following technical scheme:

A SIFT-based feature extraction and description method with mirror flip invariance, comprising the following steps:

Step 1: Perform Gaussian kernel convolution processing on the input image I(x, y), that is, L(x, y, σ)=G(x, y, σ)*I(x, y), to obtain a multi-scale space Expressed image L(x, y, σ), where, $G(x,\mathrm{the\; y},\mathrm{\σ})=\frac{1}{2\mathrm{\π}{\mathrm{\σ}}^2}{e}^{-(x^2+{\mathrm{the\; y}}^2)/2{\mathrm{\σ}}^2},$ Where σ is the variance of the Gaussian normal distribution;

Step 2: Perform Gaussian difference processing on the image L(x, y, σ) expressed in multi-scale space according to the following formula D(x, y, σ)=(G(x, y, kσ)-G(x, y, σ))*I(x, y)=L(x, y, kσ)-L(x, y, σ), detecting the extreme points of the image L(x, y, σ) expressed in multi-scale space;

Step 3: Use threshold method and Hessian matrix method to screen feature points;

Step 4: Accurately locate the position of the feature points by fitting the three-dimensional quadratic curve;

Step 5: Determine the direction parameter of each feature point according to the gradient direction θ(x, y) and the size m(x, y) information on the auxiliary neighborhood pixels of the feature point, where:

$\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\sqrt{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; L</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; L</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span>{<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; L</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; L</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>{<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; ,</span>}$

$\mathrm{\&theta;}(x,\mathrm{the\; y})={\mathrm{the\; tan}}^{-1}\left(\frac{L(x,\mathrm{the\; y}+1)-L(x,\mathrm{the\; y}-1)}{L(x+1,\mathrm{the\; y})-L(x-1,\mathrm{the\; y})}\right),$ And in the form of a histogram, combined with the Gaussian weight window centered on the feature point, the main direction is determined according to the gradient size and direction of each pixel in the entire neighborhood;

上的梯度模值，％代表取模操作；Step 6: Sum the gradient moduli on both sides with the main direction as the dividing line, and calculate $m_r=\underset{k=1}{\overset{(N_{\mathrm{bin}}-2)/2}{\mathrm{\&Sigma;}}}{L}_{({\mathrm{no}}_d-k+N_{\mathrm{bin}})\%N_{\mathrm{bin}}}$ and $m_l=\underset{k=1}{\overset{(N_{\mathrm{bin}}-2)/2}{\mathrm{\&Sigma;}}}{L}_{({\mathrm{no}}_d+k+N_{\mathrm{bin}})\%N_{\mathrm{bin}}},$ where N _bin is the total number of directions, n _d is the index of the main direction and _Li represents the direction

The gradient modulus value on , % represents the modulo operation;

Step 7: If m _r >m _l , organize the pixel units in the Gaussian weight window from top to bottom and from right to left, and each gradient in each unit is coded clockwise from the gradient angle relative to the main direction 0 degrees, otherwise , the pixel units in the Gaussian weight window are organized from top to bottom and from right to left, each gradient in each unit is coded counterclockwise from the gradient angle relative to the main direction 0 degrees, and normalized to generate a pair The description data of the image.

The invention increases the robustness of the feature extraction and description method to the mirror imaging problem, and expands the application field of computer vision. All the feature extraction and description methods in the field of computer vision today do not consider and deal with the situation of mirror imaging, although methods such as Harris corner detector, SIFT, SURF, GLOH can be used under changes in rotation, scale, and illumination. Maintains a certain degree of stability, but is helpless in the case of mirror imaging. The present invention proposes a method aiming at this situation, and while successfully solving the mirror imaging situation, it has similar performance to SIFT for the non-mirror imaging situation. The feature extraction and description method proposed by the present invention is called MIFT here, and the comparison result with SIFT in the case of mirror imaging is shown in Figure 1. Under the same threshold as 0.60, MIFT matches 258 feature point pairs, and SIFT Match 12 feature point pairs. The renderings in Figure 2 compare the matching results of several complex images.

Description of drawings

Figure 1: Matching effect diagram in the case of mirror imaging. Figure 1(a) is the MIFT matching result and Figure 1(b) is the SIFT matching result.

Figure 2: Comparison of MIFT and SIFT matching results. (a) MIFT matching results in the case of non-specular imaging. (b) SIFT matching results in the case of non-specular imaging. (c) MIFT matching results in the case of mirror imaging. (d) SIFT matching results in the case of mirror imaging.

Figure 3 is a schematic representation of an image at three adjacent scales.

Figure 4: Analysis of feature descriptor organization structure before and after image flipping. Among them, (a) Representation of feature points and their auxiliary regions on the unflipped image. (b) Representation of the same feature point and its auxiliary region on the flipped image. (c) Representation of the 8 gradients in the 14th unit from (b). (d) Representation of the 8 gradients in the 14th unit from (a). (e) Feature descriptors for SIFT and MIFT in the case of (a). (f) Feature descriptors for SIFT in the case of (b). (g) Feature descriptors for MIFT in the case of (b).

Figure 5: Gradient information map of feature points. In the figure, n _d is the index subscript of the main direction, and % represents the modulo operation.

Figure 6: Flow chart of feature detection and feature description.

Figure 7: Feature matching flowchart.

Detailed ways

In the present invention, the scale change of the input image is performed firstly, and the change is completed through Gaussian convolution. In a series of images of different scales, the extreme point of the gray value of the pixel is searched for each pixel. However, not all extreme points meet the criteria as feature points. Since the feature points need to have certain prominence and robustness, by setting appropriate thresholds for Difference of Gaussian and Hessian matrix , so as to filter the candidate points with low contrast and edge response of feature points respectively. The extreme points left after these two steps of screening are the desired feature points. For these extreme points, after fitting them with a three-dimensional quadratic curve, their precise coordinates and scale information are obtained. The coordinates, scale and other information of the feature points are preserved to provide usable information for the subsequent matching stage.

The above part can be considered as a feature point detector, that is to say, this part is mainly to find the feature points of the image. Then, the next step is how to organize the information of feature points to become available feature descriptions to provide higher-level applications-feature matching. The feature point description part is divided into the statistical gradient information specifying the direction parameters and the construction of feature descriptors.

For the mirror imaging problem, images are divided into four categories, namely: original image, horizontally flipped image (mirror), vertically flipped image (inverted) and completely flipped image (there are both horizontal and vertical flips). However, it is easy to verify that a fully flipped image is equivalent to the image obtained by rotating the coordinate system of the original image by 180 degrees. Similarly, a vertically flipped image and a horizontally flipped image also have a similar relationship as between the original image and the completely flipped image. And because the operation of specifying the orientation parameter before the feature point description makes the feature description invariant to rotation, the four cases can be simplified into two, namely the original image and the horizontally flipped image. There is a fixed relationship between the original image and the horizontally flipped image, which can be divided into two levels. Each feature point in SIFT is affected by pixels in an area. This strategy is to reduce the influence of noise and improve the robustness of feature points. Then for the area corresponding to the same feature point, in the original image and the horizontally flipped image, only the order of the columns of the area pixel units is reversed, but the order of the rows remains unchanged, which is the higher level of the two levels. The other level is a relatively microscopic unit. In each small pixel unit, according to the corresponding relationship between the two images, the 8 gradient directions in each unit satisfy the following formula:

Among them, the subscript ori represents the original image, and the subscript hor represents the horizontally flipped image. The encoding strategy fully exploits the two-level relationship, resulting in a mirror-flip invariant feature descriptor.

The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

Step 1: Multi-scale spatial representation of the input image

Koendetink and Lindeberg proved that the Gaussian convolution kernel is the only invariant convolution kernel that realizes scale transformation linearly. The two-dimensional Gaussian function has the form: $G(x,\mathrm{the\; y},\mathrm{\&sigma;})=\frac{1}{2\mathrm{\&pi;}{\mathrm{\&sigma;}}^2}{e}^{-(x^2+{\mathrm{the\; y}}^2)/2{\mathrm{\&sigma;}}^2},$ where σ is the variance of a Gaussian normal distribution. Then a two-dimensional image at different scales can be obtained by convolving the image with a Gaussian kernel: L(x, y, σ)=G(x, y, σ)*I(x, y).

Step 2: Detect scale space extreme points

From the multi-scale image obtained in step 1, the extremum with pixel as the observation unit can be obtained by calculating and comparing from the entire scale space. In addition to comparing a certain number of pixels on adjacent scales, it is also necessary to compare adjacent pixels on the same two-dimensional image space to detect extreme points more comprehensively. The present invention utilizes DoG (Difference-of-Gaussian) operator to approximate LoG (Laplacian-of-Gaussian) to detect extremum points. Although the accuracy of DoG is slightly lower than that of LoG, the operation speed of the former is better than that of the latter. Among them, the definition form of the DoG operator is as follows:

D(x,y,σ)=(G(x,y,kσ)-G(x,y,σ))*I(x,y)=L(x,y,kσ)-L(x,y , σ).

Figure 3 shows the representation of an image at three adjacent scales. The black cross is the currently calculated pixel position, and the gray dots are all the pixels that need to be compared, a total of 2×9+8=26 points.

Step 3: Filter feature points from extreme points

The extreme points obtained by completing the above two steps constitute the candidate set of feature points, and the feature points will be selected from the candidate points in this set, that is to say, not all extreme points satisfy the requirement of feature points. Require. Because there are low-contrast and edge-response points in this set, their uniqueness and stability are not prominent enough as image features, so two different strategies are used to remove these two types of points. Firstly, while computing the Difference of Gaussian (DoG), an appropriate threshold is set to effectively eliminate low-contrast extreme points. Second, since the Difference of Gaussian (DoG) itself has an edge response, the Hessian matrix method is used to filter out points with edge response (saddle points).

Step 4: Accurate positioning of feature point positions

After the above operations, the feature points have been determined. Due to the transformation on the scale and the influence of the size of the pixel unit, the coordinates and scale information of the feature points may be somewhat deviated. Fitting and approximation of information by means of a method to obtain more accurate coordinates and image scale information.

Step 5: Determine Orientation Parameters

In order to meet the rotation invariance of feature points, according to the gradient direction and size information on the auxiliary neighborhood pixels of feature points, specify the direction parameters for each feature point. Among them, the size of the gradient m(x, y) and the direction angle θ(x, y) can be calculated based on the difference between pixels, and the calculation method is:

$\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\sqrt{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; L</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; L</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span>{<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; L</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; L</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>{<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; ,</span>}$

$\mathrm{<span\; class="notranslate">\; \&theta;</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; the\; tan</span>}}^{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; L</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; L</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>}{\mathrm{<span\; class="notranslate">\; L</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; L</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; .</span>$

Then, use the form of histogram for information statistics, specify the range of the histogram as 0-360 degrees, 10 degrees as a unit, and combine the Gaussian weighted windows with the feature point as the center, according to the entire neighborhood The magnitude and direction of the gradient of each pixel in the domain are counted. Among the 36 directions in the histogram, the main direction is the most intense. Due to the influence of noise or deformation, the image may change or distort slightly, and these changes or distortions may cause the deviation of the main direction parameters of the feature points. In order to alleviate Or to avoid the impact of these deviations on the direction parameters, the present invention uses the auxiliary direction as in SIFT. An auxiliary direction is defined as a direction whose strength is greater than 80% of the strength of the main direction, and there may be more than one auxiliary direction. In fact, each auxiliary direction is considered as equally important as the main direction in the process of generating feature descriptors and a separate descriptor is established.

Step 6: Feature point descriptor encoding

The feature descriptor used in the present invention is represented by a vector, and this vector contains gradient information of all pixels in the Gaussian weight window. The vector contains 4×4×8=128-dimensional information. This level of dimensionality was proved to be a successful one by Lowe. Similarly, MIFT also uses 128-dimensional vectors to represent descriptors, although the size of the dimensionality is not mandatory. changing. Taking Figure 4 as an example, the two levels are analyzed separately. where (a) represents a feature point and its auxiliary region in the unflipped image, and (b) is the representation of the same feature point and its auxiliary region in the flipped image, both of which have been specified direction parameter.

First, the feature descriptors are analyzed at a macro level - the sequence of 16 units. After specifying the main direction, SIFT uses a fixed order to organize the 16 units in column-first order (or row-first order), as shown in Figure 4, and the SIFT feature descriptor vector is shown in (e). In the image after flipping, that is, corresponding to (b), the row order of these 16 units is unchanged, but the column order is reversed, which leads to the organization of the feature descriptor vector of SIFT under (b) as ( f) as shown. It is undeniable that SIFT has good stability and robustness for rotation, scaling, lighting, affine transformation, etc., but it is also undeniable that SIFT is powerless for situations such as mirror imaging. Therefore, the present invention proposes an encoding method that can obtain a unique form of descriptor before and after image flipping. In two different cases, there are only two encodings in the case of column-major encoding, one is a right-to-left order and the other is a reverse order. Intuitively, the left and right pointing gradient modulus (dotted line) in the figure can be used as a basis for judging the adoption of one of the above two methods. However, only the gradient information pointing to the left and right is more sensitive to noise and other influences, so the present invention uses a strategy of summing all gradient moduli pointing to the same side instead. As shown in Figure 5, the abstract mathematical formula is:

${\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; bin</span>}}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 2</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; L</span>}}_{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; no</span>}}_{\mathrm{<span\; class="notranslate">\; d</span>}}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; bin</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \%</span>{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; bin</span>}}}<span\; class="notranslate">\; ,</span>$

${\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; bin</span>}}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 2</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; L</span>}}_{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; no</span>}}_{\mathrm{<span\; class="notranslate">\; d</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; bin</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \%</span>{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; bin</span>}}}<span\; class="notranslate">\; .</span>$

上的梯度模值，％代表取模操作。其中m_r和m_l分别是图5中右下方虚线箭头和左上方点划线箭头之和。据此，我们将编码策略由原来的固定顺序编码改变成为由m_r和m_l比较结果决定。理论上而言，在翻转前后通过本发明提出的方法可以得到同样的描述符如图4中(g)所示。与主辅方向原理相似，为了减少各种噪音和光照条件变化等影响，增强MIFT的健壮性，如果满足min{m_r，m_l}＞τmax{m_r，m_l}，(其中τ为阈值，这里我们将之设置为0.70，)那么另一个描述符则随之生成。Among them, N _bin is the total number of all directions, here N _bin =36, _nd is the index of the main direction and _Li represents the direction

The modulus value of the gradient above, % represents the modulo operation. Among them, m _r and m _l are respectively the sum of the dotted line arrow at the lower right and the dotted line arrow at the upper left in Fig. 5 . Accordingly, we change the encoding strategy from the original fixed sequence encoding to be determined by the comparison results of m _r and m _l . Theoretically speaking, the same descriptor can be obtained by the method proposed by the present invention before and after flipping, as shown in (g) in FIG. 4 . Similar to the principle of primary and secondary directions, in order to reduce the impact of various noises and changes in lighting conditions and enhance the robustness of MIFT, if min{m _r , m _l }>τmax{m _r , m _l }, (where τ is the threshold , here we set it to 0.70,) then another descriptor is generated accordingly.

The second level is more microcosmic, which is specific to the relationship of gradients in each direction in each unit. As shown in (c) and (d) in Figure 4, they respectively correspond to the same part of the image after flipping and before flipping, and the relationship between gradients in each direction between (c) and (d), see the above specific description for details . According to this special relationship, combined with the analysis of the first level, the final feature descriptor can be generated by the following strategy: calculate and compare m _r and m _l , and use the larger one as the direction of the coding order, combined with Figure 4, In the case shown in (a) m _r <m _l , then the encoding order of the 16 units is from top to bottom and from left to right, and the 8 gradient directions in each unit are as shown in (d), Starting from A, encode counterclockwise; if m _r > m _l in the case shown in (b), then the encoding order of the 16 units is from top to bottom, from right to left, and 8 gradients in each unit The direction is shown in (c), starting from A, coded clockwise, the feature descriptors obtained in these two cases are the same, that is to say, this feature description can solve the problem of mirror imaging. Of course, in order to increase the stability of feature descriptors to changes in lighting conditions, the final normalization operation is essential. The process of feature detection and feature description is shown in Figure 6.

After the processing of the input image, the detection of feature points, and the feature description, the part of feature matching is essential, and it also needs to be carefully designed to match as many correct feature point pairs as possible and reduce the purpose of wrong matching. . The matching method adopted by SIFT is to obtain a set of values by performing an inner product operation on the current feature point description vector to be matched in image 2 and all the feature point description vectors in image 1. These values are sorted from small to large, ranking first A group of feature points corresponding to the result with the smallest value is the object to be judged whether to match. If the ratio between the smallest value and the second smallest value is lower than a certain threshold, then the two feature points corresponding to the smallest value are considered to match, otherwise, there is no match. Compared with the general idea (that is, the method of setting the global threshold), this method has certain stability and rationality, because various deformations and noises of the image will affect this, so a fixed index cannot effectively and reasonably explain the matching question of whether or not.

Similarly, MIFT also adopts a matching strategy similar to SIFT, but it has been improved on top of it. This improvement is to reduce or even avoid the situation of mistakenly missing the point pairs that should be matched. The inner product result of two unit vectors reflects the cosine value of the angle between the two vectors, that is to say, the smaller the value, the closer the two vectors are, and the higher the matching degree. Since the same feature point may generate multiple feature descriptors due to the main-auxiliary direction and the similarity problem of m _r and m _l , this situation leads to an increase in the possibility of multiple feature descriptors with high similarity. In order to reduce the impact of this situation on the matching results, the present invention improves the matching method of SIFT, and checks the additional information of the feature descriptor, if the coordinates, scales, etc. The information is the same, then skip the set of feature points corresponding to the larger value, and compare it with the next value corresponding to the matching degree of a set of feature points until the additional information of the two sets of feature points is exhausted Similarly, if the ratio of the two values is lower than a certain threshold, then the set of feature points corresponding to the minimum value matches, otherwise, there is no match. The feature matching process is shown in Figure 7.

Claims (1)

1. a feature extraction and description method with mirror flip invariance based on SIFT, is characterized in that, comprises the following steps: Step 1: Perform Gaussian kernel convolution processing on the input image I(x, y), that is, L(x, y, σ)=G(x, y, σ)*I(x, y), to obtain a multi-scale space Expressed image L(x, y, σ), where, $G(x,\mathrm{the\; y},\mathrm{\&sigma;})=\frac{1}{2\mathrm{\&pi;}{\mathrm{\&sigma;}}^2}{e}^{-(x^2+{\mathrm{the\; y}}^2)/2{\mathrm{\&sigma;}}^2},$ Where σ is the variance of the Gaussian normal distribution; Step 2: Perform Gaussian difference processing on the image L(x, y, σ) expressed in multi-scale space according to the following formula D(x, y, σ)=(G(x, y, kσ)-G(x, y, σ))*I(x, y)=L(x, y, kσ)-L(x, y, σ), detecting the extreme points of the image L(x, y, σ) expressed in multi-scale space; Step 3: Use threshold method and Hessian matrix method to screen feature points; Step 4: Accurately locate the position of the feature points by fitting the three-dimensional quadratic curve; Step 5: Determine the direction parameter of each feature point according to the gradient direction θ(x, y) and size m(x, y) information on the auxiliary neighborhood pixels of the feature point, where: $\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\sqrt{{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; L</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; L</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; L</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; L</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; ,</span>$ $\mathrm{\&theta;}(x,\mathrm{the\; y})={\mathrm{the\; tan}}^{-1}\left(\frac{L(x,\mathrm{the\; y}+1)-L(x,\mathrm{the\; y}-1)}{L(x+1,\mathrm{the\; y})-(L(x-1),\mathrm{the\; y})}\right),$ And in the form of a histogram, combined with the Gaussian weight window centered on the feature point, the main direction is determined according to the gradient size and direction of each pixel in the entire neighborhood; Step 6: Sum the gradient moduli on both sides with the main direction as the dividing line, and calculate $m_r=\underset{k=1}{\overset{(N_{\mathrm{bin}}-2)/2}{\mathrm{\&Sigma;}}}{L}_{({\mathrm{no}}_d-k+N_{\mathrm{bin}})\%N_{\mathrm{bin}}}$ and $m_l=\underset{k=1}{\overset{(N_{\mathrm{bin}}-2)/2}{\mathrm{\&Sigma;}}}{L}_{({\mathrm{no}}_d+k+N_{\mathrm{bin}})\%N_{\mathrm{bin}}},$ where N _bin is the total number of directions, n _d is the index of the main direction and _Li represents the direction

The gradient modulus value on , % represents the modulo operation; Step 7: If m _r >m _l , organize the pixel units in the Gaussian weight window from top to bottom and from right to left, and each gradient in each unit is coded clockwise from the gradient angle relative to the main direction 0 degrees, otherwise , the pixel units in the Gaussian weight window are organized from top to bottom and from right to left, each gradient in each unit is coded counterclockwise from the gradient angle relative to the main direction 0 degrees, and normalized to generate a pair The description data of the image.

Images