CN116824183B - Image feature matching method and device based on multiple feature descriptors - Google Patents

Abstract

The invention provides an image feature matching method and device based on multiple feature descriptors, relates to the field of image processing and image feature matching, and aims to construct multiple feature descriptors and finish accurate matching of images by detecting feature points on the images. The multiple feature descriptors in the invention are constructed by using different permutation and combination methods of symbol, mean value and central value descriptors, and the direction information, numerical value information and global information of feature points are considered. And scanning the pixel matrix in the range cut around the feature points by adopting a sliding window, extracting three feature descriptors in each window, combining and splicing to generate a corresponding matrix numerical distribution histogram and the feature descriptors. And carrying out feature point matching according to feature descriptors of different images, and selecting partial feature points with the closest descriptors as optimal points to carry out image matching, so that the image matching effect is more accurate. The method provides technical support for image matching used in mine mining environment, scene modeling and industrial production.

Description

Image feature matching method and device based on multiple feature descriptors

Technical field

The invention relates to the fields of image processing and image feature matching, in particular to an image feature matching method and image feature matching device based on multiple feature descriptors.

Background technique

Feature descriptor construction and image matching are key technologies in the field of computer vision for identifying, matching, and tracking specific features in images. These technologies have wide applications in many fields, including computer graphics, robotics, autonomous driving, virtual reality, etc.

However, due to the various posture changes, lighting changes and noise interference of the camera during the shooting process, it is easy to reduce the quality of the generated feature descriptors, resulting in a decrease in matching accuracy. At the same time, in some embedded devices or mobile devices, hardware resources are limited and large-scale feature extraction and matching may not be possible. As a result, the computational efficiency and application scenarios of image matching are greatly limited.

Currently, in view of the above-mentioned problems in the construction of feature descriptors and image matching, there is no method that fully considers feature point direction information, numerical information and global information in the existing technology. Some methods use the gradient histogram around the statistical feature points of the image to generate feature descriptors. This type of method is easy to miss the global information of the image, resulting in mismatching of some feature points with the same gradient but different numerical differences, which leads to the failure of this type of algorithm. There are unstable factors in the application process, which can easily cause large image matching errors.

There are also some methods that directly perform brute force retrieval and matching on some areas around feature points to solve the complex problem of generating feature descriptors. However, these methods have higher requirements on hardware conditions and are ineffective when facing large-scale feature extraction and matching. It is poor and cannot meet the actual needs in large-scale operation scenarios such as cities or industries.

Contents of the invention

In view of the above problems, the present invention proposes an image feature matching method and image feature matching device based on multiple feature descriptors.

Embodiments of the present invention provide an image feature matching method based on multiple feature descriptors. The image feature matching method includes:

Detect feature points in each image based on feature point detection algorithm;

According to the distribution of the feature points and actual needs, set the first threshold, the second threshold and the third threshold, where the first threshold is used to set the size of the pixel matrix around the feature points to be intercepted, and the second threshold Used to set the sliding window radius, the third threshold is used to set the bit width of the feature descriptor;

Use the first threshold and the second threshold to scan and calculate the feature points to obtain symbol descriptors, mean descriptors and central value descriptors;

Based on the symbol descriptor, the mean descriptor and the central value descriptor, combined with the third threshold, a feature descriptor is obtained;

Feature point matching is performed on the feature descriptors of two or more images, and based on the comparison results, the feature point with the best matching result is selected as the final image matching result.

Optionally, the feature point detection algorithm is only used to detect feature points in each image, and the feature point detection algorithm includes: FAST, SIFT, SURF and SuperPoint algorithms.

Optionally, use the first threshold and the second threshold to scan and calculate the feature points to obtain symbol descriptors, mean descriptors and central value descriptors, including:

Using a sliding window, the pixel matrix intercepted around the feature point is scanned and calculated using the sliding window radius to obtain the descriptor, the mean descriptor and the central value descriptor.

Optionally, the first threshold is the patch_size threshold;

The second threshold is a radius threshold;

The third threshold is the bit_width threshold.

Optionally, the respective threshold sizes of the first threshold, the second threshold and the third threshold are calculated based on the distribution of the feature points and actual requirements or are obtained by network self-training.

Optionally, based on the symbol descriptor, the mean descriptor and the central value descriptor, combined with the third threshold, a feature descriptor is obtained, including:

The symbol descriptor, the mean descriptor and the center value descriptor are spliced and combined in different ways to directly generate a feature descriptor with the third threshold setting bit width; or,

The symbol descriptor, the mean descriptor and the center value descriptor are spliced and combined in different ways to generate a corresponding matrix numerical distribution histogram, and the third threshold is generated according to the matrix numerical distribution histogram. Feature descriptor that sets the bit width.

Optionally, the third threshold setting bit width feature descriptor remains consistent under different rotations, scales, flips and affine transformations; the symbol descriptor, the mean descriptor and the central value descriptor The calculation methods of each character include:

Calculate the absolute value of each surrounding point pixel in each sliding window except the center point pixel compared to the absolute value of the center point pixel. If the absolute value of the surrounding point pixel is greater than the absolute value of the center point pixel, set it to 1 , if it is less than 0, set it to 0, and arrange the results in order to generate the symbolic descriptor;

Calculate the average of all pixels in each sliding window and compare it with the average of pixels in the sliding window where the feature point is located. If the average of all pixels is greater than the average of pixels in the sliding window where the feature point is located Set 1 if it is less than 0, and arrange the results in order to generate the mean descriptor;

Calculate the size of the center point value of each sliding window compared to the average value of the pixel matrix around the feature point and the average value of the full-image pixel matrix. If the center point value of each sliding window is greater than the average value of the pixel matrix around the feature point, value and the average value of the full-image pixel matrix are set to 1, otherwise set to 0, and the results are arranged in order to generate the central value descriptor.

Optionally, the symbol descriptor, the mean descriptor and the central value descriptor are spliced and combined in different ways, including:

Perform sequential splicing to generate the feature descriptor in the order of the central value description, the symbol descriptor and the mean descriptor; or,

After adding the symbol descriptor and the mean description bitwise, the central value descriptor is added to the high bit to generate the feature descriptor.

Optionally, perform feature point matching on the feature descriptors of two or more images, including:

Use L1 norm matching or L2 norm matching to match the feature points; or,

The feature point matching is performed by calculating the Hamming distance between the feature descriptor of the first image and the feature descriptor of the second image; or,

Calculate the two adjacent bits from right to left of the feature descriptor of the first image and the feature descriptor of the second image. If it is not all 0, record it as a 1, and count the number of digits of the new 1. The feature points match.

An embodiment of the present invention provides an image feature matching device based on multiple feature descriptors. The image feature matching device includes:

The detection module 410 is used to detect feature points in each image based on the feature point detection algorithm;

The threshold setting module 420 is used to set the first threshold, the second threshold and the third threshold according to the distribution of the feature points and actual needs, wherein the first threshold is used to set the pixel matrix around the feature points to be intercepted. Size, the second threshold is used to set the sliding window radius, and the third threshold is used to set the bit width of the feature descriptor;

A scanning module 430 is configured to scan and calculate the feature points using the first threshold and the second threshold to obtain a symbol descriptor, a mean descriptor and a central value descriptor;

The feature descriptor module 440 is configured to obtain a feature descriptor based on the symbol descriptor, the mean descriptor and the central value descriptor, combined with the third threshold;

The matching selection module 450 is used to perform feature point matching on the feature descriptors of two or more images, and select the feature point with the best matching result as the final image matching result according to the comparison result.

Optionally, the scanning module is specifically used for:

Using a sliding window, the pixel matrix intercepted around the feature point is scanned and calculated using the sliding window radius to obtain the descriptor, the mean descriptor and the central value descriptor.

Optionally, the respective threshold sizes of the first threshold, the second threshold and the third threshold in the threshold setting module are calculated based on the distribution of the feature points and actual requirements or are obtained by network self-training;

Among them, the first threshold is the patch_size threshold;

The second threshold is a radius threshold;

The third threshold is the bit_width threshold.

Optionally, the feature descriptor module is specifically used to:

The symbol descriptor, the mean descriptor and the center value descriptor are spliced and combined in different ways to directly generate a feature descriptor with the third threshold setting bit width; or,

The symbol descriptor, the mean descriptor and the center value descriptor are spliced and combined in different ways to generate a corresponding matrix numerical distribution histogram, and the third threshold is generated according to the matrix numerical distribution histogram. Set the bit width of the feature descriptor;

Wherein, the symbol descriptor, the mean descriptor and the central value descriptor are spliced and combined in different ways, including:

Perform sequential splicing to generate the feature descriptor in the order of the central value description, the symbol descriptor and the mean descriptor; or,

After adding the symbol descriptor and the mean description bitwise, the central value descriptor is added to the high bit to generate the feature descriptor.

Optionally, the feature descriptors of the third threshold setting bit width remain consistent under different rotations, scales, flips and affine transformations; the symbol descriptors, the mean descriptors in the scanning module and The respective calculation methods of the central value descriptors include:

Calculate the absolute value of each surrounding point pixel in each sliding window except the center point pixel compared to the absolute value of the center point pixel. If the absolute value of the surrounding point pixel is greater than the absolute value of the center point pixel, set it to 1 , if it is less than 0, set it to 0, and arrange the results in order to generate the symbolic descriptor;

Calculate the average of all pixels in each sliding window and compare it with the average of pixels in the sliding window where the feature point is located. If the average of all pixels is greater than the average of pixels in the sliding window where the feature point is located Set 1 if it is less than 0, and arrange the results in order to generate the mean descriptor;

Calculate the size of the center point value of each sliding window compared to the average value of the pixel matrix around the feature point and the average value of the full-image pixel matrix. If the center point value of each sliding window is greater than the average value of the pixel matrix around the feature point, value and the average value of the full-image pixel matrix are set to 1, otherwise set to 0, and the results are arranged in order to generate the central value descriptor.

Optionally, the matching selection module is specifically used to:

Use L1 norm matching or L2 norm matching to match the feature points; or,

The feature point matching is performed by calculating the Hamming distance between the feature descriptor of the first image and the feature descriptor of the second image; or,

Calculate the two adjacent bits from right to left of the feature descriptor of the first image and the feature descriptor of the second image. If it is not all 0, record it as a 1, and count the number of digits of the new 1. The feature points match.

The image feature matching method based on multiple feature descriptors provided by the present invention first detects the feature points in each image based on the feature point detection algorithm; and then sets the pixels around the feature points that need to be intercepted according to the distribution of the feature points and actual needs. There are three thresholds: the size of the matrix, setting the radius of the sliding window, and setting the bit width of the feature descriptor.

Then the first two thresholds are used to scan and calculate the feature points to obtain the symbolic descriptor, the mean descriptor and the central value descriptor; then based on the symbolic descriptor, the mean descriptor and the central value descriptor, combined with the third threshold, we get Feature descriptors; finally, feature point matching is performed on the feature descriptors of two or more images, and based on the comparison results, the feature point with the best matching result is selected as the final image matching result.

The multiple feature descriptor construction method in the present invention uses different permutations and synthesis methods of symbolic descriptors, mean descriptors and central value descriptors as multiple feature descriptors, which fully takes into account the direction information, numerical information and global information of the feature points. , which can make image matching based on this feature more accurate and effective. The global information of the image will not be missed, and it will naturally not lead to mismatching of some feature points with the same gradient but different numerical differences, making the image matching more accurate. At the same time, the requirements for hardware conditions are low, and the effect is better when faced with large-scale feature extraction and matching. It well meets the actual needs in large-scale operation scenarios such as cities or industries, especially for mining environments, scene modeling, and industrial production. The image matching used in provides good technical support.

Description of the drawings

Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are for the purpose of illustrating preferred embodiments only and are not to be construed as limiting the invention. Also throughout the drawings, the same reference characters are used to designate the same components. In the attached picture:

Figure 1 is a flow chart of an image feature matching method based on multiple feature descriptors according to an embodiment of the present invention;

Figure 2 is a schematic diagram of a method for constructing multiple feature descriptors exemplified in the embodiment of the present invention;

Figure 3 is a schematic diagram of a multiple feature descriptor combination and splicing method exemplified in the embodiment of the present invention;

Figure 4 is a block diagram of an image feature matching device based on multiple feature descriptors according to an embodiment of the present invention.

Detailed ways

In order to make the above objects, features and advantages of the present invention more obvious and understandable, the present invention will be described in further detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention. They are only some embodiments of the present invention, not all embodiments, and are not used to limit the present invention.

Referring to Figure 1, a flow chart of an image feature matching method based on multiple feature descriptors according to an embodiment of the present invention is shown. The image feature matching method includes:

Step 101: Detect feature points in each image based on the feature point detection algorithm.

First, the feature points in each image are detected based on the feature point detection algorithm. If there are multiple images, naturally each image needs to detect feature points. The following steps 102 to 104 are explained and illustrated by taking the method that needs to be executed after detecting feature points in any image as an example.

In a preferred embodiment, the feature point detection algorithm proposed by the present invention is only used to detect feature points in each image, and no other operations are performed. The feature point detection algorithms include: FAST, SIFT, SURF, SuperPoint and other algorithms.

Step 102: Set the first threshold, the second threshold and the third threshold according to the distribution of feature points and actual needs, where the first threshold is used to set the size of the pixel matrix around the feature points to be intercepted, and the second threshold is used to set The sliding window radius is determined, and the third threshold is used to set the bit width of the feature descriptor.

For any image, after obtaining its feature points, three thresholds are set respectively according to the distribution of feature points and actual needs, that is, the first threshold, the second threshold and the third threshold, where the first threshold is used to set Determine the size of the pixel matrix around the feature points to be intercepted, the second threshold is used to set the sliding window radius, and the third threshold is used to set the bit width of the feature descriptor. By setting these three thresholds, we lay a solid foundation for subsequently applying the direction information, numerical information and global information of feature points to the construction of feature descriptors and image matching.

In a possible embodiment, the respective threshold sizes of the first threshold, the second threshold and the third threshold can be directly set manually, or can be calculated based on the distribution of feature points and actual requirements or obtained by network self-training.

In a preferred embodiment, the first threshold can be set as the patch_size threshold; the second threshold can be set as the radius threshold; and the third threshold can be set as the bit width threshold.

Step 103: Use the first threshold and the second threshold to scan and calculate the feature points to obtain symbol descriptors, mean descriptors and central value descriptors.

After the size of the pixel matrix around the feature points to be intercepted, the radius of the sliding window, and the bit width of the feature descriptor are all set, the first threshold and the second threshold can be used, that is, using the pixel matrix around the feature points to be intercepted. The size and sliding window radius are scanned and calculated to obtain the symbol descriptor, mean descriptor and central value descriptor.

A better way is to use a sliding window to scan and calculate the pixel matrix intercepted around the feature point using the sliding window radius to obtain the descriptor, mean descriptor and central value descriptor.

Taking the first threshold as the patch_size threshold, the second threshold as the radius threshold, and the third threshold as the bit_width threshold as an example, assume that the patch_size threshold is set to 2, the radius threshold is 1, and the bit_width threshold is 18. Referring to the schematic diagram of the construction method of multiple feature descriptors shown in Figure 2, the parameter patch_size threshold is set to 2, which means that a 5×5 area around the feature point is extracted. In this area, the parameter radius threshold is set according to the sliding window size. is 3×3.

Feature descriptors with bit widths set based on the bit_width threshold remain consistent under different rotations, scales, flips and affine transformations. On this basis, the calculation methods of symbolic descriptors, mean descriptors and central value descriptors include:

Calculate the absolute value of each surrounding point pixel in each sliding window except the center point pixel compared to the absolute value of the center point pixel. If the absolute value of the surrounding point pixel is greater than the absolute value of the center point pixel, it is set to 1, if it is less, it is set to 0. And arrange the results in order to generate symbolic descriptors.

Calculate the average of all pixels in each sliding window and compare it with the average of pixels in the sliding window where the feature point is located. If the average of all pixels is greater than the average of pixels in the sliding window where the feature point is located, it is set to 1, if it is less than Set to 0 and arrange the results in order to generate a mean descriptor.

Calculate the size of the center point value of each sliding window compared to the average value of the pixel matrix around the feature point and the average pixel matrix of the entire image. If the center point value of each sliding window is greater than the average value of the pixel matrix around the feature point and the entire image The average value of the pixel matrix is set to 1, if it is less than 0, the results are arranged in order to generate a central value descriptor.

Combined with Figure 2, when calculating the symbolic descriptor, as shown in the top row in Figure 2, there are a total of 9 sliding windows from the upper left to the lower right. Calculate the pixels in each sliding window except the center point (for example, the leftmost pixel in the top row The absolute value of each surrounding point pixel outside the eight small white boxes surrounded by shadows in the picture is compared to the absolute value of the center point pixel. If the absolute value of the surrounding point pixel is greater than the absolute value of the center point pixel, it is set to 1, and if it is less, it is set to 0. , then S1, S2,..., S8, S9 are obtained respectively, and the results are arranged in order to obtain S, thereby generating a symbolic descriptor. The rightmost one in Figure 2 is exemplarily represented by 9*xxxxxxxx.

When calculating the mean descriptor, as shown in the middle row of Figure 2, there are 9 sliding windows from the upper left to the lower right. The average of all pixels in each sliding window is calculated, and the sliding window where the feature point is located (Figure 2 Compare the average values of pixels within the shadow) in the leftmost figure of the middle row. If the average value of all pixels is greater than the average value of the pixels in the sliding window where the feature point is located, it is set to 1, and if it is less than the average value of the pixels in the sliding window, it is set to 0. M1, M2,..., M8, M9, and arrange the results in sequence to obtain M, thereby generating a mean descriptor. The rightmost one in Figure 2 is exemplarily represented by 9*xxxxxxxx.

When calculating the center value descriptor, as shown in the bottom row in Figure 2, there are 9 sliding windows from the upper left to the lower right. Calculate the value of the center point of each sliding window (such as the shadow in the bottom row and leftmost figure). Based on the average size of the intercepted pixel matrix around the feature point and the average value of the whole image, if the center point value of each sliding window is greater than the average value of the pixel matrix around the feature point and the average value of the whole image pixel matrix, it is set to 1, and if it is smaller than the average value of the pixel matrix of the whole image, it is set to 1. , respectively obtain C1, C2,..., C8, C9, and arrange the results in order to obtain C, and arrange the results in order to generate a central value descriptor. The rightmost one in Figure 2 is exemplarily represented by 9*xx.

Through the above method, the symbol descriptor, mean descriptor and central value descriptor can be obtained.

Step 104: Based on the symbol descriptor, mean descriptor and central value descriptor, combined with the third threshold, obtain a feature descriptor.

After obtaining the symbol descriptor, mean descriptor, and central value descriptor, the feature descriptor is obtained based on the symbol descriptor, the mean descriptor, and the central value descriptor, combined with the third threshold, and the feature descriptor is obtained. specific:

The symbol descriptor, mean descriptor and central value descriptor can be spliced and combined in different ways to directly generate a feature descriptor with a third threshold setting bit width; or, the symbol descriptor, mean descriptor and central value descriptor can be combined The symbols are spliced and combined in different ways to generate a corresponding matrix value distribution histogram, and then a feature descriptor with a third threshold setting bit width is generated based on the matrix value distribution histogram.

In a possible embodiment, a method of splicing and combining symbol descriptors, mean descriptors and central value descriptors in different ways includes:

According to the order of the central value description, symbolic descriptor and mean descriptor, perform sequential splicing to generate a feature descriptor; or, after adding the symbolic descriptor and the mean description bit by bit, add the central value descriptor in the high position to generate a feature description symbol.

Referring to the schematic diagram of the multiple feature descriptor combination splicing method shown in Figure 3, the three are sequentially spliced according to the central value description C, the symbolic descriptor S and the mean descriptor M, and the feature descriptor is generated as an example. Assume that the central value description C is 10 (2bit), the symbolic descriptor S is 10110100 (8bit), and the mean descriptor M is 01110011 (8bit), then the corresponding generated multiple feature descriptor CSM is 10101101000111011, a total of 18bits.

Of course, the order of each descriptor can also be adjusted according to actual needs, or the symbolic descriptor and the mean description can be added bitwise, and the central value descriptor can be added to the high position to generate a feature descriptor, etc. Figure 3 also exemplarily shows the sequential splicing according to the central value description C, the mean descriptor M and the symbolic descriptor S, and the multiple feature descriptor CMS is generated as 100111001110110100; according to the symbolic descriptor S, the central value description C, the mean The order of the descriptors M is sequentially spliced, and the multiple feature descriptor SCM is generated as 10110100100111011.

Step 105: Perform feature point matching on the feature descriptors of two or more images, and select the feature point with the best matching result as the final image matching result based on the comparison result.

After the feature descriptors of each image are obtained according to steps 102 to 104, the feature descriptors of two or more images can be matched with feature points, and then based on the comparison results, the feature points with the best matching results are selected as the final Image matching results.

In a possible embodiment, a method for feature point matching of feature descriptors of two or more images includes:

Feature point matching is performed using L1 norm matching or L2 norm matching; or, feature point matching is performed by calculating the Hamming distance between the feature descriptor of the first image and the feature descriptor of the second image. ; Or, calculate the two adjacent bits from right to left of the feature descriptor of the first image and the feature descriptor of the second image. If not all 0, record it as a 1, and count the number of digits of the new 1. method to perform feature point matching.

In actual matching, the matching method can be selected according to different actual needs, in which the Hamming distance of two feature descriptors is calculated, that is, the sum of the number of digits of 1 in all elements is calculated.

For the matching comparison results, for example: compare each feature descriptor of the first image with any feature description of the second image, and select the pair of feature descriptors with the smallest Hamming distance as the pair. The optimal feature descriptor point pairs of points are selected as the optimal feature point matching pairs for image splicing among all feature descriptor point pairs with the smallest Hamming distance between point pairs.

Based on the above image feature matching method based on multiple feature descriptors, an embodiment of the present invention also proposes an image feature matching device based on multiple feature descriptors. Referring to the device block diagram shown in Figure 4, the image feature matching device includes:

The detection module 410 is used to detect feature points in each image based on the feature point detection algorithm;

The threshold setting module 420 is used to set the first threshold, the second threshold and the third threshold according to the distribution of the feature points and actual needs, wherein the first threshold is used to set the pixel matrix around the feature points to be intercepted. Size, the second threshold is used to set the sliding window radius, and the third threshold is used to set the bit width of the feature descriptor;

A scanning module 430 is configured to scan and calculate the feature points using the first threshold and the second threshold to obtain a symbol descriptor, a mean descriptor and a central value descriptor;

The feature descriptor module 440 is configured to obtain a feature descriptor based on the symbol descriptor, the mean descriptor and the central value descriptor, combined with the third threshold;

The matching selection module 450 is used to perform feature point matching on the feature descriptors of two or more images, and select the feature point with the best matching result as the final image matching result according to the comparison result.

Optionally, the scanning module 430 is specifically used to:

Using a sliding window, the pixel matrix intercepted around the feature point is scanned and calculated using the sliding window radius to obtain the descriptor, the mean descriptor and the central value descriptor.

Optionally, the respective threshold sizes of the first threshold, the second threshold and the third threshold in the threshold setting module 420 are calculated based on the distribution of the feature points and actual requirements or are obtained by network self-training. ;

Among them, the first threshold is the patch_size threshold;

The second threshold is a radius threshold;

The third threshold is the bit_width threshold.

Optionally, the feature descriptor module 440 is specifically used to:

The symbol descriptor, the mean descriptor and the center value descriptor are spliced and combined in different ways to directly generate a feature descriptor with the third threshold setting bit width; or,

The symbol descriptor, the mean descriptor and the center value descriptor are spliced and combined in different ways to generate a corresponding matrix numerical distribution histogram, and the third threshold is generated according to the matrix numerical distribution histogram. Set the bit width of the feature descriptor;

Wherein, the symbol descriptor, the mean descriptor and the central value descriptor are spliced and combined in different ways, including:

Perform sequential splicing to generate the feature descriptor in the order of the central value description, the symbol descriptor and the mean descriptor; or,

After adding the symbol descriptor and the mean description bitwise, the central value descriptor is added to the high bit to generate the feature descriptor.

Optionally, the feature descriptors of the third threshold setting bit width remain consistent under different rotations, scales, flips and affine transformations; the symbol descriptors, the mean descriptors in the scanning module 430 And the respective calculation methods of the central value descriptors include:

Calculate the absolute value of each surrounding point pixel in each sliding window except the center point pixel compared to the absolute value of the center point pixel. If the absolute value of the surrounding point pixel is greater than the absolute value of the center point pixel, set it to 1 , if it is less than 0, set it to 0, and arrange the results in order to generate the symbolic descriptor;

Calculate the average of all pixels in each sliding window and compare it with the average of pixels in the sliding window where the feature point is located. If the average of all pixels is greater than the average of pixels in the sliding window where the feature point is located Set 1 if it is less than 0, and arrange the results in order to generate the mean descriptor;

Calculate the size of the center point value of each sliding window compared to the average value of the pixel matrix around the feature point and the average value of the full-image pixel matrix. If the center point value of each sliding window is greater than the average value of the pixel matrix around the feature point, value and the average value of the full-image pixel matrix are set to 1, otherwise set to 0, and the results are arranged in order to generate the central value descriptor.

Optionally, the matching selection module 450 is specifically used to:

Use L1 norm matching or L2 norm matching to match the feature points; or,

The feature point matching is performed by calculating the Hamming distance between the feature descriptor of the first image and the feature descriptor of the second image; or,

Calculate the two adjacent bits from right to left of the feature descriptor of the first image and the feature descriptor of the second image. If it is not all 0, record it as a 1, and count the number of digits of the new 1. The feature points match.

To sum up, the image feature matching method based on multiple feature descriptors provided by the present invention first detects the feature points in each image based on the feature point detection algorithm; and then sets the interception requirements based on the distribution of the feature points and actual needs. The three thresholds are the size of the pixel matrix around the feature point, setting the radius of the sliding window, and setting the bit width of the feature descriptor.

Then the first two thresholds are used to scan and calculate the feature points to obtain the symbolic descriptor, the mean descriptor and the central value descriptor; then based on the symbolic descriptor, the mean descriptor and the central value descriptor, combined with the third threshold, we get Feature descriptors; finally, feature point matching is performed on the feature descriptors of two or more images, and based on the comparison results, the feature point with the best matching result is selected as the final image matching result.

The multiple feature descriptor construction method in the present invention uses different permutations and synthesis methods of symbolic descriptors, mean descriptors and central value descriptors as multiple feature descriptors, which fully takes into account the direction information, numerical information and global information of the feature points. , which can make image matching based on this feature more accurate and effective. The global information of the image will not be missed, and it will naturally not lead to mismatching of some feature points with the same gradient but different numerical differences, making the image matching more accurate. At the same time, the requirements for hardware conditions are low, and the effect is better when faced with large-scale feature extraction and matching, which well meets the actual needs in large-scale operation scenarios such as cities or industries, especially for mining environments, scene modeling, and industrial production. The image matching used in provides good technical support.

Although preferred embodiments of the embodiments of the present invention have been described, those skilled in the art will be able to make additional changes and modifications to these embodiments once the basic inventive concepts are apparent. Therefore, it is intended that the appended claims be construed to include the preferred embodiments and all changes and modifications that fall within the scope of embodiments of the invention.

Finally, it should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that these entities or any such actual relationship or sequence between operations. Furthermore, the terms "comprises," "comprises," or any other variation thereof are intended to cover a non-exclusive inclusion such that a process, method, article, or end device that includes a list of elements includes not only those elements, but also elements not expressly listed or other elements inherent to such process, method, article or terminal equipment. Without further limitation, an element defined by the statement "comprises a..." does not exclude the presence of additional identical elements in a process, method, article or terminal device including the stated element.

The embodiments of the present invention have been described above in conjunction with the accompanying drawings. However, the present invention is not limited to the above-mentioned specific implementations. The above-mentioned specific implementations are only illustrative and not restrictive. Those of ordinary skill in the art will Under the inspiration of the present invention, many forms can be made without departing from the spirit of the present invention and the scope protected by the claims, and these all fall within the protection of the present invention.

Claims (9)

1. An image feature matching method based on multiple feature descriptors, characterized in that the image feature matching method includes: Detect feature points in each image based on feature point detection algorithm; According to the distribution of the feature points and actual needs, set the first threshold, the second threshold and the third threshold, where the first threshold is used to set the size of the pixel matrix around the feature points to be intercepted, and the second threshold Used to set the sliding window radius, the third threshold is used to set the bit width of the feature descriptor; Use the first threshold and the second threshold to scan and calculate the feature points to obtain symbol descriptors, mean descriptors and central value descriptors; The symbol descriptor, the mean descriptor and the center value descriptor are spliced and combined in different ways to directly generate a feature descriptor with the third threshold setting bit width; or, The symbol descriptor, the mean descriptor and the center value descriptor are spliced and combined in different ways to generate a corresponding matrix numerical distribution histogram, and the third threshold is generated according to the matrix numerical distribution histogram. Set the bit width of the feature descriptor; Perform feature point matching on the feature descriptors of two or more images, and select the feature point with the best matching result as the final image matching result based on the comparison result; Wherein, the respective calculation methods of the symbol descriptor, the mean descriptor and the central value descriptor include: Calculate the absolute value of each surrounding point pixel in each sliding window except the center point pixel compared to the absolute value of the center point pixel. If the absolute value of the surrounding point pixel is greater than the absolute value of the center point pixel, set it to 1 , if it is less than 0, set it to 0, and arrange the results in order to generate the symbolic descriptor; Calculate the average of all pixels in each sliding window and compare it with the average of pixels in the sliding window where the feature point is located. If the average of all pixels is greater than the average of pixels in the sliding window where the feature point is located Set 1 if it is less than 0, and arrange the results in order to generate the mean descriptor; Calculate the size of the center point value of each sliding window compared to the average value of the pixel matrix around the feature point and the average value of the full-image pixel matrix. If the center point value of each sliding window is greater than the average value of the pixel matrix around the feature point, value and the average value of the full-image pixel matrix are set to 1, otherwise set to 0, and the results are arranged in order to generate the central value descriptor. 2. The image feature matching method according to claim 1, characterized in that the feature point detection algorithm is only used to detect feature points in each image, and the feature point detection algorithm includes: FAST, SIFT, SURF and SuperPoint algorithm. 3. The image feature matching method according to claim 1, characterized in that the first threshold and the second threshold are used to scan and calculate the feature points to obtain symbol descriptors, mean descriptors and center Value descriptors, including: Using a sliding window, the pixel matrix intercepted around the feature point is scanned and calculated using the sliding window radius to obtain the symbol descriptor, the mean descriptor and the central value descriptor. 4. The image feature matching method according to claim 1, wherein the first threshold is a patch_size threshold; The second threshold is a radius threshold; The third threshold is the bit_width threshold. 5. The image feature matching method according to claim 1, characterized in that the respective threshold sizes of the first threshold, the second threshold and the third threshold are determined by the distribution of the feature points and actual requirements. Obtained by calculation or network self-training. 6. The image feature matching method according to claim 1, characterized in that the feature descriptor of the third threshold setting bit width remains consistent under different rotations, scales, flips and affine transformations. 7. The image feature matching method according to claim 1, characterized in that the symbol descriptor, the mean descriptor and the central value descriptor are spliced and combined in different ways, including: Perform sequential splicing to generate the feature descriptor in the order of the central value description, the symbol descriptor and the mean descriptor; or, After adding the symbol descriptor and the mean description bitwise, the central value descriptor is added to the high bit to generate the feature descriptor. 8. The image feature matching method according to claim 1, characterized in that feature point matching of feature descriptors of two or more images includes: Use L1 norm matching or L2 norm matching to match the feature points; or, The feature point matching is performed by calculating the Hamming distance between the feature descriptor of the first image and the feature descriptor of the second image; or, Calculate the two adjacent bits from right to left of the feature descriptor of the first image and the feature descriptor of the second image. If it is not all 0, record it as a 1, and count the number of digits of the new 1. The feature points match. 9. An image feature matching device based on multiple feature descriptors, characterized in that the image feature matching device includes: The detection module is used to detect feature points in each image based on the feature point detection algorithm; Setting a threshold module, configured to set a first threshold, a second threshold and a third threshold according to the distribution of the feature points and actual needs, wherein the first threshold is used to set the size of the pixel matrix around the feature points to be intercepted , the second threshold is used to set the sliding window radius, and the third threshold is used to set the bit width of the feature descriptor; A scanning module, configured to scan and calculate the feature points using the first threshold and the second threshold to obtain a symbol descriptor, a mean descriptor and a central value descriptor; A feature descriptor module, configured to splice and combine the symbol descriptor, the mean descriptor and the central value descriptor in different ways to directly generate a feature descriptor with the third threshold setting bit width; or , The symbol descriptor, the mean descriptor and the center value descriptor are spliced and combined in different ways to generate a corresponding matrix numerical distribution histogram, and the third threshold is generated according to the matrix numerical distribution histogram. Set the bit width of the feature descriptor; The matching selection module is used to match feature points of the feature descriptors of two or more images, and select the feature point with the best matching result as the final image matching result based on the comparison result; Wherein, the respective calculation methods of the symbol descriptor, the mean descriptor and the central value descriptor in the scanning module include: Calculate the absolute value of each surrounding point pixel in each sliding window except the center point pixel compared to the absolute value of the center point pixel. If the absolute value of the surrounding point pixel is greater than the absolute value of the center point pixel, set it to 1 , if it is less than 0, set it to 0, and arrange the results in order to generate the symbolic descriptor; Calculate the average of all pixels in each sliding window and compare it with the average of pixels in the sliding window where the feature point is located. If the average of all pixels is greater than the average of pixels in the sliding window where the feature point is located Set 1 if it is less than 0, and arrange the results in order to generate the mean descriptor; Calculate the size of the center point value of each sliding window compared to the average value of the pixel matrix around the feature point and the average value of the full-image pixel matrix. If the center point value of each sliding window is greater than the average value of the pixel matrix around the feature point, value and the average value of the full-image pixel matrix are set to 1, otherwise set to 0, and the results are arranged in order to generate the central value descriptor.