CN115719320A - Tilt correction dense matching method based on remote sensing image - Google Patents

Abstract

The invention relates to the technical field of image processing, in particular to a tilt correction dense matching method based on remote sensing images, which comprises the steps of firstly generating horizontal epipolar line images by sparse homonymy points, further estimating initial parallaxes of left and right horizontal epipolar line image pairs, thereby improving the parallax estimation precision, reducing the search range of matching points, reducing the error matching probability, and reducing the calculated amount of gray level calculation by adopting increment calculation gray level mean value when determining the initial dense homonymy points through one-dimensional search; when searching for the matching point, the searching step length is set to be 2, so that the matching times are reduced, the matching efficiency is improved, and the initial dense homonymous points are finely adjusted through least square image matching, so that the aim of improving the matching precision is fulfilled.

Description

Tilt correction dense matching method based on remote sensing image

Technical Field

The invention relates to the technical field of image processing, in particular to a tilt correction dense matching method based on remote sensing images.

Background

Currently, there are two main ways of reconstructing three-dimensional objects and scenes, namely, active and passive. The method has high reconstruction precision and high reconstruction efficiency on the three-dimensional structure of the scene, but because the method depends on expensive measuring equipment and has a complicated data acquisition process, the method needs to acquire image information while acquiring three-dimensional information when acquiring a virtual three-dimensional model with photo reality, and involves the problem of registration between point cloud and images when data is post-processed, and the acquired data amount and the workload of a calculation process are huge; the passive mode adopts an image sensor to obtain image information of the surface of an object, and recovers three-dimensional structure information of the surface of the object according to a two-dimensional image with certain parallax based on the binocular stereoscopic vision principle. Active three-dimensional scanning has become an important means for acquiring three-dimensional information in many application fields, but passive three-dimensional modeling based on images is still the most economical, flexible, easy and widely-used method and plays an important role in various fields.

The key point of reconstructing the three-dimensional object, the scene and the earth surface based on the image is dense matching of the homonymous points, and the accurate and dense homonymous points are the basis for acquiring high-precision three-dimensional information. Dense matching is widely applied to fine reconstruction of objects, reconstruction of realistic scenes and generation of digital surface models and real digital orthoimages, and if accurate, dense and efficient matching of homonymous points can be achieved, the problem of low reconstruction efficiency caused by complex and time-consuming precision and data processing in three-dimensional reconstruction based on images is solved.

In the prior art, image matching methods are divided into a feature-based method and an image correlation-based method, the feature-based method is high in precision and accuracy, but only aims at characteristic salient regions such as corners, lines and edges in an image, so that only a sparse matching point set can be obtained, and the single-point calculation cost is much higher than that of the image correlation method, and the feature-based method is generally used for providing seed points or constraint conditions for dense matching; the image correlation-based method has large calculation amount and low efficiency, has self-similarity in a single texture region, and is easy to generate mismatching, so that the use of the method is limited, and therefore, the traditional image matching method is still a main implementation way of three-dimensional reconstruction based on images.

Chinese patent publication No. CN113034556A discloses a frequency domain correlation semi-dense remote sensing image matching method, aiming at obtaining matching information between two images; twisting one of the two images according to the initial matching information to obtain an image to be matched; dividing a reference image and an image to be matched into a plurality of areas; taking four small regions as a group of region image blocks, setting the centers of the region image blocks as seed points, and counting a gradient direction histogram of the region image blocks to obtain gradient amplitudes and angles corresponding to four angle ranges; setting four priority values of different levels; obtaining a new matching result combining the gradient information of the four adjacent domains of the seed points; adding the matching results of all the seed points into the sparse matching point set to obtain a semi-dense remote sensing image matching point set; and carrying out reference point mismatching correction on the dense matching points to obtain a matching result. According to the technical scheme, when the number of the blocks in the initial matching region is large, the data processing calculation workload is large, so that the three-dimensional reconstruction difficulty is increased, and when the number of the blocks in the initial matching region is small, the matching precision is difficult to guarantee.

Disclosure of Invention

Therefore, the invention provides a tilt correction dense matching method based on a remote sensing image, which is used for solving the problem that the three-dimensional reconstruction precision is difficult to improve due to complex image matching data processing in the process of performing three-dimensional reconstruction by adopting image matching in the prior art.

In order to achieve the above object, the present invention provides a tilt correction dense matching method based on remote sensing images, comprising the following steps:

step S1, respectively extracting feature points from a left oblique image shot by a left camera and a right oblique image shot by a right camera by adopting an SIFT feature descriptor to perform feature matching to obtain sparse homonymy points;

s2, carrying out relative orientation on the stereopair based on the sparse homonymous points and the camera intrinsic parameters to obtain the relative pose relationship of the left camera and the right camera;

s3, constructing a horizontal epipolar line coordinate system based on the relative pose relationship, and correcting the left oblique image and the right oblique image into a left horizontal epipolar line image and a right horizontal epipolar line image respectively through the mapping relationship between the horizontal epipolar line coordinate system and the left camera coordinate system and the mapping relationship between the horizontal epipolar line coordinate system and the right camera coordinate system;

s4, determining a plurality of dense points to be matched at set matching intervals in the left horizontal epipolar line image acquired in the S3, and performing one-dimensional gray-scale correlation matching on any dense point to be matched in the right horizontal epipolar line image to acquire initial dense homonymous points;

and S5, performing least square image matching on the initial dense homonymous points to obtain sub-pixel level dense homonymous points.

Further, in step S2, the relative pose relationship is obtained by a relative orientation direct solution method through the sparse homonymy points and the camera intrinsic parameters and is determined after adjustment and optimization by a light beam method, and the relative pose relationship is determined by a pose transformation matrix

In the description that follows,

setting the position of any point P in the left camera coordinate system as

Position of the point of identity under the right camera coordinate system

，

Wherein,

is a rotation matrix of the right camera coordinate system relative to the left camera coordinate system,

is a translation vector.

Further, in step S3, the horizontal epipolar coordinate system is a right-handed three-dimensional rectangular coordinate system, the origin of the horizontal epipolar coordinate system is set as the origin of the left camera coordinate system, the X-axis of the horizontal epipolar coordinate system is the connection line between the left camera projection center and the right camera projection center,

the coordinates of the X-axis vector of the horizontal epipolar line coordinate system in the left camera coordinate system are as follows:

；

the level isThe coordinates of the Y-axis vector of the epipolar coordinate system in the left camera coordinate system are:

；

the coordinate of the Z-axis vector of the horizontal epipolar line coordinate system in the left camera coordinate system is as follows:

；

the rotation transformation matrix of the vector under the left camera coordinate system to the direction of the horizontal epipolar coordinate system is as follows:

the rotation transformation matrix from the vector under the coordinate system of the right camera to the direction of the horizontal epipolar line coordinate system is as follows:

wherein,

is the average of the Z-axis direction vector of the left camera coordinate system and the Z-axis direction vector of the right camera coordinate system,

，

is the Z-axis vector of the left camera coordinate system,

is the Z-axis vector of the right camera coordinate system,

for said rotation matrix

The third column of elements.

Further, in step S3, by

And

solving corresponding coordinates of each pixel point in the original image in the horizontal epipolar line coordinate system to form mapping of each pixel point position of the original image and the corresponding point position of the horizontal epipolar line image, and filling color values of each pixel point of the original image into the corresponding point position of the horizontal epipolar line image to generate a corrected horizontal epipolar line image, wherein the original image comprises a left inclined image under the left camera coordinate system and a right inclined image under the right camera coordinate system, the left inclined image is corrected to generate a left horizontal epipolar line image, and the right inclined image is corrected to generate a right horizontal epipolar line image.

Further, in step S4, the step of obtaining the initial dense homonym includes:

step S41, calculating initial parallaxes of the left horizontal epipolar line image and the right horizontal epipolar line image in the X axis and the Y axis of the horizontal epipolar line coordinate system respectively;

step S42, determining a rough matching point of any dense point to be matched in the left horizontal epipolar line image in the right horizontal epipolar line image according to the initial parallax;

step S43, determining a one-dimensional search range in the right horizontal epipolar line image according to the position coordinate of the rough matching point as a search center and a set search radius, so as to determine a plurality of matching candidate points;

step S44, respectively calculating a normalized cross correlation coefficient between the gray average value of the source image block with the dense point to be matched as the center and the gray average value of the target image block with any one matching candidate point as the center;

and S45, determining the initial dense homonymous points of the dense points to be matched according to the normalized cross-correlation coefficient.

Further, in step S41, the step of calculating the initial parallax of the left horizontal epipolar line image and the right horizontal epipolar line image is:

step S411, respectively calculating X parallax and Y parallax of each sparse homonymous point in the left horizontal epipolar line image and the right horizontal epipolar line image under the horizontal epipolar line coordinate system;

step S412, three adjacent sparse homonymous point triangulation networks are selected to form a plurality of parallax calculation areas, and linear interpolation is adopted in the single parallax calculation area in the X direction and the Y direction respectively to calculate the X parallax d of any point in the parallax calculation area _x And Y parallax d _y ；

Step S413, when the dense point to be matched is located in a single parallax calculation region, adopting an X parallax d corresponding to the single parallax calculation region _x And Y parallax d _y 。

Further, in step S45, determining whether the normalized cross correlation coefficient meets the standard to determine whether there is an initial dense homonymous point in a single dense point to be matched, setting an initial matching correlation coefficient standard t and an initial matching correlation coefficient difference standard e, where t is greater than or equal to 0.7, and e is greater than 0 and less than or equal to 0.01, setting a maximum value of the normalized cross correlation coefficient between the gray level mean of the source image block centered on the dense point to be matched and the gray level mean of the target image block centered on any one of the matching candidate points as Enccmax1, a second maximum value as Enccmax2, setting Δ Encc = Enccmax1-Enccmax2,

when Enccmax1 is larger than or equal to t and delta Encc is larger than e, judging that the normalized cross-correlation coefficient accords with an initial matching standard, and setting a matching candidate point corresponding to Enccmax1 as an initial dense homonymous point of the point to be matched;

and when Enccmax1 is less than t or delta Encc is less than or equal to e, judging that the normalized cross-correlation coefficient does not accord with the initial matching standard, eliminating the points to be matched, which correspond to Enccmax1, and not having the initial dense homonymous points.

Further, in step S43, a search step is set to 2 pixels to determine the matching candidate points within the one-dimensional search range.

Further, in step S44, the gray level mean value is calculated in an incremental manner, where the incremental calculation formula is:

wherein, the target image block taking the mth matching candidate point as the center is recorded as the mth target image block,

is the gray-scale mean value of the mth target image block,

is the gray level mean value of the (m + 1) th target image block,

is the gray scale average value of the pixels of the leftmost two columns in the mth target image block,

and the gray level average value of the pixels in the rightmost two columns in the (m + 1) th target image block is obtained.

Further, in step S5, performing position adjustment on each initial dense homonymous point by using least square image matching to obtain two times dense homonymous points, determining position coordinates of sub-pixel level dense homonymous points according to a maximum value Encc2 in a normalized cross correlation coefficient between a gray level mean value of each image block corresponding to the two times dense homonymous points and a gray level mean value of a source image block with the corresponding dense point to be matched as a center, and setting a two times matching correlation coefficient standard tm, wherein tm is not less than 0.9,

when Encc2 is larger than or equal to tm, judging that the two-times normalized cross-correlation coefficient meets a sub-pixel level matching standard, and setting a two-times dense homonymy point corresponding to Encc2 as a sub-pixel level dense homonymy point of the dense point to be matched;

when Encc2 < tm, the two-times normalized cross-correlation coefficient is determined not to meet the sub-pixel level matching criterion.

Further, in step S5, the sub-pixel level dense same-name points are obtainedThen, carrying out actual position deviation checking calculation on the sub-pixel level dense homonymy points and the rough matching points in the X direction in the horizontal epipolar line image to obtain X direction deviation

And according to

It is determined whether an adjustment is made to the search radius r,

when in use

＞

Then, the search radius is judged to be smaller than the actual position deviation, r is adjusted, and the setting is made

；

When in use

≤

And then, judging that the search radius accords with the actual position deviation without adjusting r.

Further, in step S5, when all the sub-pixel level dense synonym points in any scanning line are obtained, calculating the actual position deviation of each sub-pixel level dense synonym point and the corresponding rough matching point in the X direction in the horizontal epipolar line image to obtain an X direction deviation mean value

And according to

Determining an adjustment to the search radius r,

when in use

When r is less than or equal to r, judging that the search radius is large, adjusting r, and setting

；

When in use

When r is greater than r, the search radius is judged to be small, r is adjusted and set

。

Further, the source image block and the target image block are both rectangular and have the same size, the dense point to be matched is located in the center of the source image block, the matching candidate point is located in the center of the target image block, and the size of the image block is taken

Wherein the length unit is a pixel.

Compared with the prior art, the tilt correction dense matching method based on the remote sensing image has the advantages that the horizontal epipolar line image is generated by the sparse homonymy points, the initial parallax of the left and right horizontal epipolar line image pairs is estimated, accordingly, the parallax estimation precision is improved, the search range of the matching points is reduced, the gray average value is calculated by adopting the increment when the initial dense homonymy points are determined through one-dimensional search, and the calculated amount of gray calculation is reduced; when the matching point is searched, the searching step length is set to be 2, so that the matching times are reduced, and the initial dense homonymous points are finely adjusted through least square image matching so as to achieve the purposes of improving the matching efficiency and reducing mismatching.

Furthermore, when sparse homonymy points are obtained, the accurate relative orientation elements of the stereo image can be obtained by adopting a relative orientation direct solution method and determining the relative orientation elements after adjustment optimization by a beam method, so that the accuracy of the relative pose relationship of two cameras forming the stereo model is determined, and the accuracy degree of the subsequent left and right horizontal epipolar line images is ensured.

Furthermore, the original images shot by the left camera and the right camera are respectively converted into the left horizontal epipolar line image and the right horizontal epipolar line image under the horizontal epipolar line coordinate system by constructing the horizontal epipolar line coordinate system, the parallax of the three-dimensional images shot by the left camera and the right camera is analyzed in the same coordinate system, the convenience degree and the accuracy of parallax analysis of the left image and the right image are effectively improved, the generated left horizontal epipolar line image and the right horizontal epipolar line image can be used as the basis for dense matching, and the efficiency of dense homonymy point matching is improved.

Further, the invention carries out dense homonymy point matching on the left horizontal epipolar line image and the right horizontal epipolar line image in the horizontal epipolar line coordinate system, because the parallax calculation in the X direction and the Y direction is carried out on the horizontal epipolar line coordinate system, when the dense homonymy point matching is carried out on a single point to be matched, the matching precision of the corresponding matched rough matching point reaches certain precision, and because the X axis of the constructed horizontal epipolar line coordinate system is constructed by the original points of the left camera coordinate system and the right camera coordinate system, the parallax in the Y axis direction of the horizontal epipolar line coordinate system has fixity theoretically.

Furthermore, the similarity degree of the two image blocks is measured by adopting the normalized cross-correlation coefficient, and the normalized cross-correlation coefficient has better robustness on linear brightness change between the image blocks, so that the linear brightness change existing in the image block characteristic represented by the gray mean value can be overcome, the characteristic correlation degree of matching correlation conforming to the matching point when the dense homonymous point is matched is further improved, and the degree of closeness of the dense homonymous point obtained by matching with the actual image by the method is improved.

Further, the invention determines whether the single point to be matched has initial dense homonymy points by judging whether the normalized cross correlation coefficient accords with the standard or not, and is provided with an initial matching correlation coefficient standard t and an initial matching correlation coefficient difference standard e, so as to calculate the normalized cross correlation coefficient of the single point to be matched and a plurality of matching candidate points to obtain the initial dense homonymy points, the initial dense homonymy points are ensured to have certain matching precision by judging the size relation of the normalized cross correlation coefficient and the initial matching correlation coefficient standard t, the correctness of the initial matching is ensured, the increase of invalid calculation amount caused by carrying out least square matching on the points with poor matching precision is avoided, the calculation efficiency of homonymy point matching is further improved, meanwhile, the initial matching correlation coefficient difference standard e is used as the matching significance standard of the initial dense homonymy points, the judged initial dense homonymy points and other matching candidate matching points have better matching performance with the points to be matched, and the matching credibility is improved.

Furthermore, the search step length is set to be 2 pixels during one-dimensional search, so that on one hand, the number of the matching candidate points is reduced to half of the number of the matching candidate points during pixel-by-pixel matching by expanding the step length, the matching calculation amount is reduced by half of the original calculation amount, and the search calculation efficiency is improved; on the other hand, since the final dense homonymy point is obtained by performing least square matching after the initial dense homonymy point is obtained subsequently, the matching performance of pixels adjacent to the initial dense homonymy point as matching candidate points through the covering calculation of the least square matching is ensured by setting the search step length to be 2 pixels, so that the matching of the dense homonymy point by the method disclosed by the invention can cover each pixel in the search range, the optimal matching point of the matched dense homonymy point and the non-to-be-matched point caused by the sparse search point is avoided, and the balance of the search efficiency and the accuracy degree is further ensured.

Furthermore, the gray level mean value of the target image block is calculated in an incremental calculation mode, and the matching candidate points have certain density, so that the adjacent target image blocks are overlapped in a certain area, and when the gray level mean value of the previous target image block is obtained, the gray level mean value is quickly calculated by only calculating the gray level value of the non-overlapped area in the adjacent target image block, so that the problem of large calculation amount caused by repeatedly calculating the gray level value of the same pixel block is solved, and the calculation efficiency of the method is further improved.

Further, after the sub-pixel level dense homonymy points are obtained, the sub-pixel level dense homonymy points and the rough matching points are subjected to actual position deviation checking calculation in the X direction in the horizontal epipolar line image to obtain X direction deviation

And according to

Determining whether to adjust the search radius r, and when all the sub-pixel-level dense homonymy points in any scanning line are obtained, calculating the actual position deviation of each sub-pixel-level dense homonymy point and the corresponding rough matching point in the X direction in the horizontal epipolar line image to obtain an X-direction deviation mean value

And according to

And determining an adjustment mode of the search radius r, and adjusting the search radius r to ensure that the subsequent one-dimensional search range is more accurate, further reducing invalid calculation amount and improving the calculation efficiency of the method.

Drawings

FIG. 1 is a diagram of the steps of the tilt correction dense matching method based on remote sensing images of the present invention;

FIG. 2 is a diagram of the steps of the present invention to obtain the initial dense homonyms;

FIG. 3 is a diagram illustrating an initial parallax calculation step according to the present invention;

FIG. 4 is a schematic diagram of a triangle difference according to an embodiment of the present invention.

Detailed Description

In order that the objects and advantages of the invention will be more clearly understood, the invention is further described below with reference to examples; it should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

Preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only for explaining the technical principles of the present invention, and do not limit the scope of the present invention.

It should be noted that in the description of the present invention, the terms of direction or positional relationship indicated by the terms "upper", "lower", "left", "right", "inner", "outer", etc. are based on the directions or positional relationships shown in the drawings, which are only for convenience of description, and do not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation, and thus, should not be construed as limiting the present invention.

Furthermore, it should be noted that, in the description of the present invention, unless otherwise explicitly specified or limited, the terms "mounted," "connected," and "connected" are to be construed broadly, and may be, for example, fixedly connected, detachably connected, or integrally connected; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meanings of the above terms in the present invention can be understood by those skilled in the art according to specific situations.

Referring to fig. 1, which is a step diagram of the tilt correction dense matching method based on the remote sensing image according to the present invention, the present invention provides a tilt correction dense matching method based on the remote sensing image, including:

step S1, respectively extracting feature points from a left oblique image shot by a left camera and a right oblique image shot by a right camera by adopting an SIFT feature descriptor to perform feature matching to obtain sparse homonymy points;

s2, carrying out relative orientation on the stereopair based on the sparse homonymous points and the camera intrinsic parameters to obtain the relative pose relationship of the left camera and the right camera;

s3, constructing a horizontal epipolar line coordinate system based on the relative pose relationship, and correcting the left oblique image and the right oblique image into a left horizontal epipolar line image and a right horizontal epipolar line image respectively through the mapping relationship between the horizontal epipolar line coordinate system and the left camera coordinate system and the mapping relationship between the horizontal epipolar line coordinate system and the right camera coordinate system;

step S4, determining a plurality of dense points to be matched at set matching intervals in the left horizontal epipolar line image obtained in the step S3, and performing one-dimensional gray-scale correlation matching on any dense point to be matched in the right horizontal epipolar line image to obtain initial dense homonymy points;

and S5, performing least square image matching on the initial dense homonymous points to obtain sub-pixel level dense homonymous points.

According to the tilt correction dense matching method based on the remote sensing image, the horizontal epipolar line image is generated by the sparse homonymy points, the initial parallax of the left and right horizontal epipolar line image pairs is estimated, the parallax estimation precision is improved, the search range of the matching points is reduced, the gray level mean value is calculated in an increment mode when the initial dense homonymy points are determined through one-dimensional search, and the calculation amount of gray level calculation is reduced; when the matching point is searched, the searching step length is set to be 2, so that the matching times are reduced, and the initial dense homonymy points are finely adjusted through least square image matching so as to achieve the purposes of improving the matching efficiency and reducing mismatching.

Referring to fig. 1, in step S2, the relative pose relationship is obtained by the sparse homonym point and the camera intrinsic parameter by using a relative orientation direct solution and determined by performing adjustment optimization by a light beam method, and the relative pose relationship is determined by using a pose transformation matrix

In the description that follows,

setting the position of any point P in the left camera coordinate system as

Position of the point of identity under the right camera coordinate system

，

Wherein,

is a rotation matrix of the right camera coordinate system relative to the left camera coordinate system,

is a translation vector.

According to the method, when sparse homonymy points are obtained, the sparse homonymy points are obtained by adopting a relative orientation direct solution method and are determined after adjustment optimization by a light beam method, and accurate relative orientation elements of the stereoscopic image can be obtained, so that the relative pose relationship of two cameras forming the stereoscopic model is determined to have accuracy, and the accuracy degree of the subsequent left and right horizontal epipolar line images is ensured.

Referring to fig. 1, in step S3, the horizontal epipolar coordinate system is a right-handed three-dimensional rectangular coordinate system, the origin of the horizontal epipolar coordinate system is set as the origin of the left-camera coordinate system, the X-axis of the horizontal epipolar coordinate system is the connection line between the left-camera projection center and the right-camera projection center,

the coordinates of the X-axis vector of the horizontal epipolar line coordinate system in the left camera coordinate system are as follows:

；

the coordinates of the Y-axis vector of the horizontal epipolar line coordinate system in the left camera coordinate system are as follows:

；

the coordinate of the Z-axis vector of the horizontal epipolar line coordinate system in the left camera coordinate system is as follows:

；

the rotation transformation matrix from the vector under the left camera coordinate system to the direction of the horizontal epipolar line coordinate system is as follows:

the rotation transformation matrix of the vector under the right camera coordinate system to the direction of the horizontal epipolar coordinate system is as follows:

wherein,

is the average of the Z-axis direction vector of the left camera coordinate system and the Z-axis direction vector of the right camera coordinate system,

，

is the Z-axis vector of the left camera coordinate system,

is the Z-axis vector of the right camera coordinate system,

for said rotation matrix

The third column of elements.

According to the invention, the horizontal epipolar line coordinate system is constructed to respectively convert the original images shot by the left camera and the right camera into the left horizontal epipolar line image and the right horizontal epipolar line image under the horizontal epipolar line coordinate system, the parallax of the three-dimensional images shot by the left camera and the right camera is analyzed in the same coordinate system, the convenience and accuracy of parallax analysis of the left image and the right image are effectively improved, the generated left horizontal epipolar line image and the right horizontal epipolar line image can be used as the basis for carrying out dense matching, and the efficiency of dense homonymy point matching is improved.

Continuing to refer to FIG. 1, in step S3, the process proceeds by

And

solving corresponding coordinates of each pixel point in the original image in the horizontal epipolar line coordinate system to form mapping of each pixel point position of the original image and the corresponding point position of the horizontal epipolar line image, and filling the color value of each pixel point of the original image in the corresponding point position of the horizontal epipolar line image to generate a corrected horizontal epipolar line image, wherein the original image comprises a left inclined image under the left camera coordinate system and a right inclined image under the right camera coordinate system, the left inclined image is corrected to generate a left horizontal epipolar line image, and the right inclined image is corrected to generate a right horizontal epipolar line image.

Please refer to fig. 2, which is a diagram illustrating the steps of obtaining the initial dense homonym according to the present invention, in step S4, the step of obtaining the initial dense homonym includes:

step S41, respectively calculating the initial parallaxes of the left horizontal epipolar line image and the right horizontal epipolar line image in the X axis and the Y axis of the horizontal epipolar line coordinate system;

step S42, determining a rough matching point of any dense point to be matched in the left horizontal epipolar line image in the right horizontal epipolar line image according to the initial parallax;

step S43, determining a one-dimensional search range in the right horizontal epipolar line image according to the position coordinate of the rough matching point as a search center and a set search radius, so as to determine a plurality of matching candidate points;

step S44, respectively calculating a normalized cross correlation coefficient between the gray mean value of the source image block with the dense points to be matched as the center and the gray mean value of the target image block with any one of the matching candidate points as the center;

and S45, determining the initial dense homonymous points of the dense points to be matched according to the normalized cross-correlation coefficient.

Referring to fig. 3, which is a diagram illustrating an initial parallax calculation step according to the present invention, in step S41, the initial parallax calculation steps of the left horizontal epipolar line image and the right horizontal epipolar line image are as follows:

step S411, respectively calculating an X parallax and a Y parallax of each sparse homonymy point in the left horizontal epipolar line image and the right horizontal epipolar line image under the horizontal epipolar line coordinate system;

step S412, selecting three adjacent sparse homonymous point triangulation networks to form a plurality of parallax calculation areas, and calculating the X parallax d of any point in the parallax calculation areas in the single parallax calculation area in the X direction and the Y direction by adopting linear interpolation _x And Y parallax d _y ；

Step S413, when the dense point to be matched is located in a single parallax calculation region, adopting an X parallax d corresponding to the single parallax calculation region _x And Y parallax d _y 。

Specifically, the initial parallax of the left horizontal epipolar line image and the right horizontal epipolar line image is calculated, and any dense point to be matched in the left horizontal epipolar line image is determined according to the initial parallax

The rough matching point in the right horizontal epipolar line image is recorded as

Will be

Set as the search center, r is the search halfIs directed at the second of the right horizontal epipolar line images

Of a scanning line

One-dimensional search is carried out in the pixel range to determine a plurality of matching candidate points, and the dense points to be matched are respectively calculated

Normalized cross-correlation coefficient between the gray mean of the centered source image block and the gray mean of the target image block centered on each of the matching candidate points to determine an initial dense homonymous point,

is composed of

The coordinates in the left horizontal epipolar line image,

is composed of

Coordinates in the right horizontal epipolar image.

The invention carries out dense homonymy point matching on the left horizontal epipolar line image and the right horizontal epipolar line image in the horizontal epipolar line coordinate system, because the parallax calculation in the X direction and the Y direction is carried out on the horizontal epipolar line coordinate system, when the dense homonymy point matching is carried out on a single point to be matched, the matching precision of a corresponding matched rough matching point reaches certain precision, and because the X axis of the constructed horizontal epipolar line coordinate system is constructed by the original points of the left camera coordinate system and the right camera coordinate system, the parallax in the Y axis direction of the horizontal epipolar line coordinate system has fixity theoretically.

Continuing to refer to fig. 2, in step S45, determining whether the normalized cross-correlation coefficient meets the criterion to determine whether there is an initial dense homonymous point at a single point to be matched, setting an initial matching correlation coefficient criterion t and an initial matching correlation coefficient difference criterion e, where t is greater than or equal to 0.7, e is greater than 0 and less than or equal to 0.01, recording the maximum value of the calculated normalized cross-correlation coefficients as Enccmax1, and the second maximum value as Enccmax2, setting Δ enccc = Enccmax1-Enccmax2,

when Enccmax1 is larger than or equal to t and delta Encc is larger than e, judging that the normalized cross-correlation coefficient accords with an initial matching standard, and setting a matching candidate point corresponding to Enccmax1 as an initial dense homonymous point of the point to be matched;

and when Enccmax1 is less than t or delta Encc is less than or equal to e, judging that the normalized cross-correlation coefficient does not accord with the initial matching standard, eliminating the points to be matched, which correspond to Enccmax1, and not having the initial dense homonymous points.

The method measures the similarity degree of the two image blocks by adopting the normalized cross-correlation coefficient, and the normalized cross-correlation coefficient has better robustness to the linear brightness change between the image blocks, so that the linear brightness change existing in the image block characteristic represented by the gray mean value can be overcome, the matching correlation degree of the method for matching the dense homonymous points is further improved to accord with the characteristic correlation degree of the matching points, and the proximity degree of the dense homonymous points matched by the method of the invention and the homonymous points of the actual image is improved.

The method determines whether the single point to be matched has initial dense homonymy points or not by judging whether the normalized cross-correlation coefficient accords with the standard or not and setting an initial matching correlation coefficient standard t and an initial matching correlation coefficient difference standard e so as to calculate the normalized cross-correlation coefficient of the single point to be matched and a plurality of matching candidate points to obtain the initial dense homonymy points, ensures that the initial dense homonymy points have certain matching precision by judging the size relationship between the normalized cross-correlation coefficient and the initial matching correlation coefficient standard t so as to ensure the correctness of the initial matching, avoids the increase of invalid calculation amount caused by the least square matching of the points with poor matching precision, further improves the calculation efficiency of the homonymy point matching, and simultaneously takes the initial matching correlation coefficient difference standard e as the matching significance standard of the initial homonymy points, further ensures that the judged initial dense homonymy points have better matching with the points to be matched compared with other matching candidate matching points, and improves the matching credibility.

Referring to fig. 2, in step S43, the first horizontal epipolar line image

Of the scanning line

When one-dimensional search is carried out in the pixel range, the search step length is set to be 2 pixels,

the coordinates of the matching candidate points are obtained by calculation

Wherein i =0,1, \8230;, r.

According to the method, the search step length is set to be 2 pixels during one-dimensional search, so that on one hand, the number of the matching candidate points is reduced to half of the number of the matching candidate points during pixel-by-pixel matching by enlarging the step length, the matching calculation amount is reduced by half of the original calculation amount, and the search calculation efficiency is improved; on the other hand, because the least square matching is carried out after the initial dense homonymy point is obtained subsequently to obtain the final dense homonymy point, the searching step length is set to be 2 pixels, the matching performance that the pixels adjacent to the initial dense homonymy point can be covered and calculated as matching candidate points through the least square matching is ensured, so that the matching of the dense homonymy point can cover each pixel in the searching range by the method, the optimal matching point of the matched dense homonymy point and the point not to be matched due to the sparse searching points is avoided, and the balance of the searching efficiency and the accuracy degree is further ensured.

As shown in fig. 1, in step S4, the gray-scale mean value is calculated in an incremental manner, where the incremental formula is:

wherein, the target image block taking the mth matching candidate point as the center is recorded as the mth target image block,

is the gray-scale mean value of the mth target image block,

is the gray level average value of the (m + 1) th target image block,

is the gray scale average value of the pixels of the leftmost two columns in the mth target image block,

and the gray level average value of the pixels in the rightmost two columns in the (m + 1) th target image block is obtained.

According to the invention, the gray average value of the target image block is calculated by adopting an incremental calculation mode, and the matching candidate points have certain dense degree, so that the adjacent target image blocks are overlapped in a certain area, therefore, when the gray average value of the previous target image block is obtained, the gray average value is quickly calculated by only calculating the gray value of the non-overlapped area in the adjacent target image block, the problem of large calculation amount caused by repeatedly calculating the gray value of the same pixel block is reduced, and the calculation efficiency of the method is further improved.

As shown in fig. 1, in the step S5, performing position adjustment on each initial dense homonymous point by using least square image matching to obtain two times dense homonymous points, determining a position coordinate of a sub-pixel level dense homonymous point according to a maximum value Encc2 in a normalized cross-correlation coefficient between a gray level of an image block corresponding to each two times dense homonymous point and a gray level of a source image block centered on the corresponding dense point to be matched, and setting a two times matching correlation coefficient standard tm, where tm is greater than or equal to 0.9,

when Encc2 is larger than or equal to tm, judging that the two-times normalized cross-correlation coefficient meets a sub-pixel level matching standard, and setting a two-times dense homonymy point corresponding to Encc2 as a sub-pixel level dense homonymy point of the dense point to be matched;

when Encc2 < tm, the two-times normalized cross-correlation coefficient is determined not to meet the sub-pixel level matching criterion.

Specifically, in step S5, after the sub-pixel level dense homonymy point is obtained, the sub-pixel level dense homonymy point and the rough matching point are subjected to actual position deviation checking in the X direction in the horizontal epipolar line image to obtain an X direction deviation

And according to

Determining whether to adjust the preset search radius r,

when in use

＞

Then, the preset search radius is judged to be smaller than the actual position deviation, r is adjusted, and the preset search radius is set

；

When in use

≤

When it is determined that the preset search half is setThe diameter conforms to the actual position deviation without adjusting r.

Specifically, in step S5, after the sub-pixel level dense homonymy point is obtained, the sub-pixel level dense homonymy point and the rough matching point are subjected to actual position deviation checking in the X direction in the horizontal epipolar line image to obtain an X direction deviation

And according to

It is determined whether an adjustment is made to the search radius r,

when in use

＞

Then, the search radius is judged to be smaller than the actual position deviation, r is adjusted, and the setting is carried out

；

When in use

≤

And then, judging that the search radius accords with the actual position deviation without adjusting r.

Specifically, in step S5, when all the sub-pixel level dense synonym points in any scanning line are obtained, calculating the actual position deviation of each sub-pixel level dense synonym point and the corresponding rough matching point in the X direction in the horizontal epipolar line image to obtain an X direction deviation mean value

And according to

Determining an adjustment to the search radius r,

when the temperature is higher than the set temperature

When the radius is less than or equal to r, judging that the search radius is large, adjusting r, and setting

；

When in use

When the radius is larger than r, judging that the search radius is small, adjusting r, and setting

。

Further, the source image block and the target image block are both rectangular and have the same size, the dense point to be matched is located in the center of the source image block, the matching candidate point is located in the center of the target image block, and the size of the image block is taken

Wherein the length unit is a pixel.

Specifically, when the first step is obtained

When all the sub-pixel level dense homonymous points in the scanning line are scanned, calculating actual position deviation of each sub-pixel level dense homonymous point and the corresponding rough matching point in the X direction of the horizontal epipolar line image to obtain deviation differences in the X direction, and averaging the deviation differences to obtain

。

The embodiment is as follows:

the embodiment provides a specific calculation process of a tilt correction dense matching method based on a remote sensing image, and for a specific stereoscopic image E, a left tilt image obtained by shooting the stereoscopic image E by a left camera and a right tilt image obtained by shooting the stereoscopic image E by a right camera are adopted for an original image, and the method comprises the following steps:

step one, feature matching;

firstly, extracting feature points of a left oblique image and a right oblique image by using an SIFT feature descriptor, and performing feature matching; and solving an affine transformation relation of the two images according to the characteristic matching points by adopting a RANSAC algorithm, eliminating a small number of mismatching points, finally obtaining a certain number of reliable homonymous image points distributed in the whole overlapping area, and forming a three-dimensional model by the left image and the right image.

Step two, relative orientation;

based on the homonymous points and camera intrinsic parameters of the left oblique image and the right oblique image, the precise relative orientation element of the stereoscopic image E can be obtained by a relative orientation direct solution method and by combining with adjustment optimization of a light beam method, namely the relative pose relationship of the left camera and the right camera forming the stereoscopic image E is determined, namely the rotation matrix relation of a right camera coordinate system relative to a left camera coordinate system is determined

And translation vector

。

Step three, generating a horizontal epipolar line image of the stereoscopic image E, comprising:

step 31, constructing a horizontal epipolar coordinate system;

and taking the origin of the left camera coordinate system as the origin of the horizontal epipolar coordinate system, and defining the horizontal epipolar coordinate system by determining the coordinates of the vectors corresponding to the X, Y and Z three axes of the horizontal epipolar coordinate system under the left camera coordinate system. Setting a connecting line vector of the projection centers of the left camera and the right camera as an X axis of a horizontal epipolar coordinate system, wherein the coordinate of the X axis vector of the horizontal epipolar coordinate system under a left camera coordinate system is a translation vector t obtained by relative orientation, and the coordinate of the X axis vector of the horizontal epipolar coordinate system under the left camera coordinate system is as follows:

；

the coordinates of the Y-axis vector of the horizontal epipolar line coordinate system in the left camera coordinate system are as follows:

；

the coordinate of the Z-axis vector of the horizontal epipolar line coordinate system in the left camera coordinate system is as follows:

；

to this end, the coordinates of the three axis vectors of the horizontal epipolar coordinate system in the left camera coordinate system have been determined, and the process described above constructs a right-hand three-dimensional rectangular coordinate system.

After the horizontal epipolar coordinate system is constructed, virtual left and right horizontal epipolar cameras are constructed simultaneously, wherein the three axes X, Y and Z of the left and right horizontal epipolar camera coordinate systems are parallel to the three axes of the horizontal epipolar coordinate system, and the origins of the left and right horizontal epipolar camera coordinate systems are respectively set as the origins of the left and right oblique camera coordinate systems, namely the projection centers of the left and right oblique cameras.

After the horizontal epipolar coordinate system is constructed, the rotation transformation matrix from the vector under the left camera coordinate system to the horizontal epipolar coordinate system direction is as follows:

the rotation transformation matrix of the vector under the right camera coordinate system to the direction of the horizontal epipolar coordinate system is as follows:

wherein,

is the Z-axis direction vector of the left camera coordinate system and the Z-axis direction of the right camera coordinate systemThe average of the vectors is then calculated,

，

is the Z-axis vector of the left camera coordinate system,

is the Z-axis vector of the right camera coordinate system,

for the rotation matrix

The third column of elements.

Step 32, correcting the image;

taking the left oblique image corresponding to the left camera as an example, the process of correcting the oblique image into the horizontal image is as follows,

for any pixel point in the left inclined image

Let the corresponding point in the left horizontal epipolar line image be

And is and

corresponding normalized homogeneous coordinates of

The method comprises the following steps:

（1）

and

the conversion relationship between the two is as follows:

（2）

wherein,

is the inverse of the orientation matrix in the camera,

is the focal length of the left camera and,

the coordinates of the image principal point of the left camera;

is the internal orientation matrix of the horizontal epipolar line camera,

is the focal length of the horizontal epipolar camera, setting

To ensure that the horizontal epipolar line image has a similar resolution to the original image,

is the image principal point coordinate of the horizontal epipolar line camera.

Combining the formula (1) and the formula (2), any pixel in the left oblique image can be mapped to the left horizontal epipolar line image, and vice versa;

for the right oblique image corresponding to the right camera, the image is obtained by using the image obtained by the formula (2)

Instead of using

The mapping relationship between the right oblique image and the right horizontal epipolar line image can be obtained.

In the practice, first, the

、

Setting the coordinate of four vertex pixels of the inclined image imaging range rectangle in the horizontal epipolar line image as 0, calculating the coordinate of four vertex pixels of the inclined image imaging range rectangle by adopting an equation (1) and an equation (2), obtaining a minimum rectangular bounding box of the horizontal epipolar line image imaging range, and calculating the coordinate of the four vertex pixels in the horizontal epipolar line image in the minimum rectangular bounding box of the horizontal epipolar line image imaging range

Set as the x coordinate of the lower left corner of the minimum rectangular bounding box,

Setting the y coordinate of the lower left corner of the minimum rectangular bounding box (namely the minimum x and y coordinates of the bounding box), so that the lower left corner of the minimum rectangular bounding box is superposed with the lower left corner of the horizontal epipolar line image, setting the width and height of the minimum rectangular bounding box as the corresponding width and height of the horizontal epipolar line image, scanning each pixel point of the horizontal epipolar line image, solving the corresponding point coordinate of the pixel point on the original image, and obtaining the color value of each point on the original image by gray resampling and filling the color value into the corresponding point of the horizontal epipolar line image, thereby finally generating the corrected horizontal image.

Step four, gray scale correlation matching comprises;

step 41, disparity estimation;

in order to estimate the initial parallax of a horizontal image pair, firstly adopting an equation (2) to convert the original image coordinates of all sparse homonymous points into horizontal epipolar line image coordinates; and then, interpolating the x-disparity and the y-disparity of each pixel of the horizontal image in the triangle according to the horizontal image disparity of three vertexes of each triangle in the x-direction and the y-direction by using a sparse point triangulation network.

Please refer to fig. 4, which is a schematic diagram of a triangle difference according to an embodiment of the present inventionIn (1)

、

、

Three vertexes of the triangle are provided, and the corresponding x parallaxes are respectively d _x1 、d _x2 、d _x3 ，

Is a point inside the triangle and is,

、

is the y-th pixel row and two sides P of the triangle ₁ P ₂ 、P ₁ P ₃ The intersection point of (a). First, a point p is found by linear interpolation in the y direction ₁₂ 、p ₁₃ X parallax of d _x12 、d _x13 As shown in formula (3):

(3)

by converting the parallax in the equation (3) into the x coordinate of the corresponding point, the point can be obtained

、

X coordinate x of ₁₂ 、x ₁₃ . Then the point of origin

、

Linear interpolation in the x direction, i.e.To find out

X parallax d of points _x As shown in formula (4):

(4)

by the same method, can obtain

Y parallax d of points _y 。

It should be noted that the y-parallax at any position in the horizontal epipolar image pair is theoretically fixed, and the y-parallax is theoretically corrected by the principal point row coordinates

It is determined, but in actual processing, it is found that due to errors of distortion coefficients of the camera and uncertainty of distortion, the parallax of the generated horizontal epipolar line image in the y direction is not fixed, but fluctuates within a plurality of pixel ranges near a theoretical value, so that the y parallax is interpolated according to three vertexes of a triangle, the estimation accuracy of the parallax is greatly improved, and the matching accuracy is further improved.

Step 24, correlation matching;

the similarity degree of the two image blocks is measured by adopting a normalized cross-correlation coefficient, the normalized cross-correlation coefficient is robust to linear brightness change between the image blocks, and the calculation formula is as follows:

（5）

wherein:

is the mean of the gray levels of the two related blocks, N is the number of pixels in the block;

it should be noted that, for a color image, the gray-scale mean value of each color channel should be calculated separately, and all the color channels should be connected in parallel to calculate the correlation coefficient, and the corresponding calculation formula is as follows:

（6）

wherein c is the number of color channels;

setting the matching interval of dense points to be matched as mxn, namely matching one dense same-name point every m rows and n columns, wherein the smaller the interval, the higher the density and the higher the reconstruction fineness, and the values of m and n can be set according to the three-dimensional scene complexity of an actual scene, which is not described again;

for any point to be matched on the left horizontal epipolar line image

Estimating the coarse matching point in the right horizontal epipolar line image according to the parallax at the point as

And are combined with

Centered on the right horizontal image

Of the scanning line

Performing one-dimensional search matching in a pixel range, wherein r is a search radius;

it should be noted that although the y parallax is not fixed, the fluctuation of the y parallax is small in a small range of the search area, so that by fixing the y parallax, the search range can be reduced, and the amount of calculation data can be reduced; because the estimation precision of the rough matching point is higher, r is not necessarily large, on one hand, the search range is reduced, the matching efficiency is improved, on the other hand, the probability of mismatching is reduced, and the matching accuracy is improved.

For any rough point to be matched, calculating a normalized correlation coefficient between a target image block taking the rough point to be matched as a center and a source image block taking a corresponding dense point to be matched as a center according to the formula (6); taking the rough point to be matched corresponding to the maximum value of the normalized correlation coefficient calculation value from the normalized correlation coefficient calculation values of all the rough points to be matched, and if the correlation coefficient of the point is greater than t and the difference between the maximum value of the normalized correlation coefficient calculation value and the second largest value is greater than e, considering the point as an initial dense homonymous point; otherwise, rejecting the point; t is more than or equal to 0.7 and is used for ensuring the correctness of initial matching, and e is more than 0 and less than or equal to 0.01 so as to ensure that the homonymy points have certain significance and improve the reliability.

Specifically, the target image block size is generally 11 pixels × 11 pixels.

Specifically, when searching for the initial dense homonym point, the following 2 strategies are adopted to improve the matching efficiency:

the method comprises the steps that firstly, when one-dimensional search is carried out in the search range of any scanning line of the right horizontal epipolar line image, the search step length is set to be 2 pixels, and the coordinates of the matching candidate points are obtained through calculation;

that is, the coordinates of the matching candidate points are not traversed through each pixel in the search range, but the correlation coefficient is obtained every other pixel, so that the search amount is halved, and the purpose of improving the matching efficiency is achieved.

The rationale for this is: because the searched initial dense homologous points are subjected to fine adjustment by adopting least square matching, and the correction range of the least square matching can reach 2 pixels, even if the actual best matching point is in the interval pixels which are not scanned, the searched initial dense homologous points are adjacent pixels of the actual best matching point and can be corrected to the actual best position by the least square matching, and point-by-point matching is not needed.

And a second strategy is that when the initial dense homonymy points are searched in the search range, incremental calculation is adopted when the gray average value in the correlation coefficient is calculated:

calculating to obtain the mean value of the source image at the initial matching candidate point of the search range

And target image mean

(ii) a In the subsequent searching matching in sequence along the searching range,

is kept unchanged, to

Updating is performed by summing the gray levels from the previous window (N in a pattern)

) Subtracting the gray scale of the front 2 columns of the window and adding the gray scale of the rear 2 columns of the window to obtain the gray scale sum of the rear window, and further obtaining the gray scale sum of the rear window

；

The incremental calculation formula is:

wherein, the target image block taking the mth matching candidate point as the center is recorded as the mth target image block,

is the gray-scale mean value of the mth target image block,

is the gray level average value of the (m + 1) th target image block,

is the gray scale average value of the pixels of the leftmost two columns in the mth target image block,

and the gray level average value of the pixels in the rightmost two columns in the (m + 1) th target image block is obtained.

The use of the incremental calculation process can reduce the amount of gradation calculation.

Strategy three, adding order consistency constraint to the matching points along the scanning line, namely adding order consistency constraint to two points on the left horizontal epipolar line image

、

The corresponding matching points on the right horizontal epipolar line image are respectively

、

Set up

Is located at

To the right side of

Must be located at

To the right, such processing further enhances the reliability of the match.

Step five, least square matching;

and C, performing least square image matching on the initial dense homonymous points acquired in the step four, so that the matching precision of the homonymous points reaches a sub-pixel level.

Because point-by-point searching and matching can only reach the matching precision of 1 pixel, the searched initial matching point is finely adjusted by adopting least square matching, so that the homonymy point reaches the matching precision of a sub-pixel level, and the visual effect of a reconstructed scene is improved.

So far, the technical solutions of the present invention have been described in connection with the preferred embodiments shown in the drawings, but it is apparent to those skilled in the art that the scope of the present invention is not limited to these specific embodiments. Equivalent changes or substitutions of related technical features can be made by those skilled in the art without departing from the principle of the invention, and the technical scheme after the changes or substitutions can fall into the protection scope of the invention.

The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention; various modifications and alterations to this invention will become apparent to those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (10)

1. A tilt correction dense matching method based on remote sensing images is characterized by comprising the following steps:

step S1, respectively extracting feature points from a left oblique image shot by a left camera and a right oblique image shot by a right camera by adopting an SIFT feature descriptor to perform feature matching to obtain sparse homonymy points;

s2, carrying out relative orientation on the stereopair based on the sparse homonymous points and the camera intrinsic parameters to obtain the relative pose relationship of the left camera and the right camera;

s3, constructing a horizontal epipolar line coordinate system based on the relative pose relationship, and correcting the left oblique image and the right oblique image into a left horizontal epipolar line image and a right horizontal epipolar line image respectively through the mapping relationship between the horizontal epipolar line coordinate system and the left camera coordinate system and the mapping relationship between the horizontal epipolar line coordinate system and the right camera coordinate system;

s4, determining a plurality of dense points to be matched at set matching intervals in the left horizontal epipolar line image acquired in the S3, and performing one-dimensional gray-scale correlation matching on any dense point to be matched in the right horizontal epipolar line image to acquire initial dense homonymous points;

and S5, performing least square image matching on the initial dense homonymous points to obtain sub-pixel-level dense homonymous points.

2. The tilt correction dense matching method based on remote sensing images as claimed in claim 1, wherein in step S2, the relative pose relationship is obtained by the sparse homonymy point and camera intrinsic parameters by using a relative orientation direct solution and determined after adjustment optimization by a beam method, and the relative pose relationship is determined by using a pose transformation matrix

In the description that follows,

setting the position of any point P in the left camera coordinate system as

Position of the point of identity under the right camera coordinate system

，

Wherein,

is a rotation matrix of the right camera coordinate system relative to the left camera coordinate system,

is a translation vector.

3. The tilt-correction dense matching method based on remote sensing images according to claim 2, wherein in step S3, the horizontal epipolar line coordinate system is a right-hand three-dimensional rectangular coordinate system, the origin of the horizontal epipolar line coordinate system is set as the origin of the left camera coordinate system, the X-axis of the horizontal epipolar line coordinate system is the connecting line of the left camera projection center and the right camera projection center,

the coordinates of the X-axis vector of the horizontal epipolar line coordinate system in the left camera coordinate system are as follows:

；

the coordinates of the Y-axis vector of the horizontal epipolar line coordinate system in the left camera coordinate system are as follows:

；

the coordinate of the Z-axis vector of the horizontal epipolar line coordinate system in the left camera coordinate system is as follows:

；

the rotation transformation matrix from the vector under the left camera coordinate system to the direction of the horizontal epipolar line coordinate system is as follows:

the rotation transformation matrix of the vector under the right camera coordinate system to the direction of the horizontal epipolar coordinate system is as follows:

wherein,

is the average of the Z-axis direction vector of the left camera coordinate system and the Z-axis direction vector of the right camera coordinate system,

，

is the Z-axis vector of the left camera coordinate system,

is the Z-axis vector of the right camera coordinate system,

for said rotation matrix

The third column of elements.

4. The method for tilt-corrected dense matching based on remote-sensing images as claimed in claim 3, wherein in said step S3, said step S3 is performed by

And

solving the corresponding coordinates of each pixel point in the original image in the horizontal epipolar line coordinate system to form a mapping between the position of each pixel point in the original image and the position of the corresponding point in the horizontal epipolar line image, and filling the color value of each pixel point in the original image into the corresponding point in the horizontal epipolar line image to generate a corrected horizontal epipolar line image,

the original image comprises a left inclined image under the left camera coordinate system and a right inclined image under the right camera coordinate system, the left inclined image generates a left horizontal epipolar line image after being corrected, and the right inclined image generates a right horizontal epipolar line image after being corrected.

5. The tilt-correction dense-matching method based on remote sensing images according to claim 4, wherein in the step S4, the step of obtaining the initial dense homonymy points comprises:

step S41, respectively calculating the initial parallaxes of the left horizontal epipolar line image and the right horizontal epipolar line image in the X axis and the Y axis of the horizontal epipolar line coordinate system;

step S42, determining a rough matching point of any dense point to be matched in the left horizontal epipolar line image in the right horizontal epipolar line image according to the initial parallax;

step S43, determining a one-dimensional search range in the right horizontal epipolar line image according to the position coordinate of the rough matching point as a search center and a set search radius, so as to determine a plurality of matching candidate points;

step S44, respectively calculating a normalized cross correlation coefficient between the gray average value of the source image block with the dense point to be matched as the center and the gray average value of the target image block with any one matching candidate point as the center;

and S45, determining the initial dense homonymous points of the dense points to be matched according to the normalized cross-correlation coefficient.

6. The tilt-corrected dense-matching method based on remote-sensing images according to claim 5, wherein in the step S41, the step of calculating the initial parallax of the left horizontal epipolar line image and the right horizontal epipolar line image comprises:

step S411, respectively calculating an X parallax and a Y parallax of each sparse homonymy point in the left horizontal epipolar line image and the right horizontal epipolar line image under the horizontal epipolar line coordinate system;

step S412, selecting three adjacent sparse homonymous point triangulation networks to form a plurality of parallax calculation areas, and calculating the X parallax d of any point in the parallax calculation areas in the single parallax calculation area in the X direction and the Y direction by adopting linear interpolation _x And Y parallax d _y ；

Step S413, when the dense point to be matched is located in the single parallax calculation areaThen, the X parallax d corresponding to the single parallax calculation region is adopted _x And Y parallax d _y 。

7. The tilt-corrected dense matching method based on remote sensing images according to claim 5, wherein in step S45, whether the normalized cross-correlation coefficient meets the standard or not is determined to determine whether an initial dense homonymous point exists at a single dense point to be matched, an initial matching correlation coefficient standard t and an initial matching correlation coefficient difference standard e are set, wherein t is greater than or equal to 0.7, and 0 < e is less than or equal to 0.01, the maximum value of the normalized cross-correlation coefficient between the gray-scale average value of the source image block centered on the dense point to be matched and the gray-scale average value of the target image block centered on any one of the matching candidate points is designated as Enccmax1, the next maximum value is designated as Enccmax2, and Δ Encc = Enccmax1-Enccmax2 are set,

when Enccmax1 is larger than or equal to t and delta Encc is larger than e, judging that the normalized cross-correlation coefficient accords with an initial matching standard, and setting a matching candidate point corresponding to Enccmax1 as an initial dense homonymous point of the point to be matched;

and when Enccmax1 is less than t or delta Encc is less than or equal to e, judging that the normalized cross-correlation coefficient does not accord with the initial matching standard, eliminating the points to be matched corresponding to Enccmax1 without the initial dense homonymy points.

8. The remote sensing image-based tilt-correction dense matching method according to claim 5, wherein in step S43, a search step is set to 2 pixels to determine the matching candidate points within the one-dimensional search range.

9. The tilt-correction dense-matching method based on remote-sensing images according to claim 5, wherein in the step S44, the gray-scale mean value is calculated in an incremental manner, and the incremental formula is as follows:

wherein, the target image block taking the mth matching candidate point as the center is recorded as the mth target image block,

is the gray scale mean value of the mth target image block,

is the gray level mean value of the (m + 1) th target image block,

is the gray scale average value of the pixels of the leftmost two columns in the mth target image block,

and the gray level average value of the pixels in the rightmost two columns in the (m + 1) th target image block is obtained.

10. The remote-sensing-image-based tilt-correction dense-matching method according to claim 8, wherein in the step S5, the position of each initial dense homonymous point is adjusted by using least square image matching to obtain two times dense homonymous points, the position coordinates of the sub-pixel-level dense homonymous points are determined according to a maximum value Encc2 in the normalized cross-correlation coefficient between the gray-scale mean value of each image block corresponding to each two times dense homonymous points and the gray-scale mean value of the source image block centered on the corresponding dense point to be matched, and a two times matching correlation coefficient standard tm is set, wherein tm is greater than or equal to 0.9,

when Encc2 is larger than or equal to tm, judging that the two-times normalized cross-correlation coefficient meets a sub-pixel level matching standard, and setting a two-times dense homonymy point corresponding to Encc2 as a sub-pixel level dense homonymy point of the dense point to be matched;

when Encc2 < tm, the two-times normalized cross-correlation coefficient is determined not to meet the sub-pixel level matching criterion.

Images