CN101509782A - Small-sized ground marker capturing and positioning method - Google Patents

Abstract

The invention discloses a small-scale landmark capture and positioning method, which belongs to the field of automatic target identification and navigation guidance positioning. The steps are as follows: 1) establishing the spatial constraint relationship between the small-scale landmark and the prominent features of the scene and a reference map of the prominent features; 2) Detect the salient features of the scene based on the acquired real-time image of the salient features; 3) Limit the matching search area of the real-time image of the salient features according to the spatial constraint relationship between the small landmarks and the salient features, and match the small landmarks in this matching search area , to get the rough matching points of small landmarks; 4) In the real-time image of salient features, determine the search area of interest for small landmarks centered on the rough matching points; 5) Perform precise positioning of small landmarks in the search area of interest. The invention accurately captures and positions small-sized targets in complex ground scenes, thereby accurately implementing real-time navigation and guidance position correction of aircraft.

Description

A Small Landmark Acquisition and Positioning Method

technical field

The invention belongs to the technical field of target recognition and navigation guidance and positioning, and in particular relates to a method for automatic capture and positioning of small landmarks.

Background technique

Continuously improving the navigation accuracy of aircraft has always been an important research topic in the field of aerospace technology. The accumulated error of the inertial navigation system gradually increases with time, so relying on the inertial navigation system alone cannot meet the requirements of high-precision navigation. The imaging system of the aircraft-borne optoelectronic platform can be used to obtain external reference information, and the landmarks can be captured and positioned, so as to correct the error of the aircraft navigation system and realize the precise navigation and positioning of the aircraft.

Small landmarks refer to feature-stable features whose landmark features occupy an image size within the range of 3×3 9×9 pixels. A small landmark is a widespread ground object, and its characteristics are generally not easy to change. If the potential of small landmarks for navigation and positioning can be tapped, the autonomy of the aircraft navigation system will be significantly improved, which is of great significance to the national economy and national security. Small landmarks not only refer to those stable ground objects with small geometric size, but also those ground objects with relatively large short-range size but relatively small size in long-distance imaging. Therefore, the capture and positioning of small landmarks is widely used. Small landmarks are also a kind of target. For small landmarks in complex backgrounds under moving platform conditions, there are many difficulties in identifying and positioning them: First, small landmarks account for a small proportion of the entire background image, and their features are not significant compared to the background image. ; secondly, the background is complex and constantly changing (such as the rising and falling of rivers, the disappearance or appearance of local features), and there are a large number of patterns similar to landmark images in the background; again, the moving platform itself is unstable, and the inertial navigation information of the aircraft has error. Therefore, the capture and positioning of small landmarks in complex backgrounds is a difficult problem.

Moon Y S, Zhang Tianxu, Zuo Zhengrong in "Detection of sea surface small targets in infrared images based on multi-level filter and minimum risk bayestest", International Journal of Pattern Recognition and Artificial Intelligence, 2000, 14-9: 1 A sea-surface infrared small target detection algorithm based on multi-stage filtering is proposed. The algorithm analyzes the background, small target and noise spectrum, and uses basic filter templates to form filter templates with different bandwidths to filter the infrared image, thereby detecting Target. The main problem of this method is: for small landmarks in complex ground background, due to the existence of a large number of similar patterns, the application of this method may cause a large number of false alarms.

Gianhca Marsiglia, Luea Fortunato, and Aurora Ondini proposed an infrared target recognition algorithm based on template matching in "Template matching techniques for automatic IR target recognition in real and simulated scenarios: tests and evaluations", Proceeding of SPIE 2003, 5094: 159-16. Methods The structure map or texture map of the target is used to make the target reference map, which is matched with the real-time map output by the sensor to identify the target, which is suitable for large-scale target recognition. The main problem with this method is: for small landmarks in complex backgrounds, the landmarks themselves have insignificant features compared to the surrounding complex scenes, so when matching with reference images, the matching recognition probability is low due to the existence of similar patterns .

Even for the identification and positioning of prominent feature parts, such as a certain intersection point of a bridge and a river, the traditional method is to use the matching based on the reference image, that is, first prepare a binary or ternary reference image on the ground based on the satellite image, and then It is stored in the computer of the moving platform, and then according to the inertial navigation information of the moving platform, the perspective transformation is performed on the reference image in the computer, and the transformed reference sub-image is matched with the real-time image of the photoelectric imaging sensor to capture the positioning feature. But this method is not reliable, because there may be multiple intersections of bridges and rivers in complex backgrounds, and there may be local regions similar to the intersections. In this way, when identifying and positioning, it is possible to locate these similar areas instead of the specified navigation anchor points. However, if the small target detection method is directly used to identify the positioning feature, because there are a large number of areas similar to the navigation positioning feature points in the complex background, there will be a large number of similar areas in the image, and the specified landmark point cannot be accurately located. . Therefore, traditional matching methods cannot meet the requirements of reliable navigation and positioning of aircraft.

Small targets in complex backgrounds account for a small proportion of the entire background, and their feature salience is weaker compared to the entire background. Traditional full-image detection or full-image matching is used to detect and locate small landmarks in complex backgrounds. Due to the existence of a large number of similar patterns in complex backgrounds, the accuracy of capture and positioning will be low, and the requirements for reliable and accurate positioning of aircraft navigation cannot be met.

Contents of the invention

The invention provides a method for capturing and positioning small landmarks, which accurately captures and positions small targets in complex ground scenes, thereby accurately implementing real-time navigation and guidance position correction for aircraft.

A small-scale landmark automatic capture and positioning method is carried out according to the following steps:

(1) Establish the spatial constraint relationship between small landmarks and salient features and the reference map of salient features;

(2) Detect the salient features of the scene based on the real-time images of the captured notable features;

(3) Limit the matching search area of the real-time image of the salient features according to the spatial constraint relationship between the small landmarks and the salient features, and match the small landmarks in this matching search area to obtain the rough matching points of the small landmarks;

(4) In the real-time image of salient features, determine the search area of interest for small landmarks centered on the coarse matching point;

(5) Precise positioning of small landmarks in the search area of interest.

The spatial constraint relationship includes the inclusion relationship, adjacent relationship and distance between small landmarks and prominent features.

The search region of interest in the step (4) is centered on the coarse matching point, and the length and width of the region are determined by m*(h/tanθ), h is the flight height, and θ is the pitch angle, and the value range of the constant m is [0 , 0.1].

Aiming at the defects of existing navigation positioning and matching positioning, the present invention proposes a method for automatic capture and positioning of small landmarks on a moving platform based on spatial constraint relationships according to the characteristics of small landmarks in complex backgrounds. The technical effects of this method are reflected in:

(1) The present invention pre-establishes a knowledge base of spatial constraint relationships between small landmarks and prominent features on the ground, and when locating small landmarks, first detects prominent features, and then further locates small landmarks according to the spatial constraint relationship. This method of recursive capture and positioning of small landmarks under moving platform conditions has good robustness and precision. This is because the correct detection rate of salient features is high, although the accuracy is not enough, but the matching reliability based on the reference map of salient features is high, and the constraint relationship between small landmarks and each salient feature is more than one, and it is reliable.

(2) On the ground, establish the geometric relationship between the landmarks and the prominent features in the scene based on the satellite images. For example, when there are airports, bridges, large waters and other features in the scene, establish the geometric relationship between these features and the landmarks, Get N constraints. For the real-time image during flight, the prominent ground objects in the image are sequentially detected. According to the N constraints, the matching search area for the real-time image of the salient ground objects is determined. In this way, the search area is no longer the whole image, but a local area limited by N constraints, which significantly reduces the impact of ambiguity and improves the matching accuracy.

(3) When accurately locating small landmarks, instead of performing precise locating on the whole image, feature template matching is performed on the region of interest of the coarse matching points in the reference image, which further reduces the impact of ambiguity, and at the same time Improve the detection accuracy.

Description of drawings

Fig. 1 is a schematic flow chart of the present invention;

Fig. 2 is a flow chart of strip region detection;

Fig. 3 is a strip detection template diagram, Fig. 3 (a) is a horizontal template, Fig. 3 (b) is a vertical template, Fig. 3 (c) is a 45-degree template, and Fig. 3 (d) is a 135-degree template;

Fig. 4 is a perspective transformation model diagram;

Figure 5 is a satellite image containing bridges, rivers, and land;

Figure 6 is a spatial constraint relationship diagram between small landmarks (bridges and river intersections) and rivers, bridges, and land;

Fig. 7 is a bridge strip reference diagram;

Figure 8 is a reference map including bridges, rivers, waterways, land and other prominent features;

Figure 9 is an example of a real-time map of the bridge area captured by the photoelectric imaging sensor on the moving platform;

Fig. 10 is the bridge region binary image extracted from this real-time image;

Fig. 11 is a Radon transformation reference point diagram of the straight section of the bridge;

Fig. 12 is a strip area map obtained by Radon transformation of the straight section of the bridge;

Figure 13 is a striped search area map for rough matching of prominent features;

Fig. 14 is the sub-reference image used for rough matching in the prominent feature reference image (perspective transformation has been carried out according to the flight inertial navigation parameters);

Fig. 15 is a rough matching positioning result diagram;

Fig. 16 is a region-of-interest diagram for searching landmarks;

Fig. 17 is a small-scale landmark (bridge/river intersection) feature template diagram;

Fig. 18 is a map of accurate matching and positioning results of bridge/river intersections in the region of interest;

Figure 19 is a satellite image containing small islands in the lake, lake shores, and strip features;

Figure 20 is a diagram of the spatial constraint relationship between small landmarks (small islands in the lake) and lake shores and strip features;

Fig. 21 is a reference map including small islands in the lake, lake shores, and strip features;

Fig. 22 is an example of a real-time map containing small islands in the lake obtained by the photoelectric imaging sensor on the moving platform;

Fig. 23 is a diagram of detection results of strip ground objects;

Figure 24 is a Radon transformation reference point map of strip features;

Fig. 25 is a straight line diagram after Radon transformation of strip features;

Fig. 26 is a schematic diagram of a strip-shaped matching search area for rough matching of prominent features;

Figure 27 is a schematic diagram of the template used for rough matching in the salient object reference map (according to the flight inertia

navigation parameter for perspective transformation);

Fig. 28 is a rough matching positioning result diagram;

Fig. 29 is a schematic diagram of a search area of interest for an island in a lake;

Figure 30 is a binary feature template diagram of a small landmark (island in a lake);

Figure 31 is a schematic diagram of the precise matching and positioning results of small islands in the lake in the ROI.

Detailed ways

Below in conjunction with accompanying drawing and two examples the present invention is described in further detail.

Example 1:

Taking the satellite image of bridges, rivers, and land in Figure 5 as an example, the landmark point is the circled part in the figure, that is, a certain intersection point between the bridge and the river. It can be seen that the intersection point has an obvious spatial constraint relationship with the bridge and the river. That is, it is both on the bridge strip and at the river bank.

The pre-flight processing steps are:

(1) Ground preparation

桥梁的两个端点坐标分别为

和

且地标点

所处的河流区域为R。建立的空间几何关系表达式为：On the ground, the spatial geometric relationship between the small landmarks and the salient features in the background is established according to the satellite images. The salient features mainly include three types: salient planar features, prominent line features and salient area features. In Figure 5, the bridge is a prominent line object, and the landmark point is a certain intersection point between the bridge and the river. The spatial geometric relationship between the landmark point and the bridge is established according to the satellite image, and the river is shown in Figure 6. The coordinates of the landmark points in Figure 6 are

The coordinates of the two endpoints of the bridge are

and

and the landmark

The river area where it is located is R. The established spatial geometric relationship expression is:

(x ₀ , y ₀ )∈l and (x ₀ ,y ₀ )∈R,

The straight line l where the bridge is located is determined by (x ₁ , y ₁ ) and (x ₂ , y ₂ )

Prepare a reference map of prominent line-like features. The preparation of the reference image is closely related to the category of the actual image and the type of the target, and different reference images are prepared according to different actual types of images and different targets. For optical images, binary segmentation is performed on visible light images containing landmarks, targets and surrounding environments, and a suitable threshold is manually found. The follow-up work is to remove black and white noise, and then adjust the corresponding edges and regions according to the original image. It makes the landmarks and target features that need to be matched more prominent, and removes the corresponding noise and interference as much as possible according to the characteristics of the optical imaging image.

According to the characteristics of satellite images including bridges, rivers, and land and infrared imaging sensors, the reference map of bridge strips prepared on the ground is shown in Figure 7, and the reference map of significant features including bridges, rivers, waterways, and land is shown in Figure 8 shown. At the same time, considering that the prominent feature is a line-like feature, in order to better determine the strip line in flight and facilitate the in-flight projection (Radon) transformation, Radon transformation points are selected on the ground. On the reference map, find two points a and b on the long strip of the bridge on the ground, and record the coordinates (x _a , y _a ) and (x _b , y _b ) of these two points on the reference map , and store it in the aircraft computer, as shown in Figure 11 (the circle part is two points a and b found on the ground).

The in-flight processing steps are:

(2) Detection of significant line objects

Figure 9 is an example of a real-time map of the bridge area captured by the photoelectric imaging sensor on the moving platform. Firstly, the bridge strips in the figure are detected, and the process of detecting bridge strip areas is shown in Figure 2.

The process of detecting the straight line of the bridge in Figure 9 is as follows: first, use 4 templates (as shown in Figure 3) to perform convolution operation on the real-time image, and 4 convolution result images can be obtained. For the 4 convolution result images, the maximum value of the 4 images is taken pixel by pixel, and a comprehensive convolution result image that integrates information in 4 directions and has the ability to resist certain rotations is obtained. Calculate the mean value μ and standard deviation δ of the comprehensive convolution result map, take the threshold G=μ+Kσ, and take K=2 as the threshold, and perform binary segmentation on the comprehensive convolution result map. Traverse the segmented binary image to obtain the connected regions in the image, record the pixels that constitute the same connected region in the form of a linked list, and assign different flag values to each connected region to obtain a series of regions of interest. For this series of regions of interest, calculate the average gray value of the pixels in the circumscribed rectangle of the area that are not on the area. If the average gray value is not much different from the average value of the water area, it is judged as a bridge. If there are multiple eligible bridge areas, the one with the longest length is taken as the bridge area. The resulting binary map of the bridge area is shown in Figure 10.

In order to ensure that the detected bridge area is a long strip, Radon transform is performed on Figure 10 . The expression of the two-dimensional Radon transform is:

$\mathrm{<span\; class="notranslate">\; R</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; \&rho;</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; \&theta;</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; D.</span>}}{<span\; class="notranslate">\; \&Integral;</span>}<span\; class="notranslate">\; \&Integral;</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span>\mathrm{<span\; class="notranslate">\; \&delta;</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; \&rho;</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; x</span>}\mathrm{<span\; class="notranslate">\; cos</span>}\mathrm{<span\; class="notranslate">\; \&theta;</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}\mathrm{<span\; class="notranslate">\; sin</span>}\mathrm{<span\; class="notranslate">\; \&theta;</span>}<span\; class="notranslate">\; )</span>\mathrm{<span\; class="notranslate">\; dxdy</span>}$

D is the Cartesian coordinate plane of the entire real-time image; f(x, y) is the gray value of the image point (x, y); ρ is the distance from the coordinate origin to the straight line ρ=xcosθ+ysinθ; θ is the straight line ρ=xcosθ+ysinθ Angle with the x-axis. It makes f(x, y) integrate along the straight line ρ=xcosθ+ysinθ, the Radon transform can be understood as the projection of the image in the ρ-θ space, each point in the ρ-θ space corresponds to a straight line in the image space, and the Radon transform is the image Integral of pixels on each line. According to different combinations of ρ and θ, perform Radon transformation on the image to find the maximum value, and the corresponding ρ and θ are the parameters of the straight line that need to be found.

When performing Radon transformation, not all angles of θ (0～2π) are transformed, but only within a certain angle range. Considering the error of the inertial measurement combination, the angle range is measured by the inertial navigation of the moving platform Combination devices are provided so that both speed and accuracy can be considered.

为成像仪横向成像角，前视图像行数为ROW，图像列数为COL，α为方位角，θ为俯仰角，h为飞行高度。如图4所示：T₀(x₀，y₀)为光轴所在点，T₁(x₁，y₁)为地面的某个点，则在光电传感器获取的前视图像中T₀的像素点位置为(COL/2，ROW/2)，设T₁在光电传感器获取的前视图像中的像素点位置为(T₁_COL，T₁_ROW)，则计算T₁_COL和T₁_ROW的过程如下：According to the navigation information of the aircraft (the heading angle α of the seeker, the pitch angle θ and the height h), the perspective transformation of a and b in the reference image of prominent line-like objects shown in Fig. The coordinates of pixel points a' and b' in the real-time front view are (x _a' , y _a' ) and x _b' , y _b' ), respectively. The perspective transformation model is shown in Figure 4, where: φ is the longitudinal imaging field angle of the imager,

is the lateral imaging angle of the imager, the number of front-view image rows is ROW, the number of image columns is COL, α is the azimuth angle, θ is the pitch angle, and h is the flight height. As shown in Figure ₄ : T ₀ (x ₀ , y ₀ ) is the point where the optical axis is located, and T ₁ (x ₁ , y ₁ ) is a certain point on the ground. The pixel point position is (COL/2, ROW/2), and the pixel point position of T ₁ in the front-view image acquired by the photoelectric sensor is (T ₁ _COL, T ₁ _ROW), then calculate T ₁ _COL and T ₁ _ROW The process is as follows:

OT ₀ =h/tanθ

$\mathrm{<span\; class="notranslate">\; om</span>}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; OT</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; +</span>\sqrt{{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; cos</span>}\mathrm{<span\; class="notranslate">\; \&alpha;</span>}$

tan(∠OMP)＝h/OM

T ₁ _ROW＝(∠OMP-θ)*ROW/φ

Among them, OT ₀ is the distance between the intersection point T ₀ of the optical axis and the earth and the projection of the imager to the point O of the earth, point M is the intersection point between the projection of point T ₁ to the longitudinal direction of the optical axis and the straight line OT ₀ , and OM is point O to point M distance.

For two known points (x _a , ya ) and (x _b , _{y b )} in the reference image, their positions on the real-time image captured by the photoelectric imaging sensor are x _a′ , y _a′ ) and x _{b ′} , y _b′ ), then the perspective transformation model can be used to calculate x _a′ , y _{a′ ,} x _b′ , and y _b ′ according to the azimuth α, pitch angle θ, and height h of the navigation system as follows (the coordinates are respectively Substitute into the perspective transformation model formula):

${\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; arctan</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; h</span>}<span\; class="notranslate">\; /</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; h</span>}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; the\; tan</span>}\mathrm{<span\; class="notranslate">\; \&theta;</span>}<span\; class="notranslate">\; +</span>\sqrt{{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; a</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; a</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; cos</span>}\mathrm{<span\; class="notranslate">\; \&alpha;</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; \&theta;</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; ROW</span>}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; \&phi;</span>}$

${\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; arctan</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; h</span>}<span\; class="notranslate">\; /</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; h</span>}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; the\; tan</span>}\mathrm{<span\; class="notranslate">\; \&theta;</span>}<span\; class="notranslate">\; +</span>\sqrt{{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; b</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; b</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; cos</span>}\mathrm{<span\; class="notranslate">\; \&alpha;</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; \&theta;</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; ROW</span>}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; \&phi;</span>}$

Knowing two points, a straight line can be determined and the slope of the bar in the real-time image can be calculated $\mathrm{\&sigma;}=\frac{{\mathrm{the\; y}}_{b\&prime;}-{\mathrm{the\; y}}_{a\&prime;}}{x_{b\&prime;}-x_{a\&prime;}},$ σ is the straight line parameter of the bridge strip when the inertial navigation information of the aircraft is accurate, that is to say, if the inertial navigation information of the aircraft is accurate, then ω is the slope of the straight line in the real-time image. However, since there are still errors in the inertial navigation information of the aircraft, the error is defined as k=ΔαΔθΔh, where Δα, Δθ, and Δh are the errors of heading angle α, pitch angle θ, and height h, respectively. Therefore, Radon transformation is required to determine the straight line parameters in the real-time image, and the angle of Radon transformation should change within a range centered on σ (converted into radian form) [σ-ck, σ+ck], where c is a constant , the value range of c is [0, 100], which is 20 in this embodiment, and the step size of Radon transform is set to 2 in order to take into account both the accuracy and the operation speed. The straight line equation obtained after Radon transformation is:

ρ＝xcosω+ysinω

All the white pixels in Fig. 10 are processed as follows: For all the points whose gray value is equal to 255, calculate the distance from these points to the straight line ρ=xcosω+ysinω, if the distance is greater than Δd (for example, Δd is 3), then the The gray value of the point whose distance is greater than Δd is set to 0, otherwise it remains unchanged. The strip area obtained by the Radon transformation result is shown in Figure 12, and the smallest rectangle containing this area is the matching search area, as shown in Figure 13.

(3) Matching of salient object reference maps along the direction of the line area

According to the spatial constraint relationship between small landmarks and prominent features, the matching search area of the real-time image of prominent features is limited, and the small landmarks are roughly positioned in this matching search area to obtain the rough matching point of the small landmark, and the coarse matching point is the center , to determine a local region with an appropriate size, and detect landmark features in the local region for fine positioning.

For the real-time image, see Fig. 9, the attitude parameters of the seeker are: pitch angle θ, heading angle α, height h. According to the attitude parameters, the perspective transformation is carried out on the reference image of prominent line objects, and the front-view transformation image of the reference image is obtained. Select a square matching template in the transformed salient feature reference map, the center of the matching template is a small landmark, the length and width of the template are determined by θhμ, where θ is the pitch angle, h is the height, μ is a constant, and the value of μ The range is [0, 0.01], which is taken as 0.005 in this embodiment. Match the selected matching template with the real-time image along the white strip area in Figure 13.

The submap template of the selected reference map of salient features is shown in Figure 14. According to the spatial constraint relationship between the small landmarks and the surrounding prominent features obtained in the ground preparation stage, it can be known that the small landmarks to be detected are located at a certain intersection of the bridge and the river. It is difficult to detect small landmarks directly by full-image detection or full-image matching, because there are many intersections between bridges and rivers, and the features of the small landmarks are not significant. However, according to the spatial constraint relationship between the small landmarks and the surrounding prominent features obtained in the ground preparation stage, the prominent bridge features can be detected first, and then the reference image matching for the small landmarks can be carried out along the direction of the bridge strip, which not only reduces the The amount of calculation is reduced, and this will significantly improve the detection accuracy. The bridge strip area has been detected and located in the previous step. It can be known that the small landmark point is on this strip area, but the specific position of the small landmark on the strip is not yet known, so according to the spatial constraint relationship (the small landmark is on the bridge Above), along the direction of the strip, match the reference map of prominent features containing small landmarks, and the search area for matching is shown in the white strip area in Figure 13. If it is a full-image search, the detection will be inaccurate due to the existence of a large number of areas similar to small landmarks in the image, and the matching based on the spatial constraint relationship between small landmarks and surrounding prominent features will limit the scope of the matching search area , thereby improving the detection accuracy.

The metric used for matching is de-mean normalized gray-scale cross-correlation, and the definition of the de-mean normalized gray-scale cross-correlation matching algorithm is as follows: Let the real-time image be G _r , whose size is M _r ×N _r , and the reference image be G _s , its size is M _s ×N _s , and M _s <M _r , N _s <N _r . _Then the de-mean _normalized cross-correlation measure _ρ ( _u , v) is:

$\mathrm{<span\; class="notranslate">\; \&rho;</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; the\; s</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; the\; s</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}<span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; G</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{{\mathrm{<span\; class="notranslate">\; G</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; \&times;</span><span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; G</span>}}_{\mathrm{<span\; class="notranslate">\; the\; s</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{{\mathrm{<span\; class="notranslate">\; G</span>}}_{\mathrm{<span\; class="notranslate">\; the\; s</span>}}}<span\; class="notranslate">\; ]</span>}{\sqrt{\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; the\; s</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; the\; s</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{<span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; G</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{{\mathrm{<span\; class="notranslate">\; G</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; ]</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}\sqrt{\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; the\; s</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; the\; s</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{<span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; G</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{{\mathrm{<span\; class="notranslate">\; G</span>}}_{\mathrm{<span\; class="notranslate">\; the\; s</span>}}}<span\; class="notranslate">\; ]</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}}$

与

分别为G_r(u，v)与G_s的灰度均值。由ρ(u，v)构成的相关系数矩阵就是相关面。再从计算出的匹配相关面数据ρ(u，v)中选取极值点得到匹配定位点，互相关匹配算法的运算量主要集中在相关面数据ρ(u，v)的计算上。in

and

are the gray mean values of G _r (u, v) and G _s respectively. The correlation coefficient matrix composed of ρ(u, v) is the correlation surface. Then select the extreme points from the calculated matching related surface data ρ(u, v) to obtain the matching positioning point, and the calculation amount of the cross-correlation matching algorithm is mainly concentrated on the calculation of the related surface data ρ(u, v).

The matching results are shown in Figure 15.

In this way, the position of the small landmark is roughly obtained, but it is not accurate yet. With the rough matching point as the center, select a local area (the size of the area is determined by m(h/tanθ), h is the flight height, θ is the pitch angle, m is a constant, and the value range is [0, 0.1], which is 0.01 in this embodiment), the local area is the area of interest (potential location area of small landmarks), as shown in Figure 16.

(4) Detect and pinpoint small landmarks

After the previous steps, the region of interest where the small landmarks are located has been determined, but the specific location of the small landmarks in the region of interest cannot be precisely located. Therefore, precise positioning of small landmarks in this ROI is also required. The method adopted in this embodiment is to perform perspective transformation in the previous step to transform the perspective transformation of the reference map of prominent features into the front view of the reference map of prominent features. In the front view of the salient object reference map, a small feature template is selected centering on the small landmark point, and the template is used to perform feature matching with the ROI of the real-time map to obtain the precise position of the small landmark. The size of the selected small template is determined according to θhs, where θ is the pitch angle, h is the height, s is a constant, and the value range of s is [0, 0.005], which is 0.0025 in this embodiment. An example of the selected landmark reference template is shown in Figure 17, and the results of landmark detection and positioning are shown in Figure 18.

Example 2:

Take the satellite image of a small island in a lake as shown in Figure 19 as an example. The landmark point is the small island contained in the circle in the picture. The landmark small island is included in the lake. You can see that there is an obvious strip feature next to the small island , there is a significant spatial constraint relationship between the small island in the lake, the features of the strip and the shore of the lake.

The pre-flight processing steps are:

(1) Ground preparation

On the ground, the spatial geometric relationship between the small landmarks and the salient features in the scene is established based on the satellite images. The salient features mainly include three types: salient plane-like features, prominent line-like features and salient area-like features. In Figure 19, there is a bridge near the island, and there is an obvious geometric relationship between the island and the lake shore. Figure 20 shows the spatial constraint relationship of the island.

According to the characteristics of satellite images and optical images of small islands in the lake, the binary reference map of salient ground features prepared on the ground is shown in Figure 21. At the same time, considering that the prominent feature is a line-like feature, in order to better determine the strip line in flight and facilitate the Radon transformation in flight, the Radon transformation point is selected on the ground. On the reference map, find two points α and b on the long strip of the bridge on the ground, and record the coordinates (x _a , y _a ) and (x _b , y _b ) of these two points on the reference map , and store it in the aircraft computer, as shown in Figure 24.

The in-flight processing steps are:

(2) Detection of significant line objects

Figure 22 is the real-time image of the small island in the lake captured by the photoelectric imaging sensor. The strip-shaped bridge near the small island has obvious features, so it is detected and positioned.

First, use 4 templates (as shown in Figure 3) to perform convolution operations on the image, and 4 convolution result images can be obtained. For the 4 convolution result images, the maximum value of the 4 images is taken pixel by pixel, and a comprehensive convolution result image that integrates information in 4 directions and has the ability to resist certain rotations is obtained. Calculate the mean value μ and standard deviation δ of the comprehensive convolution result map, take the threshold G=μ+Kσ, and take K=2 as the threshold, and perform binary segmentation on the comprehensive convolution result map. Traverse the segmented binary image to obtain the connected regions in the image, record the pixels that constitute the same connected region in the form of a linked list, and assign different flag values to each connected region to obtain a series of regions of interest. For this series of regions of interest, calculate the gray mean value of the pixels in the circumscribed rectangle of the region that are not on the region. If the gray value mean value is not much different from the water area mean value, it is judged as a band. If there are multiple eligible strip areas, the one with the longest length is taken as the strip area. The resulting striped area is shown in Figure 23.

In order to obtain the straight line equation, Radon transform is performed on Fig. 23 . According to the real-time parameters (pitch angle θ, heading angle α, height h), the points a and b in Figure 23 are respectively transformed to obtain the coordinate positions (x _a′ , y _a′ ) and (x a′ ) of the corresponding points in the real-time graph _b' , y _b' ). pass $\mathrm{\&sigma;}=\frac{{\mathrm{the\; y}}_{b\&prime;}-{\mathrm{the\; y}}_{a\&prime;}}{x_{b\&prime;}-x_{a\&prime;}}$ Calculate the slope of the strip feature in the real-time image, and σ is the strip feature parameter under the condition that the aircraft navigation information is accurate. However, due to the error in the navigation information of the aircraft, the error is defined as k=ΔαΔθΔh, Δα, Δθ, and Δh are the errors of the heading angle, pitch angle, and altitude respectively. Therefore, the angle of the Radon transformation should change within a range [σ-ck, σ+ck] centered on σ (converted into radian form), wherein c is a constant, which is taken as 30 in this embodiment, in order to take into account the accuracy and Operation speed, the step size of Radon transform is set to 2. Perform Radon transformation on the white pixels in the picture

$\mathrm{<span\; class="notranslate">\; R</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; \&rho;</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; \&theta;</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; D.</span>}}{<span\; class="notranslate">\; \&Integral;</span>}<span\; class="notranslate">\; \&Integral;</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span>\mathrm{<span\; class="notranslate">\; \&delta;</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; \&rho;</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; x</span>}\mathrm{<span\; class="notranslate">\; cos</span>}\mathrm{<span\; class="notranslate">\; \&theta;</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}\mathrm{<span\; class="notranslate">\; sin</span>}\mathrm{<span\; class="notranslate">\; \&theta;</span>}<span\; class="notranslate">\; )</span>\mathrm{<span\; class="notranslate">\; dxdy</span>}$

After transformation, the polar coordinate expression of the straight line is obtained:

ρ=xcosθ+ysinθ

The straight line r obtained after Radon transformation is shown in Figure 25.

(3) Matching of salient object reference maps along the direction of the line area

According to the flight parameters (pitch angle θ, heading angle α, height h), the perspective transformation of the reference image is carried out to obtain the real-time forward-looking transformation image of the reference image. Select a reference subgraph of a certain size (the selected size is determined by θhμ, where θ is the pitch angle, h is the flight height, μ is a constant, and μ is taken as 0.005 in this embodiment), and the selected reference subgraph is shown in Figure 27.

According to the spatial constraint relationship between the small islands in the lake and the prominent strip features, it can be known that there is a certain geometric relationship between the small islands in the lake and the prominent strip features. There is a certain distance with ground objects. In the spatial constraint relationship between the small island in the lake and the prominent features shown in Figure 20, the distance l between the small island in the lake and the strip bridge feature is subjected to perspective transformation to obtain l′ in the real-time image. Therefore, it can be known that the small island in the lake in the real-time map should be in the strip area whose distance from the prominent strip feature does not exceed l′. Then define the search area for matching the reference image of the small island in the lake as:

D _xr ≤ l′

Among them, D _xr represents the distance between the pixel point x in the real-time image and the straight line r obtained by Radon transformation. In this way, it is not a full-image search, but the spatial constraint relationship between small islands in the lake and prominent line features is used, which greatly limits the matching search area and improves the detection accuracy. Figure 26 is a matching search area.

The matching method used in this embodiment is the de-mean normalized gray-scale cross-correlation algorithm:

$\mathrm{<span\; class="notranslate">\; \&rho;</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; the\; s</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; the\; s</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}<span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; G</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{{\mathrm{<span\; class="notranslate">\; G</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; \&times;</span><span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; G</span>}}_{\mathrm{<span\; class="notranslate">\; the\; s</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{{\mathrm{<span\; class="notranslate">\; G</span>}}_{\mathrm{<span\; class="notranslate">\; the\; s</span>}}}<span\; class="notranslate">\; ]</span>}{\sqrt{\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; the\; s</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; the\; s</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{<span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; G</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{{\mathrm{<span\; class="notranslate">\; G</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; ]</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}\sqrt{\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; the\; s</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; the\; s</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{<span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; G</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{{\mathrm{<span\; class="notranslate">\; G</span>}}_{\mathrm{<span\; class="notranslate">\; the\; s</span>}}}<span\; class="notranslate">\; ]</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}}$

Figure 28 shows the result after matching the reference image of the small island in the lake.

In this way, the position of the small island in the lake is roughly obtained, but it is not accurate yet. With the matching point as the center, select a local area (the size of the area is determined by m(h/tanθ), h is the flight height, θ is the pitch angle, m is a constant, in this embodiment, m is taken as 0.01), and this local area is the area of interest (the potential location area of the small island in the lake), as shown in Figure 29.

(4) Detection of small landmarks

After the previous steps, the area of interest of the small island in the lake has been determined, but the specific location of the small island in the lake in the area of interest cannot be precisely located. Therefore, it is also necessary to accurately locate the small island in the lake in this area of interest. The method adopted in this embodiment is to perform perspective transformation in the previous step to transform the perspective transformation of the reference map of prominent features into the front view of the reference map of prominent features. In the front view of the salient feature reference map, select a small template centered on the small landmark point (island in the lake), and use this template to match the ROI of the real-time map to obtain the precise position of the small landmark. The size of the selected small template is determined according to θhs, where θ is the pitch angle, h is the height, and the value range of the constant s is [0, 0.005], which is 0.0025 in this embodiment. An example of the selected landmark reference template is shown in Figure 30, and the results of landmark detection and positioning are shown in Figure 31.

Claims (5)

1. A small-scale landmark automatic capture and positioning method is carried out according to the following steps: (1) Establish the spatial constraint relationship between small landmarks and salient features and the reference map of salient features; (2) Detect the salient features of the scene based on the real-time images of the captured notable features; (3) Limit the matching search area of the real-time image of the salient features according to the spatial constraint relationship between the small landmarks and the salient features, and match the small landmarks in this matching search area to obtain the rough matching points of the small landmarks; (4) In the real-time image of salient features, determine the search area of interest for small landmarks centered on the coarse matching point; (5) Precise positioning of small landmarks in the search area of interest. 2. The method for automatic capture and positioning of small landmarks according to claim 1, wherein the spatial constraint relationship includes the inclusion relationship, adjacent relationship and distance between small landmarks and prominent features. 3. A method for automatically capturing and locating small landmarks according to claim 1, wherein the search area of interest in the step (4) is centered on the coarse matching point, and the length and width of the area are determined by m*(h/ tanθ), h is the flight height, θ is the pitch angle, and the value range of the constant m is [0, 0.1]. 4. A method for automatically capturing and locating small landmarks according to claim 1, 2 or 3, characterized in that, in the step (3), before matching the small landmarks, first determine the reference map of prominent features in the following manner The first matching template: the front view is obtained by performing perspective transformation on the reference image of prominent features, and the first matching template is selected in the front view image. The first matching template is centered on the position of the small landmark, and the length and width of the template are determined by θ*h *μ is determined, where θ is the pitch angle, h is the flight height, and the value range of the constant μ is [0, 0.01]. 5. The method for automatic capture and positioning of small landmarks according to claim 1, 2 or 3, characterized in that, in the step (5), before precise positioning, first determine the second part of the reference map of prominent features in the following manner: Matching template: Perform perspective transformation on the reference image of prominent features to obtain the front view, select the second matching template in the front view image, the second matching template is centered on the position of the small landmark, and the length and width of the template are determined by θ*h*s , where θ is the pitch angle, h is the flight height, and the value range of the constant s is [0, 0.005].

Images