CN105809700B - A kind of drogue image detection localization method blocked by oily plug - Google Patents

Abstract

The invention discloses a kind of drogue image detection localization method blocked by oily plug, belong to the technical field of Computer Image Processing.The present invention quickly rejects the interference such as umbrella frame, the hose on internal black circle oil filler periphery in tapered sleeve image using Mathematical Morphology Method, then judge to obtain the image-region where the oil filler of tapered sleeve inside with rejectings, subcircular shape condition judgment etc. by contours extract, occlusion area registration, calculating process is simple, can quickly detect tapered sleeve picture position and region of the positioning under by oily plug circumstance of occlusion.

Description

A Method for Image Detection and Positioning of Aerial Refueling Drogue Blocked by Oil Plug

technical field

The invention discloses an image detection and positioning method of an air refueling drogue sleeve blocked by an oil plug, and belongs to the technical field of computer image processing.

Background technique

Most of the existing visual navigation methods for autonomous aerial refueling often use special optical markings installed on the refueling drogue, including LED cursors, artificial special color markings, etc. For example, a set of VisNav system is used in the U.S. autonomous aerial refueling project (AAR). This VisNav system needs to install multiple LED cursors on the refueling drogue, and then use the semiconductor position detector PSD for image positioning (see Dong Xinmin, Xu Yuejian , Chen Bo, "Research Progress and Key Issues of Automatic Aerial Refueling Technology", "Journal of Air Force Engineering University (Natural Science Edition), 2008(12), 9(6): 1-5). In addition, Wang Xufeng of Air Force Engineering University in China established a visual relative navigation semi-physical ground test platform for autonomous aerial refueling, and added a red marking ring to the section area of the refueling drogue sleeve to enhance the contrast between the feature area and the background image. Thereby, image positioning information can be obtained by color recognition (see Wang Xufeng, Dong Xinmin, Kong Xingwei, "Autonomous Aerial Refueling Simulation of Plug Drogue UAV Assisted by Machine Vision", "Science Technology and Engineering", 2013 (6), 13 ( 18): 5245-5250).

These methods above require the refueling drogue to be equipped with additional optical markers, especially LED cursors that require power, which increases the risk of aerial refueling operations. Therefore, an AAR vision method that does not rely on additional markings but only uses the characteristics of the refueling drogue has better versatility, convenience and safety.

Through the analysis of the shape characteristics of the refueling drogue sleeve, it can be known that the internal refueling port is circular with a radius of about 13cm. Shape features can reduce the computational load of image localization processing. With the development of UAVs, the demand for autonomous aerial refueling is becoming more and more urgent. The visual navigation method for autonomous aerial refueling without additional optical markers has high versatility and convenience, especially based on the internal refueling drogue. The computer vision method of the circular fuel filler has obvious characteristics and fast image processing speed. However, the camera is often installed behind the side of the oil receiving plug, and the oil receiving plug as the foreground object of the image will block the refueling drogue sleeve, so as to solve the problem of interference caused by the oil receiving plug. The image detection and positioning of the refueling drogue is the key to the visual navigation of autonomous aerial refueling. This scheme is based on the aforementioned ideas.

Contents of the invention

The technical problem to be solved by the present invention is to overcome the disadvantages of installing additional optical markers in the existing autonomous aerial refueling method, and the higher risk of aerial refueling brought by additional charging devices, etc., from only relying on the characteristics of the refueling drogue, From the perspective of the oil plug image occlusion problem in visual navigation, a method for detecting and locating the aerial refueling drogue image covered by the oil plug is provided. This method uses the mathematical morphology method to quickly eliminate the inner black circular refueling port in the drogue image The surrounding umbrella ribs, hoses, etc. interfere, and then obtain the image area where the oil filler port inside the drogue sleeve is located through contour extraction, judgment and elimination of the coincidence degree of the occluded area, and near-circular shape condition judgment.

The present invention adopts following technical scheme for realizing above-mentioned purpose of the invention:

A method for image detection and positioning of an aerial refueling drogue covered by an oil plug, comprising the following steps:

A. Perform reverse binarization processing on the mask image blocked by the oil plug, perform mathematical morphology closing operation on the mask image after reverse binarization processing to obtain the first binary mask image, and perform the first binary mask image on the first two performing a mathematical morphology erosion operation on the value mask image to obtain a second binary mask image;

B. Carry out inverse binarization processing on the grayscale image of the drogue sleeve to be detected;

C. Perform an OR operation on the grayscale image of the drogue sleeve to be detected after the reverse binarization process and the mask image blocked by the oil plug, and perform a mathematical morphological opening operation on the binary image obtained by the OR operation to obtain a binary image ;

D, carry out contour extraction to the binary image that step C obtains and obtain peripheral contour, the set of points on each peripheral contour forms peripheral contour edge point set;

E. Arrange the areas surrounded by each contour in descending order, filter out the peripheral contours that are smaller than the contour area threshold, and remove the set of points on the filtered peripheral contours from the peripheral contour edge points;

F. Judgment of the overlapping degree of the occlusion mask on the peripheral contour left by the screening, enter step F1 when it does not overlap, and enter step F2 when it overlaps,

F1. Directly perform near-circle target detection on the peripheral contour, including area range condition constraints, circularity condition constraints, and least squares ellipse fitting condition constraints, and retain target contours that meet the near-circle target detection requirements.

F2. Perform occlusion and near-circle target detection on the peripheral contour, and retain the target contour that meets the occlusion and near-circle target detection requirements: first remove the occlusion interference in the peripheral contour, and then carry out the restriction of the area range condition on the peripheral contour after removing the occlusion interference. Near-circle target detection constrained by square ellipse fitting conditions;

G. Record the ellipse fitting parameters of each candidate target contour and calculate the logarithmic circularity of each fitted ellipse, and select the candidate target contour with the largest logarithmic circularity as the final image positioning result of the oil filler port inside the drogue sleeve.

As a further optimization scheme of the aerial refueling drogue image detection and positioning method covered by the oil plug, step B adopts the expression: Inverse binarization processing is performed on the grayscale image of the drogue sleeve to be detected,

Among them, (x, y) represents the pixel coordinates of the image, B ₀ (x, y) represents the gray value of the pixel at the position (x, y) of the grayscale image of the cone sleeve to be detected after reverse binarization processing, and D _origin (x, y) represents the pixel gray value of the drogue gray image to be detected at the position (x, y), and T ₁ represents the threshold value of the inverse binarization processing of the drogue gray image to be detected.

Further, in the above-mentioned method for image detection and positioning of an aerial refueling drogue covered by an oil plug, in step F, the outer contours left by the screening are judged on the coincidence degree of the occlusion mask, and the specific method is as follows:

Perform an AND operation on the extracted binary image of the current peripheral contour area and the mask image processed by inverse binarization to extract the sub-image of interest,

Count the number of pixels with the same pixel value in the occlusion mask area in the sub-image:

When the number of counted pixels exceeds the set threshold, it is determined that the current peripheral contour area overlaps with the occlusion mask area,

When the counted number of pixels is less than or equal to the set threshold, it is determined that the current peripheral contour area does not overlap with the occlusion mask area.

Still further, in the described method for image detection and positioning of an aerial refueling drogue covered by an oil plug, in step F2, the blocking interference in the peripheral contour is removed, and the specific method is as follows:

Perform an AND operation on the extracted binary image of the peripheral contour area and the first binary mask image, perform a mathematical morphology opening operation on the image after the AND operation, and extract the contour of the binary image after the opening operation to obtain the points of the current peripheral contour set, eliminating the points that coincide with the second binary mask image in the current peripheral contour point set.

Furthermore, in the step F2 of the image detection and positioning method of the aerial refueling drogue covered by the oil plug,

The range condition of the area is: (x _d >T ₃ )&(y _d >T ₃ )&(y _d >T ₄₁ ·x _d )&(x _d >T ₄₁ ·y _d ),

The least squares ellipse fitting condition is: {2a>T ₃ }&{2b>T ₃ }&{a>T ₄₁ ·b}&{b>T ₄₁ ·a} and in, (x _k1 ,y _k1 ), (x _k2 ,y _k2 ),..., are the pixel coordinates of the 1st, 2nd, ..., n _kth points on the k-th candidate target contour respectively, T ₃ is the area range threshold, T ₄₁ is the area ratio threshold, & represents the logical AND operation, (a, b) is the semi-major and minor axes of the fitted ellipse, for collection The mean and standard deviation of T ₆ is the threshold of the mean value of the fitting error, T ₇ is the threshold of the standard deviation of the fitting error,

(x _c , y _c ) are the coordinates of the center point of the fitted ellipse, and θ is the rotation angle of the fitted ellipse.

As a further optimization scheme of the aerial refueling drogue image detection and positioning method covered by the oil plug, the expression for calculating the logarithmic circularity f(a, b) of the fitted ellipse in step G is:

The present invention adopts the above-mentioned technical scheme, and has the following beneficial effects:

(1) There is no need to install additional optical marks on the refueling drogue, and only use the characteristics of the refueling drogue, which has better versatility, convenience and safety;

(2) This method is aimed at the detection of the refueling drogue sleeve image under the condition of being blocked by the oil plug. The method of contour extraction after binarization is used to quickly determine the target area to be detected, and the mask is eliminated for the contour that overlaps with the mask area. Eliminate the image interference caused by the occlusion of the oil plug, and finally judge whether it is the contour of the target through the contour shape. This method has a clear idea and a simple calculation process. It can quickly detect and locate the image position and area of the drogue sleeve under the occlusion of the oil plug.

Description of drawings

Fig. 1 is the calculation flowchart of the inventive method;

Fig. 2 is the collected original cone sleeve color image;

Figure 3 is the mask image of the reverse oil-receiving plug;

Fig. 4 is the binarized image of the cone sleeve and the mask image or the binary image after the operation;

Fig. 5 is the binary image of the cone sleeve after the morphology opening operation;

Fig. 6 is the contour extraction result after removing the small area contour;

Fig. 7 is the binary image of the contour area overlapping with the mask area;

Fig. 8 is the contour extraction result after removing the mask area from the binary image in Fig. 7;

Fig. 9 is the result after removing the contour points coincident with the mask area from the contour in Fig. 8;

Figure 10 shows the final image detection and positioning results of the oil filler port inside the drogue sleeve.

Detailed ways

The technical solution of the invention will be described in detail below in conjunction with the accompanying drawings.

A method for detecting and locating aerial refueling drogue images that are blocked by oil plugs, using only the inner circular refueling opening feature of the refueling drogue itself, and quickly removing the inner black circle in the drogue image by performing mathematical morphology operations on the binary image Interference around the refueling port, and then on the basis of contour extraction, use the oil plug mask image to judge and remove the occluded part of the contour, and use the near-circular shape condition to quickly obtain the candidate near-circular contour, and finally select the logarithmic circle The candidate solution with the highest degree is used as the result of drogue sleeve image detection and positioning. Specifically include the following steps.

Step 1. Prepare the mask image covered by the oil plug: in the mask image M covered by the oil plug, the pixel value of the area covered by the oil plug is set to 0, and the other areas are set to 1; calculate the reverse mask image, that is, the block The pixel value of the area is 1, and the other area is 0. The mask image after inverse binarization processing is denoted as M _i ; the mask image M blocked by the oil plug using a disc-shaped structural element D _r with a radius of r ₁ Perform a mathematical morphology closing operation to obtain the first binary mask image M _c ; use a disc-shaped structural element D _r with a radius r ₁ to perform a mathematical morphology erosion operation on the image M _c to obtain the second binary mask image M _e , r ₁ ranges from 3 to 12, and using a disc-shaped structural element D _r with a radius of r ₁ to perform mathematical morphological operations on the image can expand the area.

Step 2. Load the grayscale image D _origin of the drogue to be detected, and use the threshold T ₁ to inversely binarize the image D _origin to obtain the grayscale image B ₀ of the drogue to be detected after reverse binarization. The specific conversion process is as follows:

In the formula, T ₁ represents the threshold value of the inverse binarization processing of the grayscale image of the drogue sleeve to be detected, (x, y) represents the image pixel coordinates, D _origin (x, y) and B ₀ (x, y) respectively represent the The pixel gray value at position (x, y) of the detected drogue gray image D _origin and the reverse binarized drogue gray image B ₀ to be detected.

Step 3. Perform OR operation on the grayscale image B ₀ of the drogue sleeve to be detected after inverse binarization processing and the mask image M blocked by the oil plug to obtain a binary image B ₁ in the shape of a disk with a radius of r ₁ The structural element D _r performs the mathematical morphology opening operation on the binary image B ₁ to obtain the binary image B ₂ .

Step 4. Perform contour extraction operation on the binary image _B2 to obtain the peripheral contour edge point set Among them, N ₁ represents the number of contours, C _k (k=1,2,...,N ₁ ) represents the set of points on each contour, which can be expressed as where n _k represents the number of contour edge points.

Step 5. Count the area of each contour in the contour edge point set C, record the area of the contour C _k as s _k , and pair the sequence Arrange in descending order, and eliminate the contours whose area is smaller than T ₂ (contour area threshold), the selection range of T ₂ is 60 to 200, and finally obtain the contour numbers from large to small areas, recorded as Satisfies the following formula:

Step 6. Set the flag f _mask =0, and sequentially perform the following processing on each contour in the numbered set d={d ₁ ,d ₂ ,...,d _m }:

For the contours currently numbered d _j =k,j=1,2,...,m,

If f _mask = 0, then go to step 7 to judge the coincidence degree of the occlusion mask for the current contour, if the judgment result is coincidence, then set f _mask = 1, and go to step 8 to detect the occluded near-circle target and record the detection result ; If the judging result is non-overlapping, proceed to step 9, directly perform the near-circle target detection operation on the current contour, record the detection result,

If f _mask = 1, go directly to step 9, perform a near-circle target detection operation on the current contour, record the detection result,

When all contours are processed, go to step 10.

Step 7. Judgment of overlapping degree of occlusion mask: judging whether the contour currently numbered d _j =k, j = 1, 2, . . . , m coincides with the area of the oil receiving plug occlusion mask.

Step 701, extracting the binary image C ₀ of the current peripheral contour area, setting the pixel value of the contour area to 1, and setting the pixel value of other areas to 0,

Step 702, the binary image C ₀ of the current peripheral contour area is ANDed with the mask image M _i after inverse binarization processing to obtain a binary image C ₁ ,

Step 703 : Count the number of pixels with a pixel value of 1 in the binary image C ₁ , and if the number of pixels exceeds the threshold T _N , determine that the current outline area overlaps with the mask area covered by the oil plug; otherwise, determine that they do not overlap.

Step 8: Detection of occluded near-circle targets: detection and judgment of occluded near-circle targets is performed on the binary image of the current peripheral contour region that satisfies the judging condition of the coincidence degree of the occlusion mask.

Step 801. Perform an "AND" operation on the extracted binary image C ₀ of the current peripheral contour area and the first binary mask image M _c to obtain a binary image C ₂ ,

Step 802, using the disk-shaped structural element D _r with radius r ₂ to perform mathematical morphology opening operation on binary image C ₂ to obtain binary image C ₃ , r ₂ ranges from 3 to 6, and adopts radius r The disc-shaped structural element of ₂ performs mathematical morphology opening operation on the image, which can remove the interference such as umbrella ribs.

Step 803, perform contour extraction operation on the binary image _C3 to obtain the peripheral contour point set, and remove all points in the contour point set that coincide with the second binary mask image _Me ,

Step 804, enter step 9, perform near-circular object detection operation on each contour, adopt new area ratio threshold T ₄ =T ₄₁ , do not judge circularity condition, T ₄₁ ranges from 0.35 to 0.95.

Step 9. Near-circle target detection: for the contour point set The indicated contour is used for near-circle target judgment.

Step 901, the candidate target contour must meet the following area range conditions:

Calculate the range (x _d , y _d ) of the image coordinates of the contour point set as follows:

The region-wide criteria are:

(x _d >T ₃ )&(y _d >T ₃ )&(y _d >T ₄ ·x _d )&(x _d >T ₄ ·y _d ),

In the formula, T ₃ is the threshold of the area range, T ₄ is the threshold of the area ratio, & represents the logical AND operation, the selection range of T ₃ is 7-15, and the selection range of T ₄ is 0.5-0.95.

Step 902, the candidate object contour must meet the following circularity conditions:

In the formula, T ₅ is the circularity threshold, the selection range of T ₅ is 0.60-0.95, l _k is the perimeter of the k-th candidate target contour.

Step 903, the candidate target contour must meet the following least squares ellipse fitting conditions:

(1) Least squares ellipse fitting: use the least squares method to point set Carry out ellipse fitting, and the fitting result is recorded as (x _c , y _c , a, b, θ), where (x _c , y _c ) is the coordinates of the center point of the ellipse, (a, b) are the semi-major and minor axes of the ellipse, and θ is the rotation angle of the ellipse.

(2) Judgment of the size range of the ellipse: the outline of the candidate target must meet the following conditions:

{2a>T ₃ }&{2b>T ₃ }&{a>T ₄ b}&{b>T ₄ a}

(3) Fitting distance error judgment: calculation point set The distance error e _i to the fitted ellipse is as follows:

In the formula, (e _xi , e _yi ) are normalized ellipse point coordinates, which can be calculated as follows:

A candidate target profile must satisfy the following conditions:

In the formula, _T6 is the threshold of the mean value of the fitting error, _T7 is the threshold of the standard deviation of the fitting error, for collection The average value and standard deviation of T ₆ are selected from 0.01 to 0.10, and the selected range of T ₇ is 0.01 to 0.10.

Step 10. Record the ellipse fitting parameters of the candidate regions satisfying the above conditions, and calculate the logarithmic circularity of the ellipse The candidate solution with the largest logarithmic circularity is selected as the final image positioning result of the fuel filler inside the drogue.

Figure 2 is the collected original color image of the cone sleeve, the number of pixels is 640×480. After being converted into a grayscale image, the method of the present invention is used to process the grayscale image of the cone sleeve, and the relevant parameters are set as follows:

T ₁ =75, r ₁ =8, r ₂ =3, T ₂ =300, T ₃ =9, T ₄ =0.707, T ₄₁ =0.4, T ₅ =0.65, T ₆ =0.05, T ₇ =0.03

Figure 3 is the mask image of the reverse oil receiving plug, and Figure 4 is the binary image after binarization of the drogue sleeve image. Form occlusion; Figure 4 is the binary image of the drogue sleeve and the mask image or the binary image after the operation; Figure 5 is the binary image of the drogue sleeve after the morphological opening operation. Isolation; Fig. 6 is the contour extraction result after removing the small-area contour; Fig. 7 is the binary image of the contour area overlapping with the mask area; Fig. 8 is the contour extraction result after the binary image of Fig. 7 is removing the mask area; Fig. 9 Figure 8 is the result of removing the contour points that coincide with the mask area, and it can be seen that the interference contour points caused by occlusion have been eliminated; Figure 10 is the final image detection and positioning result of the oil filler port inside the drogue sleeve, it can be seen that The detection and positioning results are basically coincident with the circular oil filling port inside the taper sleeve. The above results have verified the correctness of the method of the present invention.

Claims (6)

1. A kind of aerial refueling drogue image detection positioning method that is blocked by oil plug, is characterized in that, comprises the steps: A. Perform reverse binarization processing on the mask image blocked by the oil plug, perform mathematical morphology closing operation on the mask image after reverse binarization processing to obtain the first binary mask image, and perform the first binary mask image on the first two performing a mathematical morphology erosion operation on the value mask image to obtain a second binary mask image; B. Carry out inverse binarization processing on the grayscale image of the drogue sleeve to be detected; C. Perform an OR operation on the grayscale image of the drogue sleeve to be detected after the reverse binarization process and the mask image blocked by the oil plug, and perform a mathematical morphological opening operation on the binary image obtained by the OR operation to obtain a binary image ; D, carry out contour extraction to the binary image that step C obtains and obtain peripheral contour, the set of points on each peripheral contour forms peripheral contour edge point set; E. Arrange the areas surrounded by each contour in descending order, filter out the peripheral contours that are smaller than the contour area threshold, and remove the set of points on the filtered peripheral contours from the peripheral contour edge points; F. Judgment of the overlapping degree of the occlusion mask on the peripheral contour left by the screening, enter step F1 when it does not overlap, and enter step F2 when it overlaps, F1. Directly perform near-circle target detection on the peripheral contour, including area range condition constraints, circularity condition constraints, and least squares ellipse fitting condition constraints, and retain target contours that meet the near-circle target detection requirements. F2. Perform occlusion and near-circle target detection on the peripheral contour, and retain the target contour that meets the occlusion and near-circle target detection requirements: first remove the occlusion interference in the peripheral contour, and then carry out the restriction of the area range condition on the peripheral contour after removing the occlusion interference. Near-circle target detection constrained by square ellipse fitting conditions; G. Record the ellipse fitting parameters of each candidate target contour and calculate the logarithmic circularity of each fitted ellipse, and select the candidate target contour with the largest logarithmic circularity as the final image positioning result of the oil filler port inside the drogue sleeve. 2. according to claim 1, a kind of aerial refueling drogue image detection and positioning method blocked by oil plug is characterized in that, step B adopts expression: Inverse binarization processing is performed on the grayscale image of the drogue sleeve to be detected, Among them, (x, y) represents the pixel coordinates of the image, B ₀ (x, y) represents the gray value of the pixel at the position (x, y) of the grayscale image of the cone sleeve to be detected after reverse binarization processing, and D _origin (x, y) represents the pixel gray value of the drogue gray image to be detected at the position (x, y), and T ₁ represents the threshold value of the inverse binarization processing of the drogue gray image to be detected. 3. according to claim 2, a kind of aerial refueling drogue image detection and positioning method blocked by oil plug, it is characterized in that, in the step F, carry out the judgment of blocking mask coincidence degree to the peripheral contour left by screening, concrete method is : Perform an AND operation on the extracted binary image of the current peripheral contour area and the mask image processed by inverse binarization to extract the sub-image of interest, Count the number of pixels with the same pixel value in the occlusion mask area in the sub-image: When the number of counted pixels exceeds the set threshold, it is determined that the current peripheral contour area overlaps with the occlusion mask area, When the counted number of pixels is less than or equal to the set threshold, it is determined that the current peripheral contour area does not overlap with the occlusion mask area. 4. according to claim 3, a kind of air refueling drogue image detection and positioning method blocked by the oil plug is characterized in that, in the step F2, the blocking interference in the peripheral contour is removed, and the specific method is: Perform an AND operation on the extracted binary image of the peripheral contour area and the first binary mask image, perform a mathematical morphology opening operation on the image after the AND operation, and extract the contour of the binary image after the opening operation to obtain the points of the current peripheral contour set, eliminating the points that coincide with the second binary mask image in the current peripheral contour point set. 5. According to any one of claims 1 to 4, the image detection and positioning method of the air refueling drogue covered by the oil plug is characterized in that, in step F2, The range condition of the area is: (x _d >T ₃ )&(y _d >T ₃ )&(y _d >T ₄₁ ·x _d )&(x _d >T ₄₁ ·y _d ), The least squares ellipse fitting condition is: {2a>T ₃ }&{2b>T ₃ }&{a>T ₄₁ ·b}&{b>T ₄₁ ·a} and in, are the pixel coordinates of the 1st, 2nd, ..., n _kth points on the k-th candidate target contour respectively, T ₃ is the area range threshold, T ₄₁ is the area ratio threshold, & represents the logical AND operation, (a, b) is the semi-major and minor axes of the fitted ellipse, for collection The mean and standard deviation of T ₆ is the threshold of the mean value of the fitting error, T ₇ is the threshold of the standard deviation of the fitting error, (x _c , y _c ) are the coordinates of the center point of the fitted ellipse, and θ is the rotation angle of the fitted ellipse. 6. according to claim 5, a kind of aerial refueling drogue image detection and positioning method blocked by oil plug is characterized in that, the expression of calculating and fitting ellipse logarithmic circularity f (a, b) in the step G is :