CN118210329A - Unmanned aerial vehicle high-precision relative positioning method based on binocular depth point cloud - Google Patents

Abstract

The invention discloses a binocular depth point cloud-based unmanned aerial vehicle high-precision relative positioning method, and belongs to the technical field of sensor information fusion. The method comprises the following steps: capturing a target image in the air by using a binocular camera, and calculating a depth point cloud in the whole visual field according to the image of the binocular camera; detecting the binocular image by using the modified YOLOv model to obtain a 2D detection frame of the taper sleeve target; generating a 3D point cloud detection area of the target according to the 2D detection frame, and calculating centroid points of dense point clouds as target points; based on a model predictive control algorithm, the gesture of the unmanned aerial vehicle is accurately adjusted, and the unmanned aerial vehicle is controlled. The unmanned aerial vehicle high-precision relative positioning method based on the binocular depth point cloud enables the unmanned aerial vehicle to perform aerial relative positioning, quickly and accurately identifies a target object, and achieves accurate attitude control.

Description

A high-precision relative positioning method for UAV based on binocular depth point cloud

Technical Field

The present invention relates to the field of sensor information fusion technology, and in particular to a high-precision relative positioning method for an unmanned aerial vehicle based on binocular depth point cloud.

Background technique

Relative positioning technology between unmanned aerial vehicles (UAVs) is a complex field developed on the basis of multidisciplinary intersection, mainly driven by military needs. Initially, this technology was designed to perform complex formation flying and tactical missions to improve the efficiency and safety of military operations. Over time, the increasing application of UAVs in civilian fields, such as disaster monitoring, traffic management, agricultural monitoring, etc., has further promoted the development of relative positioning technology. Relative positioning technology enables UAVs to accurately grasp each other's position and motion status without external reference points, which is crucial for performing precise team missions.

In modern warfare scenarios, drones play an increasingly important role in performing ultra-long-term reconnaissance and ultra-long-range strike missions. However, the execution of these missions is limited by the payload and fuel capacity of drones. In order to overcome these limitations, aerial refueling technology is used, the purpose of which is to significantly improve the flight range, air stay time and payload of drones without making major modifications to the drone structure. The application of this technology has significantly enhanced the performance of drones in tasks such as long-distance flight, target detection and weapon carrying. In the process of aerial refueling, accurate relative positioning technology is particularly important, because the tanker and drone must maintain a strict relative position during flight to ensure the safety and efficiency of the refueling operation. The challenge of this technology is that the two drones need to achieve extremely high positioning accuracy and stability in high-speed flight, while taking into account aerodynamic effects and potential environmental interference. In general, the development of aerial refueling technology based on high-precision relative positioning of drones has not only improved the combat capability of drones in modern warfare, but also promoted the advancement of drone technology in the field of automatic navigation and precision control. Through continuous technological innovation, the application scope and efficiency of drones will continue to expand, providing greater flexibility and stronger strategic advantages for modern military operations.

The docking phase of aerial refueling requires the receiving drone to automatically analyze sensor data and generate corresponding control instructions within limited autonomous operation permissions to achieve accurate, efficient and safe refueling operations. This is not only a challenge to flight technology, but also a test of the drone's autonomous navigation and automatic control system.

First, from the perspective of attitude control of the receiving aircraft, it is crucial to ensure the safety and stability of docking. This requires the UAV to have highly accurate flight path control and speed matching capabilities. The receiving UAV must be able to coordinate with the tanker and be able to provide real-time feedback and adjustments to cope with any changes in the dynamic environment. This requires not only that the UAV has an advanced flight control system, but also that it can process a large amount of sensor data in real time to achieve precise flight dynamic adjustments.

At the same time, accurate detection and positioning of the tanker drogue is crucial for successful aerial refueling. This requires the drone's sensors and navigation system to accurately identify the tanker's location and status and make precise flight adjustments accordingly. The drone needs to be able to accurately dock with the tanker under changing weather conditions and possible electronic interference. In addition, any minor errors that may occur during the refueling process must be detected and corrected in real time to avoid collisions or other dangerous situations.

Regarding the detection and positioning of the drogue, which is a key link in aerial refueling technology, the existing technology still has certain limitations. Although a variety of methods have been used to address this challenge, each has its limitations. For example, the inertial measurement unit (IMU) and global positioning system (GPS) navigation systems perform well in most cases, but in some complex flight environments, they may not provide sufficiently accurate navigation information. This is mainly because GPS signals may be interfered with in high-speed flight and complex meteorological conditions, and IMUs may accumulate large errors. Although sensing technologies that rely on infrared and laser radar (LIDAR) can provide accurate distance and speed data, they are very sensitive to changes in ambient light, which may lead to unstable performance under certain specific lighting conditions. For example, the effectiveness of infrared and laser sensors may be affected under strong sunlight or cloud changes. And in order to meet the special requirements of these sensors in relative positioning, significant design improvements may be required for the refueling pipe and drogue. This not only increases the difficulty of implementing the system, but also increases the complexity and cost of the entire refueling system.

Therefore, to address these challenges, this patent adopts a model predictive control (MPC)-based approach as an innovative and efficient solution.

Summary of the invention

The purpose of the present invention is to provide a high-precision relative positioning method for UAVs based on binocular depth point clouds, so that the UAVs can perform relative positioning in the air, quickly and accurately identify targets, and achieve precise attitude control.

To achieve the above object, the present invention provides a high-precision relative positioning method for a UAV based on binocular depth point cloud, comprising the following steps:

S1, use a binocular camera to capture the target image;

S2, based on the image data captured by the binocular camera in step S1, using the binocular stereo vision algorithm to generate a depth point cloud within the entire field of view in real time;

S3. Optimize the backbone network of the YOLOv7 detection model;

S4. Use the optimized YOLOv7 model to accurately detect the images obtained from the binocular camera and generate the corresponding 2D detection box;

S5. Locate the 3D spatial position of the cone sleeve target in the depth point cloud according to the 2D detection frame, generate the corresponding 3D point cloud detection area, optimize the calculation of the dense area of the point cloud, and apply mean filtering to improve the quality and accuracy of the point cloud data;

S6, continuously tracking the determined target point and storing the position information of the target point in continuous images;

S7, by calculating the position information, the average relative speed of the target point is obtained;

S8. Based on the model predictive control algorithm, the attitude of the UAV is accurately adjusted to control the UAV.

Preferably, step S2 generates a depth point cloud in the entire field of view in real time, including image correction, matching and depth calculation, to obtain the three-dimensional coordinates of each point in space, as shown below:

S21, image correction, the specific steps are as follows:

S211, using Zhang's calibration method to obtain the distortion coefficient of the camera, and combining the internal parameters provided by the camera manufacturer to calibrate the relative position and posture between the two cameras, that is, the external parameters include rotation and translation;

S212. Calculate the transformation matrix required for stereo correction according to the parameters obtained in the above step S211; this includes two rotation matrices, which are used for correcting the left and right camera images respectively, and two new projection matrices.

S213, using the above rotation matrix and projection matrix, remap the original image as shown below:

p ₁ '＝K ₁ 'R ₁ R ^-1 K ₁ ^-1 p ₁

p ₂ '＝K ₂ 'R ₂ (R ^-1 K ₂ ^-1 p ₂ +T)

Among them, K ₁ and K ₂ are the intrinsic parameter matrices of the left and right cameras respectively, R and T are the rotation matrix and translation vector between the two cameras respectively, K ₁ ', K ₂ ', R ₁ and R ₂ are the intrinsic parameter matrices and rotation matrices that need to be recalculated, p ₁ ' and p ₂ ' are the corrected pixel positions. Using the above transformation, the images of the two cameras can be transformed into a common viewing plane, and the projections of the same object point in the two images have the same y coordinate;

S22. Based on the improved cost function, the image is matched by introducing an adaptive Gaussian weight matrix and normalization. The formula is as follows:

Where W is the window area, which is used to define the size of the area considered around each pixel; _IL and _IR are two images to be compared; p and q are the corresponding pixels in images _IL and _IR , respectively; i and j are the relative position offsets in the window area; W(i,j,p) is the adaptive Gaussian weight, and (p+i,p+j) is the pixel after offset;

S23, depth calculation formula is as follows:

Where Z is the depth of the pixel, f is the focal length of the camera; B is the baseline distance between the two cameras, that is, the horizontal distance between the two cameras; D is the disparity value, that is, the horizontal position difference of the same scene point in the two camera images;

S24, convert the 2D image coordinates and depth values into 3D space coordinates uniformly, the formula is as follows:

Among them, (x, y) is the coordinate of the pixel point in the image plane, (X, Y, Z) is the coordinate of the pixel point in the world coordinate system, and (c _x , c _y ) is the coordinate of the optical center of the image.

Preferably, in step S3, in optimizing the backbone network of the YOLOv7 detection model, the specific steps are as follows:

S31. Set the convolution layer weight threshold and remove weights less than the threshold. The formula is as follows:

Wherein, ω _i,j ' is the weight matrix after pruning, θ is the preset threshold, and ω _i,j is the element in the i-th row and j-th column of the weight matrix;

S32. Use the CBAM module to perform spatial attention weighting and adjust the weight distribution method of CBAM as follows:

S _cone (x,y)＝σ(f _centre (I(x,y))+f _dark (I(x,y)))

Where σ is the activation function, f _centre and f _dark correspond to the attention weights of the center area of the refueling drogue and the dark block respectively;

S33. Apply the spatial attention map S _cone to the original feature map F using element-wise multiplication as follows:

S34. Use deep separable convolution to more efficiently extract the features of the refueling drogue, and use medium-scale feature maps for detection. The formula is as follows:

P _cone = F _mid

Among them, P _cone is the feature map used to detect the refueling cone sleeve, and F _mid is the feature map with moderate scale in the network.

Preferably, in step S5, the specific steps are as follows:

S51, locate the three-dimensional spatial position of the cone sleeve target in the depth point cloud according to the 2D detection frame, and generate a corresponding 3D point cloud detection area, and remove objects far away from the camera or objects that are too close, as shown below:

Wherein, D _roi (x, y) represents the point cloud depth value of the detection area;

S52. Optimize the calculation of dense area of point cloud. The formula is as follows:

Where l is an indicator function that generates a binary output for a given condition. If the condition is true, it returns 1, otherwise it returns 0.

S53, perform mean filtering on the valid point cloud, the formula is as follows:

Preferably, in step S6, in storing the position information of the target point, the centroid position of the dense point cloud is calculated as the target point, and the formula is as follows:

Among them, (X _center ,Y _center ,Z _center ) is the relative target point in 3D space.

Preferably, the average speed of the target point is calculated in step S7 as follows:

Wherein, p _i =(x _i ,y _i , _zi ) is the position coordinate of the target point in the i-th frame image.

Preferably, in step S8, in constructing the state and control input modeling of the UAV, based on the model predictive control algorithm, the state scalars of the UAV include position (x, y, z), yaw angle θ, and pitch angle ψ;

The control scalars include: velocity v, yaw rate yaw, and roll rate pitch; the dynamic model of the drone can be expressed as:

Preferably, based on the dynamic model of the UAV, for the aerial docking and refueling scenario, the objective function is as follows:

S811. First, calculate and minimize the distance between the drone and the target point. The formula is as follows:

Among them, (X _center,k ,Y _center,k ,Z _center,k ) is the position of the current target point relative to the drone, and ω _pos is the weight of the position error;

S812. Secondly, minimize the change rate of the UAV's flight speed. The formula is as follows:

Where ω _ctrl is the weight that controls the input change;

S813. Finally, the objective function is as follows:

Preferably, based on the UAV dynamics model, for the multi-UAV relative positioning scenario, the objective function is as follows:

S821, the tracking error is achieved by calculating the Euclidean distance between the current position of the drone and the target position, as shown below:

Where _Xk is the position of the UAV at the kth time step, and _ω1 is the weight factor;

S822, field of view correction, as follows:

Among them, ω ₂ is the weight factor, and ellipse is a function of the field of view angle and the position of the drone; this function can be calculated based on the position, direction and field of view angle of the drone;

S823. In 3D space, the field of view is regarded as an ellipsoid, and the target is located inside the ellipsoid. The elliptic function formula is as follows:

Among them, (x,y,z) is the position of the UAV, and ( _xu , _yu , _zu ) is the position of the target point.

Therefore, the present invention adopts the above-mentioned UAV high-precision relative positioning method based on binocular depth point cloud, and the beneficial effects are as follows:

1. Use the centroid of the dense point cloud area in the detection frame instead of the center point of the detection frame as the target point. In the actual three-dimensional space, the shape and posture of the object may cause part of its surface to be closer to the camera, and the target may not fill the detection frame. Using the centroid can more accurately represent the actual position of the object and avoid detection jitter caused by detection frame jitter.

2. When the target moves or rotates, the position change of the center of mass is smoother than the position change of the detection box center, thus providing more stable target tracking.

3. By considering the UAV state and environmental changes in the next few time steps (discrete time intervals), advanced path planning and UAV docking are achieved. By continuously updating the target position, the proposed MPC scheme enables the UAV to dynamically track moving targets, which is suitable for aerial refueling scenarios. It is also particularly important for search and rescue, surveillance and other applications.

4. Special consideration is given to the visibility constraints of limited field of view. This ensures that the target always remains within the camera field of view when the UAV is performing mission tracking. The MPC controller can generate control instructions at a high frequency, which is feasible on most embedded computing platforms, indicating that the scheme is suitable for real-time applications.

The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a flow chart of a UAV aerial refueling docking method based on a binocular depth point cloud high-precision relative positioning method for UAVs;

Figure 2 is a disparity map of binocular image weights using an improved SAD algorithm for a high-precision relative positioning method for UAVs based on binocular depth point clouds;

FIG3 is a schematic diagram of the allocation of dark color block weights by an improved SAD algorithm of a high-precision relative positioning method for UAVs based on binocular depth point clouds;

FIG4 is a block diagram of an improved YOLOv7 recognition algorithm for a high-precision relative positioning method of a UAV based on binocular depth point cloud;

FIG5 is a diagram of identifying a target object of a UAV using a high-precision relative positioning method for a UAV based on binocular depth point cloud;

FIG6 is a simulation trajectory of an MPC-controlled UAV tracking a dynamic target using a high-precision relative positioning method for UAVs based on binocular depth point clouds.

Detailed ways

The technical solution of the present invention is further described below through the accompanying drawings and embodiments.

As shown in FIG1 , a high-precision relative positioning method for a UAV based on binocular depth point cloud includes the following steps:

S1, use a binocular camera to capture the images of the refueling machine and the drogue;

S2, based on the image data captured by the binocular camera in step S1, using the binocular stereo vision algorithm to generate a depth point cloud within the entire field of view in real time;

S3. Optimize the backbone network of the YOLOv7 detection model;

S4. Use the optimized YOLOv7 model to accurately detect the images obtained from the binocular camera and generate the corresponding 2D detection box;

S5. Locate the 3D spatial position of the cone sleeve target in the depth point cloud according to the 2D detection frame, generate the corresponding 3D point cloud detection area, optimize the calculation of the dense area of the point cloud, and apply mean filtering to improve the quality and accuracy of the point cloud data;

S6, continuously tracking the determined target point, and storing the position information of the target point in 10 consecutive frames of images;

S7, by calculating the position information, the average relative speed of the target point between 10 frames is obtained;

S8. Based on the model predictive control algorithm, the attitude of the UAV is accurately adjusted to control the UAV.

Example

S1. Use a binocular camera to capture images of the tanker and drogue in an aerial refueling docking scenario.

In the ROS framework, the binocular camera creates nodes usb_cam_node1 and usb_c am_node2 respectively, where 1 and 2 are the node numbers of the binocular camera, and the node is an independent process that describes different functions in the ROS framework. The binocular camera has a camera sensor with image capture function, and publishes RGB image messages of cam_image1 and cam_image2 respectively.

S2. Based on the image data captured by the binocular camera, the binocular stereo vision algorithm is used to generate the depth point cloud in the entire field of view in real time. This step includes image correction, matching and depth calculation to obtain the three-dimensional coordinates of each point in space.

S21. The specific steps of image correction are as follows:

S211. First, the image is calibrated. The distortion coefficient of the camera is obtained using the Zhang calibration method. Combined with the internal parameters provided by the camera manufacturer, the relative position and posture between the two cameras are calibrated, that is, the external parameters include rotation and translation.

S212: Calculate the transformation matrix required for stereoscopic correction according to the parameters obtained in step S211, wherein the transformation matrix includes two rotation matrices, which are used for correcting the left and right camera images respectively, and two new projection matrices.

S213. Finally, the original image is remapped using the above rotation matrix and projection matrix, as shown below:

p ₁ '＝K ₁ 'R ₁ R ^-1 K ₁ ^-1 p ₁

p ₂ '＝K ₂ 'R ₂ (R ^-1 K ₂ ^-1 p ₂ +T)

Among them, K ₁ and K ₂ are the intrinsic parameter matrices of the left and right cameras respectively, R and T are the rotation matrix and translation vector between the two cameras respectively, K ₁ ', K ₂ ', R ₁ and R ₂ are the intrinsic parameter matrices and rotation matrices that need to be recalculated, p ₁ ' and p ₂ ' are the corrected pixel positions. Using the above transformation, the images of the two cameras can be transformed into a common viewing plane, and the projections of the same object point in the two images have the same y coordinate.

The above steps can be automatically completed using the stereo correction function in the OpenCV library.

S22. Based on the improved cost function, image matching is achieved by introducing an adaptive Gaussian weight matrix and normalization.

As shown in Figure 2, the disparity map represents the pixel offset between the two images. For each pixel in the left image, the corresponding pixel is searched within a certain range in the right image. This search process involves a special aerial refueling visual environment, and an improved SAD (Sum of Absolute Differences) cost function is used to find the best matching position to obtain the offset or disparity value of the pixel in the disparity map.

The following are the main features of the improved cost function:

1. Adaptive Gaussian weight matrix: The cost function introduces an adaptive Gaussian weight matrix. During the pixel matching process, the environment around each pixel will be considered. In particular, within the matching window, the dark pixel area will receive a higher weight than the blue or white background pixel, as shown in Figure 3. This helps to accurately match pixels under complex lighting conditions.

2. Normalization: Normalizing pixel values helps reduce the impact of illumination and contrast on image matching. The cost function can better handle images under different illumination conditions and ensure the stability and accuracy of matching.

The improved cost function is as follows:

Where W is the window area, which is used to define the size of the area around each pixel for comparison; _IL and _IR are two images to be compared; p and q are the corresponding pixels in images _IL and _IR , respectively; i and j are the relative position offsets in the window area; W(i, j, p) represents the adaptive Gaussian weight, as shown below:

W(i,j,p)＝G(i,j)×f(p+i,p+j)

Among them, (p+i,p+j) is the pixel after offset, and G(i,j) is the standard Gaussian weight function, as shown below:

Among them, σ is the standard deviation of the Gaussian function, which controls the range of weight distribution; the function f(x) is the pixel value adaptive adjustment of the Gaussian weight, as shown below:

Among them, α and β are adaptive parameters set according to the specific lighting environment, and L(p) is a function that weights and calculates the grayscale value of the image based on the human eye's perception sensitivity to different colors, as shown below:

L(p)＝0.299Rp+0.587Gp+0.114Bp

Among them, minL and maxL are the set grayscale value intervals, which can be simply set to 0 and 255; when the pixel block is darker, the adaptive parameter is more biased towards β; when the pixel block is sky blue or white, the adaptive parameter is more biased towards α.

S23. After obtaining the disparity map, the depth of each pixel is obtained using the following formula:

Among them, Z is the depth of the pixel, f is the focal length of the camera, B is the baseline distance between the two cameras, that is, the horizontal distance between the two cameras; D is the disparity value, that is, the horizontal position difference of the same scene point in the two camera images.

S24, convert the 2D image coordinates and depth values into 3D space coordinates uniformly, the formula is as follows:

Among them, (x, y) is the coordinate of the pixel point in the image plane, (X, Y, Z) is the coordinate of the pixel point in the world coordinate system, (c _x , c _y ) is the coordinate of the optical center (principal point) of the image, which is the intrinsic parameter of the camera. In this way, a 3D coordinate is obtained for each image pixel, thus forming a depth point cloud.

S3. Optimize the backbone network of the YOLOv7 detection model.

As shown in Figure 4, the attention mechanism is introduced to improve the model's ability to recognize cone sleeve features; the pooling layer is improved to enhance the robustness of the model when processing cone sleeves of different sizes; the loss function is optimized to improve the detection accuracy; the improved YOLOv7 detection model adapts to the real-time and lightweight requirements of the UAV computing platform.

S31. In view of the particularity of the aerial refueling scenario, the YOLOv7 backbone network is modified, a new convolutional layer is added to CSPDarkNet, and redundant convolutional layers and fully connected layers are deleted to simplify the model structure; the model is pruned, and a threshold is set for the weight of each layer. If the absolute value of the weight is less than the threshold, it is deleted or set to zero. The formula is as follows:

Among them, ω _i,j ' is the weight matrix after pruning, θ is the preset threshold, and ω _i,j is the element in the i-th row and j-th column in the weight matrix.

Through the above operations, the model is lightweight and more suitable for real-time aerial flight missions, which not only improves the computational efficiency of the model, but also helps reduce the hardware resource requirements to meet the requirements of UAV visual tasks such as drogue detection and relative pose tracking.

S32. In order to ensure that the model pays more attention to the central area and dark blocks of the refueling drogue, the Convolutional Block Attention Module (CBAM) module is used for spatial attention weighting. Considering that the background color is mostly blue sky, the weight distribution method of CBAM is adjusted as follows:

S _cone (x,y)＝σ(f _centre (I(x,y))+f _dark (I(x,y)))

Among them, σ is the activation function, f _centre and f _dark correspond to the attention weights of the center area and dark block of the refueling cone sleeve respectively.

S33. Apply the spatial attention map S _cone to the original feature map F using element-wise multiplication as follows:

S34. Due to the single detection target and usage scenario, a depth-separable convolution is used to more efficiently extract the features of the refueling drogue. At the same time, considering that the size of the refueling drogue is relatively fixed, a medium-scale feature map is directly used for detection without the need for a complete feature pyramid structure. The formula is as follows:

P _cone = F _mid

Among them, P _cone is the feature map used to detect the refueling cone sleeve, and F _mid is the feature map with moderate scale in the network.

S4. Use the optimized YOLOv7 model to accurately detect the images obtained from the binocular camera. The model can accurately identify the target in the binocular image and generate the corresponding 2D detection box, as shown in Figure 5;

S5. Locate the 3D spatial position of the cone sleeve target in the depth point cloud according to the 2D detection frame, generate the corresponding 3D point cloud detection area, optimize the calculation of the dense area of the point cloud, and apply mean filtering to improve the quality and accuracy of the point cloud data.

S51. Extract the area corresponding to the 2D detection box from the complete depth map. The values outside the sub-area are set to 0 to reduce the amount of data for subsequent processing and ensure that the focus is on the area that may contain the target. Define the depth range to eliminate objects far away from the camera or objects that are too close, as shown below:

Among them, D _roi (x, y) represents the point cloud depth value of the detection area.

S52, dense areas refer to areas with more points in the point cloud. These areas usually represent the surface of the object rather than the space or noise of the object. The local density of the point is estimated by counting non-zero points in a small window around each point. The size of the window is determined by r, which represents the radius of the window. The calculation of the dense area of the point cloud is optimized as follows:

Here, l is an indicator function that generates a binary output for a given condition; specifically, it returns 1 if the condition is true and 0 otherwise.

S53. In order to reduce the influence of noise and outliers, the effective point cloud is filtered by mean, and the formula is as follows:

S6. Continuously track the determined target point and store the position information of the target point in the continuous image, taking the centroid position of the dense point cloud as the target point, as shown below:

Among them, (X _center ,Y _center ,Z _center ) is the relative target point in 3D space.

S7, by calculating the position information, the average speed of the target point is obtained;

For single target recognition scenarios, Kalman filtering is used to smooth the noise in the detection algorithm to reduce the interference of noise on target recognition. In addition, Kalman filtering can also predict the inter-frame position of the target point when the target is temporarily lost, and can still provide a continuous and consistent target position estimate when the target cannot be directly observed.

In order to further reduce the impact of target point jitter, this step also includes storing 10 consecutive frames of estimated position data. The average speed of the target point in 10 frames of images is calculated as follows:

Wherein, p _i =(x _i ,y _i , _zi ) is the position coordinate of the target point in the i-th frame image.

This calculation not only smooths out fluctuations in the target's position, but also provides critical dynamic information about the target's motion.

S8. Based on the model predictive control algorithm, the attitude of the UAV is accurately adjusted to control the UAV.

Based on the model predictive control MPC algorithm, the UAV automatically detects the attitude angle, obtains the pitch angle and yaw angle, builds the state and control input modeling of the UAV, and controls the UAV to track the dynamic target simulation trajectory through MPC, as shown in Figure 6.

The state scalars of the drone include: position (x, y, z), yaw angle θ, pitch angle ψ; the control scalars include: velocity v, yaw rate yaw, and roll rate pitch; the dynamic model of the drone can be expressed as:

S81, for aerial docking and refueling scenarios;

S811. First, minimize the distance between the drone and the target point as follows:

Among them, (X _center,k ,Y _center,k ,Z _center,k ) is the position of the current target point relative to the drone, and ω _pos is the weight of the position error.

S812. Secondly, in order to ensure the smooth flight of the aircraft, the flight speed change rate of the drone needs to be minimized. The formula is as follows:

Among them, ω _vel is the weight of velocity change; in order to ensure the smoothness of control input (such as yaw rate and pitch rate), the formula is as follows:

Among them, ω _ctrl is the weight that controls the input change.

S813. Finally, for the aerial docking and refueling scenario, the objective function is expressed as:

S82, for the relative positioning scenario of multiple drones;

The objective function is divided into the following parts:

S821. Tracking error is achieved by calculating the Euclidean distance between the current position of the drone and the target position to ensure that the drone follows the target path, aiming to minimize the acceptable distance difference between the drone and the target. The tracking error calculation formula is as follows:

Where _Xk is the position of the UAV at the kth time step and _ω1 is the weight factor.

S822, field of view correction, to ensure that the target always remains within the field of view of the drone camera. The field of view correction calculation formula is as follows:

Among them, ω ₂ is the weight factor, and ellipse is a function of the field of view angle and the position of the drone; this function can be calculated based on the position, direction and field of view angle of the drone.

S823. In 3D space, the field of view is considered as an ellipsoid, and the target should be located inside the ellipsoid. The formula of the ellipse function is as follows:

Among them, (x,y,z) is the position of the UAV, and ( _xu , _yu , _zu ) is the position of the target point.

Therefore, the present invention adopts the above-mentioned high-precision relative positioning method of UAV based on binocular depth point cloud, so that the UAV can perform relative positioning in the air, quickly and accurately identify the target object, and achieve precise attitude control.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims (9)

1. A high-precision relative positioning method for an unmanned aerial vehicle based on binocular depth point cloud, characterized in that it comprises the following steps: S1, use a binocular camera to capture the target image; S2, based on the image data captured by the binocular camera in step S1, using the binocular stereo vision algorithm to generate a depth point cloud within the entire field of view in real time; S3. Optimize the backbone network of the YOLOv7 detection model; S4. Use the optimized YOLOv7 model to accurately detect the images obtained from the binocular camera and generate the corresponding 2D detection box; S5. Locate the 3D spatial position of the cone sleeve target in the depth point cloud according to the 2D detection frame, generate the corresponding 3D point cloud detection area, optimize the calculation of the dense area of the point cloud, and apply mean filtering to improve the quality and accuracy of the point cloud data; S6, continuously tracking the determined target point and storing the position information of the target point in continuous images; S7, by calculating the position information, the average relative speed of the target point is obtained; S8. Based on the model predictive control algorithm, the attitude of the UAV is accurately adjusted to control the UAV. 2. According to claim 1, a high-precision relative positioning method for unmanned aerial vehicles based on binocular depth point cloud is characterized in that: step S2 generates a depth point cloud in the entire field of view in real time, including image correction, matching and depth calculation to obtain the three-dimensional coordinates of each point in space, as shown below: S21, image correction, the specific steps are as follows: S211, using Zhang's calibration method to obtain the distortion coefficient of the camera, and combining the internal parameters provided by the camera manufacturer to calibrate the relative position and posture between the two cameras, that is, the external parameters include rotation and translation; S212. Calculate the transformation matrix required for stereo correction according to the parameters obtained in the above step S211; this includes two rotation matrices, which are used for correcting the left and right camera images respectively, and two new projection matrices. S213, using the above rotation matrix and projection matrix, remap the original image as shown below: p ₁ '＝K ₁ 'R ₁ R ^-1 K ₁ ^-1 p ₁ p ₂ '＝K ₂ 'R ₂ (R ^-1 K ₂ ^-1 p ₂ +T) Among them, K ₁ and K ₂ are the intrinsic parameter matrices of the left and right cameras respectively, R and T are the rotation matrix and translation vector between the two cameras respectively, K ₁ ', K ₂ ', R ₁ and R ₂ are the intrinsic parameter matrices and rotation matrices that need to be recalculated, p ₁ ' and p ₂ ' are the corrected pixel positions. Using the above transformation, the images of the two cameras can be transformed into a common viewing plane, and the projections of the same object point in the two images have the same y coordinate; S22. Based on the improved cost function, the image is matched by introducing an adaptive Gaussian weight matrix and normalization. The formula is as follows: Where W is the window area, which is used to define the size of the area considered around each pixel; _IL and _IR are two images to be compared; p and q are the corresponding pixels in images _IL and _IR , respectively; i and j are the relative position offsets in the window area; W(i,j,p) is the adaptive Gaussian weight, and (p+i,p+j) is the pixel after offset; S23, depth calculation formula is as follows: Where Z is the depth of the pixel, f is the focal length of the camera; B is the baseline distance between the two cameras, that is, the horizontal distance between the two cameras; D is the disparity value, that is, the horizontal position difference of the same scene point in the two camera images; S24, convert the 2D image coordinates and depth values into 3D space coordinates uniformly, the formula is as follows: Among them, (x, y) is the coordinate of the pixel point in the image plane, (X, Y, Z) is the coordinate of the pixel point in the world coordinate system, and (c _x , c _y ) is the coordinate of the optical center of the image. 3. According to the high-precision relative positioning method of a UAV based on binocular depth point cloud according to claim 1, it is characterized in that in step S3, in optimizing the backbone network of the YOLOv7 detection model, the specific steps are as follows: S31. Set the convolution layer weight threshold and remove weights less than the threshold. The formula is as follows: Wherein, ω _i,j ' is the weight matrix after pruning, θ is the preset threshold, and ω _i,j is the element in the i-th row and j-th column of the weight matrix; S32. Use the CBAM module to perform spatial attention weighting and adjust the weight distribution method of CBAM as follows: S _cone (x,y)＝σ(f _centre (I(x,y))+f _dark (I(x,y))) Where σ is the activation function, f _centre and f _dark correspond to the attention weights of the center area of the refueling drogue and the dark block respectively; S33. Apply the spatial attention map S _cone to the original feature map F using element-wise multiplication as follows: S34. Use deep separable convolution to more efficiently extract the features of the refueling drogue, and use medium-scale feature maps for detection. The formula is as follows: P _cone = F _mid Among them, P _cone is the feature map used to detect the refueling cone sleeve, and F _mid is the feature map with moderate scale in the network. 4. The high-precision relative positioning method for unmanned aerial vehicles based on binocular depth point cloud according to claim 1, characterized in that in step S5, the specific steps are as follows: S51, locate the three-dimensional spatial position of the cone sleeve target in the depth point cloud according to the 2D detection frame, and generate a corresponding 3D point cloud detection area, and remove objects far away from the camera or objects that are too close, as shown below: Wherein, D _roi (x, y) represents the point cloud depth value of the detection area; S52. Optimize the calculation of dense area of point cloud. The formula is as follows: Where l is an indicator function that generates a binary output for a given condition. If the condition is true, it returns 1, otherwise it returns 0. S53, perform mean filtering on the valid point cloud, the formula is as follows: 5. A high-precision relative positioning method for unmanned aerial vehicles based on binocular depth point cloud according to claim 1, characterized in that in step S6, when storing the position information of the target point, the centroid position of the dense point cloud is calculated as the target point, and the formula is as follows: Among them, (X _center ,Y _center ,Z _center ) is the relative target point in 3D space. 6. The high-precision relative positioning method for unmanned aerial vehicles based on binocular depth point cloud according to claim 1, characterized in that the average speed of the target point is calculated in step S7 as follows: Wherein, p _i =(x _i ,y _i , _zi ) is the position coordinate of the target point in the i-th frame image. 7. A high-precision relative positioning method for UAV based on binocular depth point cloud according to claim 1, characterized in that: in step S8, in constructing the state and control input modeling of the UAV, based on the model predictive control algorithm, the state scalar of the UAV includes position (x, y, z), yaw angle θ, and pitch angle ψ; The control scalars include: velocity v, yaw rate yaw, and roll rate pitch; the dynamic model of the drone can be expressed as: 8. The high-precision relative positioning method of a UAV based on binocular depth point cloud according to claim 7 is characterized in that, based on the dynamic model of the UAV, for the aerial docking and refueling scenario, the objective function is as follows: S811. First, calculate and minimize the distance between the drone and the target point. The formula is as follows: Among them, (X _center,k ,Y _center,k ,Z _center,k ) is the position of the current target point relative to the drone, and ω _pos is the weight of the position error; S812. Secondly, minimize the change rate of the UAV's flight speed. The formula is as follows: Where ω _ctrl is the weight that controls the input change; S813. Finally, the objective function is as follows: 9. The high-precision relative positioning method of a UAV based on binocular depth point cloud according to claim 7 is characterized in that, based on the dynamic model of the UAV, for the relative positioning scenario of multiple UAVs, the objective function is as follows: S821, the tracking error is achieved by calculating the Euclidean distance between the current position of the drone and the target position, as shown below: Where _Xk is the position of the UAV at the kth time step, and _ω1 is the weight factor; S822, field of view correction, as follows: Among them, ω ₂ is the weight factor, and ellipse is a function of the field of view angle and the position of the drone; this function can be calculated based on the position, direction and field of view angle of the drone; S823. In 3D space, the field of view is regarded as an ellipsoid, and the target is located inside the ellipsoid. The elliptic function formula is as follows: Among them, (x,y,z) is the position of the UAV, and ( _xu , _yu , _zu ) is the position of the target point.