CN106681353A - Unmanned aerial vehicle (UAV) obstacle avoidance method and system based on binocular vision and optical flow fusion - Google Patents

Abstract

The invention discloses an unmanned aerial vehicle obstacle avoidance method and system based on binocular vision and optical flow fusion. The method obtains image information in real time through an airborne binocular camera; uses a graphics processor GPU to obtain image depth information; uses the obtained depth information Extract the geometric contour information of the most threatening obstacle and calculate its threat distance through the threat depth model; Obtain the obstacle tracking window through rectangular fitting of the geometric contour information of the obstacle and calculate the optical flow field of the area where the obstacle belongs to obtain the obstacle The speed of the object relative to the UAV; the flight control computer issues an evasive flight action command to avoid obstacles based on the calculated obstacle distance information, geometric contour information and relative speed information. The present invention effectively fuses the depth information of the obstacle with the optical flow vector, obtains the movement information of the obstacle relative to the drone in real time, and improves the rapid visual obstacle avoidance ability of the drone. Compared with Traditional algorithms have been greatly improved.

Description

UAV obstacle avoidance method and system based on binocular vision and optical flow fusion

technical field

The invention relates to an obstacle avoidance method for an unmanned aerial vehicle, in particular to an obstacle avoidance method and system for an unmanned aerial vehicle based on fusion of binocular vision and optical flow, and belongs to the technical field of obstacle avoidance for an unmanned aerial vehicle.

Background technique

With the development of UAV technology and its application market, UAVs are often faced with special tasks that are different from the past, and these tasks put forward higher requirements for their ability to quickly identify and avoid obstacles on the way. Because the vision-based obstacle avoidance system usually adopts a passive working mode, it has the characteristics of simple equipment, low cost, good economy, and wide application range.

Compared with the obstacle avoidance system based on active sensors such as ultrasonic waves and lidar, the visual obstacle avoidance system has faster response speed and higher precision, and can provide richer information such as color, texture, geometric shape, etc., so it has been more and more popular. Much attention.

Compared with monocular vision, binocular vision can obtain distance information perpendicular to the camera, which can more effectively determine the relative position of obstacles and drones, and also helps to quickly and accurately detect obstacles from complex backgrounds. Segmentation; At present, binocular vision has been widely used in many fields such as robot navigation and target tracking.

The optical flow method is a method to predict the movement of pixels by using the correlation of pixel intensity data in the image sequence, that is, to study the temporal change of the image brightness to establish the motion field of the target pixel set; in general, the optical flow can be used for Efficient and precise measurements of camera motion, object motion in the scene, or both, whose predictors can represent the instantaneous velocity of object motion.

There are usually two methods for calculating the optical flow field: the sparse optical flow method and the dense optical flow method. Sparse optical flow selects some feature points in the image scene, and fits the velocity of the entire motion field by measuring the velocity of these feature points. Dense optical flow is to calculate the motion field of the entire area, so as to obtain the moving speed of the target area relative to the camera; sparse optical flow calculation speed is fast, but the calculation value error is large; although dense optical flow calculation is relatively accurate, if there is no correct Precise segmentation of the calculated target area will greatly increase the calculation time, so it is often used in conjunction with a fast and accurate image segmentation algorithm.

The application number is CN201410565278.3 "Vehicle Motion Information Detection Method Based on Binocular Stereo Vision and Optical Flow Fusion", which mainly marks the points of interest on the ground through binocular vision, and then calculates the optical flow value of the points of interest, and finally The three-dimensional translation velocity and three-dimensional rotation velocity of the ground vehicle are then estimated by least squares fitting. This method uses the velocity information of the feature points to replace the velocity information of the vehicle. Although the calculation speed is enhanced, the estimation accuracy is difficult to guarantee. Moreover, this method only estimates the motion information of the vehicle itself, and cannot identify and avoid obstacles in the process of traveling, so it is difficult to be applied in the actual field.

Application No. CN201110412394.8 "A Binocular Stereo Vision-Based Inspection Detector Autonomous Obstacle Avoidance Planning Method", which mainly calculates the three-dimensional coordinates of all pixels in the camera image through binocular stereo vision and forms a three-dimensional map of detection points , choose an optimal path to avoid obstacles according to the three-dimensional map. This method needs to calculate the three-dimensional coordinates of all pixels in the field of view, which has high requirements on the computing performance of the processor and the capacity of the memory, and is not suitable for small embedded airborne devices.

The application is CN201510688485.2 "A UAV Autonomous Obstacle Detection System and Method Based on Binocular Vision", which focuses on the hardware architecture of using binocular cameras for obstacle detection, which mainly uses FPGA as the processing binocular image computing core. Although FPGA has the characteristics of small size and fast operation speed, FPGA is expensive and requires a special development language for programming, which is not conducive to docking with other modules. Moreover, the patent only describes the hardware architecture of UAVs using binocular vision for obstacle detection, and does not explain the specific algorithm of how to detect obstacles.

Therefore, although there are many researches in the field of obstacle avoidance using binocular vision at home and abroad, most methods cannot quickly obtain the position and speed of obstacles relative to the UAV, and it is difficult to quickly and accurately avoid obstacles. Most of them are difficult to apply to the field of UAV real-time obstacle avoidance.

Contents of the invention

The technical problem to be solved by the present invention is to provide a UAV obstacle avoidance method and system based on the fusion of binocular vision and optical flow, which can effectively fuse the depth information of obstacles with the optical flow vector, and obtain the relative distance between obstacles and unmanned vehicles in real time. The movement information of the drone realizes the real-time obstacle avoidance of the drone.

The present invention adopts the following technical solutions for solving the problems of the technologies described above:

An obstacle avoidance method for drones based on binocular vision and optical flow fusion, including the following steps:

Step 1, use the binocular camera to obtain the image in the direction of the UAV, and perform grayscale transformation on the image;

Step 2, calculate the feature information of each pixel on the image after the grayscale transformation and perform stereo matching to obtain the depth map information in the direction of the UAV;

Step 3: Divide the depth values in the depth map into two types: those belonging to obstacles or the depth values belonging to the background, divide the depth map into obstacle areas and background areas, and use the contour with the largest closed area in the obstacle area as the obstacle contour , and fit it with a rectangular frame to obtain the obstacle tracking window as the geometric information of the obstacle;

Step 4, use the dense optical flow method to calculate the velocity vector of the obstacle tracking window sliding, obtain the velocity of the window in the x, y direction, and judge the position of the obstacle tracking window in the next frame of image according to the speed, and compare the judged position with the next Compare the obstacle tracking window position actually calculated in one frame, if the difference between the two is less than the threshold, go to step 5, otherwise, return to step 1 to recalculate;

Step 5, using the depth threat model to calculate the threat depth value of the obstacle to the UAV;

Step 6, converting the obstacle geometry information calculated in step 3 and the obstacle velocity information calculated in step 4 from pixel coordinates to world coordinates, and correcting them using the motion parameters of the drone;

Step 7. Send the obstacle position information and the geometry and speed information obtained in step 6 to the flight control computer of the UAV. The flight control computer controls the UAV to make real-time evasion actions according to the above information.

As a preferred solution of the method of the present invention, the specific process of using the binocular camera to obtain the image in the direction of the UAV in step 1 is: the binocular camera is installed on the nose of the UAV, and obtained by a calibration method The internal and external parameter matrix and distortion parameters of the binocular camera, using the binocular camera to obtain the image in the direction of the UAV, and correcting the image according to the internal and external parameter matrix and distortion parameters, to obtain two images without distortion and line alignment.

As a preferred solution of the method of the present invention, the specific process of step 2 is: calculate the energy in eight directions of each pixel on the image after the grayscale transformation: up, down, left, right, upper left, lower left, upper right, and lower right The function is accumulated, and the parallax value is calculated to minimize the accumulated energy function, and the depth value information of each pixel is determined according to the parallax value.

As a preferred solution of the method of the present invention, in step 3, before using the contour with the largest closed area in the obstacle region as the obstacle contour, use a speckle filter to filter the depth map that distinguishes the obstacle region and the background region, and remove noise.

As a preferred solution of the method of the present invention, the specific process of dividing the depth values in the depth map into two types of depth values belonging to obstacles or backgrounds in step 3 is as follows:

Set the segmentation threshold D _h , classify the depth value greater than or equal to D _h as the depth value belonging to the obstacle, classify the depth value less than D _h as the depth value belonging to the background, and maximize the distance between the obstacle and the background Solving for the variance of D _h , the variance The calculation formula is:

Among them, ω ₀ and ω ₁ are respectively the probability that the depth value is divided into obstacles and background depth values by D _h , and μ ₀ and μ ₁ are the mean values of the depth values belonging to obstacles and background respectively:

Among them, D _i is a discrete credible depth layer, i=1,...,t, t is the number of depth layers, D ₁ ,...,D _h are depth layers belonging to obstacles, D _h+1 ,...,D _t is the depth layer belonging to the background, K _j is the number of depth values in each depth layer, j=1,...,t.

As a preferred version of the method of the present invention, the specific process of the step 5 is:

Set the depth value set belonging to obstacles as D _K ={d ₁ ,d ₂ ,...,d _K }, d ₁ ,d ₂ ,...,d _K are all depth values, and K is the depth value belonging to obstacles number; D ₁ ,…,D _t are discrete credible depth layers, t is the number of depth layers, K ₁ ,…,K _t are the number of depth values in each depth layer, if there is at least one 1≤j≤t, then the threat depth value of the obstacle to the UAV is: Among them, D _min is greater than K j of K _j corresponds to the minimum depth layer in the depth layer, and K _min is the number of depth values in D _min ; if all K ₁ ,...,K _t are less than Then the threat depth value of the obstacle to the UAV is:

The UAV obstacle avoidance system based on binocular vision and optical flow fusion includes an image acquisition module, an image processing module, an inertial measurement module, and a GNSS module mounted on the UAV, and the UAV includes a flight control computer; the image acquisition The module acquires the image in the forward direction of the UAV and transmits it to the image processing module synchronously. The image processing module includes a CPU module and a GPU module to calculate the geometry, speed and position information of obstacles respectively. The GNSS module and the inertial measurement module respectively Perform real-time positioning and attitude measurement, the inertial measurement module also corrects the speed information calculated by the image processing module, and the flight control computer fuses the information sent by the image processing module and controls the UAV to avoid forward obstacles action.

As a preferred solution of the system of the present invention, the system also includes an ultrasonic module. The ultrasonic module includes four ultrasonic sensors, which are respectively installed in the four directions of the front, rear, left and right of the drone, and are used to detect four directions. of obstacles.

Compared with the prior art, the present invention adopts the above technical scheme and has the following technical effects:

1. The present invention effectively segments obstacles through binocular vision, and only calculates the optical flow value in the obstacle tracking window, which solves the time-consuming problem of calculating the entire image with dense optical flow, and further improves the real-time performance of the obstacle avoidance algorithm. sex.

2. The present invention proposes a new threat depth model, which can simplify the obstacle avoidance process without introducing an overly complicated path planning algorithm, and has high practical value.

3. The present invention uses GPU and CPU to calculate at the same time, uses GPU to calculate stereo matching and optical flow, and uses CPU to calculate the geometric size, position and speed of obstacles, thereby improving the calculation speed of the algorithm.

Description of drawings

Fig. 1 is a checkerboard diagram of the calibration camera in the present invention.

Fig. 2 is a hardware structure diagram of the UAV obstacle avoidance system based on binocular vision and optical flow fusion in the present invention.

Fig. 3 is an algorithm flow chart of the UAV obstacle avoidance method based on binocular vision and optical flow fusion in the present invention.

Fig. 4 is a schematic diagram of an obstacle avoidance strategy for deep continuous obstacles in an embodiment of the present invention.

detailed description

Embodiments of the invention are described in detail below, examples of which are illustrated in the accompanying drawings. The embodiments described below by referring to the figures are exemplary only for explaining the present invention and should not be construed as limiting the present invention.

As shown in Figure 2, it is a hardware structural diagram of the UAV obstacle avoidance system based on binocular vision and optical flow fusion of the present invention. The system includes an image acquisition module, an embedded image processing module, a flight control computer, an ultrasonic module, GNSS ( GlobalNavigation Satellite System (Global Satellite Navigation System) module and inertial measurement module, wherein the embedded image processing module includes a CPU module and a GPU module, and the ultrasonic module includes four ultrasonic sensors. The ultrasonic sensor is installed in the direction as an auxiliary obstacle avoidance device. When the ultrasonic sensor detects obstacles in other directions, the flight control computer adjusts the obstacle avoidance action according to the obstacle distance information.

The image acquisition module transmits the left and right images to the embedded image processing module synchronously; the CPU module and the GPU module are responsible for calculating the obstacle avoidance parameters of the UAV; the GNSS module and the inertial measurement module are responsible for the real-time positioning and attitude measurement of the UAV. The measurement module is also responsible for sending the measurement results to the embedded image processing module through the serial port to correct the calculated speed information; the flight control computer effectively fuses the information sent by the embedded image processing module and the ultrasonic module to control the UAV Make evasive maneuvers to avoid forward obstacles.

In this example, the image acquisition module adopts a binocular camera with a resolution of 640*480 or 800*600, etc., an available frame rate of 20 to 30fps, a baseline distance of 12cm, and both baseline distance and focal length can be adjusted. Two synchronous video signals are sent to the embedded image processing module.

As shown in Figure 3, the algorithm flow of the UAV obstacle avoidance method based on binocular vision and optical flow fusion in the present invention is as follows: first, the binocular camera is calibrated using the checkerboard as shown in Figure 1, and two cameras are respectively obtained Intrinsic parameter matrix and distortion parameters, extrinsic parameters (including rotation matrix and translation vector) between the two cameras are stored in the memory of the embedded image processing module.

Read in the synchronous video data sent by the binocular camera, use the internal parameter matrix, distortion parameters and external parameters to correct the two images, so that the two images have no distortion and line alignment; transform the two images to grayscale space and use GPU The module simultaneously calculates the energy functions of each pixel in 8 directions of up, down, left, right, upper left, lower left, upper right, and lower right and accumulates them, and calculates the parallax value that minimizes the energy function by calculating each pixel at the same time Determine the depth information (ie depth map) of each pixel. The CPU module obtains the obstacle tracking window according to the depth information calculated by the GPU module, determines the geometry and position information of the obstacle, and determines the obstacle tracking window; the GPU module then calculates all pixels in the window in parallel according to the selected obstacle tracking window The optical flow value is used to calculate the relative speed, and finally the CPU module corrects the speed information and sends it to the flight control computer for processing together with the position information and geometric dimensions.

Divide the depth values in the depth map into two categories, one is the depth value D _K ={d ₁ ,d ₂ ,…,d _K } belonging to obstacles, and the other is the depth value belonging to the background Find the segmentation threshold D _h by maximizing the variance between D _K and D _b , and set the depth value smaller than D _h to 0, calculate the contour in the depth map after processing, and select the largest contour for rectangular fitting to obtain obstacles Rectangular tracking window.

Variance between two classes of depth values Can be expressed as:

where ω ₀ and ω ₁ are the probabilities of the two types of depth values divided by D _h , and μ ₀ and μ ₁ are the mean values of the two types of depth:

Among them, {D ₁ ,...,D _t }=D _r is the discrete credible depth layer, K _j is the number of depth values in each depth layer, and t is the number of depth layers.

Extract the depth map distinguished by obstacles and background and calculate the contour in the depth map; before the contour detection, use the speckle filter to filter the depth map to remove the small block depth area; the detected closed area The largest contour is used as the obstacle contour, and it is fitted with a rectangular frame, and the fitted rectangle is used as the obstacle tracking window.

Calculate the threat depth D ₀ of obstacles to the UAV, assuming that D _K ={d ₁ ,d ₂ ,…,d _K } is a set of depth values belonging to obstacles, d ₁ ,d ₂ ,…,d _K is the depth value belonging to the obstacle, K is the number of these depth values, D _r is the credible depth interval, D ₁ , D ₂ ,..., D _t is the credible depth layer discrete on this interval, {D ₁ ,D ₂ ,…,D _t }=D _r , K ₁ ,…,K _t are the number of depth values in each depth layer D ₁ ,D ₂ ,…,D _t , when 1≤j≤t, if there is The threat depth _D0 of the obstacle tracking window is:

In the formula, D _min is greater than K _j corresponds to the minimum depth layer in the depth layer, and K _min is the number of depth values in D _min ;

if all The threat depth _D0 of the obstacle tracking window is:

Use the Horn-Schunck dense optical flow method to calculate the optical flow field in the obstacle tracking window, get the speed of the tracking window in the x and y directions, and judge the position of the next frame of the tracking window according to the speed; compare the judged position with the next frame The actual calculated tracking window position is compared. If the difference between the two is less than 10 pixels, it is judged that the calculation is accurate. If it is greater than 10 pixels, it is judged that the calculation is wrong and returns to the first step to recalculate.

Use the internal and external parameter matrix to convert the obstacle geometric information and obstacle velocity information represented by the calculated obstacle tracking window from pixel coordinates to world coordinates, and use the following formula to correct the velocity of the obstacle relative to the UAV:

In the formula, v _x and v _y are the corrected drone speeds in the x, y direction; f _x and f _y are the focal lengths in the x, y direction; u and v are the obstacles x, y calculated in step 4 directional optical flow vector; D ₀ is the threat depth of the obstacle tracking window; θ, are the pitch and yaw angles of the drone, respectively; Δt is the time between two frames.

Send the calculated speed, geometry, position information and threat depth information to the UAV flight control system, and the UAV flight control system controls the UAV to make real-time avoidance actions based on the information to avoid obstacles on the flight path , the location information is the offset value of the obstacle tracking window center relative to the image center.

As shown in Figure 4, when avoiding an obstacle with continuous depth, it is decomposed into multiple obstacles, and the threat depth of each part of the obstacle is calculated in different frame images for obstacle avoidance.

The present invention obtains the depth map information through the stereo matching algorithm accelerated by the GPU, and then obtains the geometric size of the obstacle, the relative position and speed of the UAV through the steps of obstacle segmentation, threat depth calculation, and optical flow calculation, and sends them to Until the flight control computer generates an obstacle avoidance action. The algorithm uses GPU and CPU for calculation at the same time, which improves the calculation speed of the algorithm, effectively segments obstacles through binocular vision, and only calculates the optical flow value in the obstacle tracking window, which solves the problem of dense optical flow calculation for the entire image. The time-consuming problem further improves the real-time performance of the obstacle avoidance algorithm; the threat depth model proposed by the present invention can simplify the obstacle avoidance process without introducing an overly complicated path planning algorithm, and has high practical value.

The above embodiments are only to illustrate the technical ideas of the present invention, and can not limit the protection scope of the present invention with this. All technical ideas proposed in accordance with the present invention, any changes made on the basis of technical solutions, all fall within the protection scope of the present invention. Inside.

Claims (8)

1. The UAV obstacle avoidance method based on binocular vision and optical flow fusion, is characterized in that, comprises the following steps: Step 1, use the binocular camera to obtain the image in the direction of the UAV, and perform grayscale transformation on the image; Step 2, calculate the feature information of each pixel on the image after the grayscale transformation and perform stereo matching to obtain the depth map information in the direction of the UAV; Step 3: Divide the depth values in the depth map into two types: those belonging to obstacles or the depth values belonging to the background, divide the depth map into obstacle areas and background areas, and use the contour with the largest closed area in the obstacle area as the obstacle contour , and fit it with a rectangular frame to obtain the obstacle tracking window as the geometric information of the obstacle; Step 4, use the dense optical flow method to calculate the velocity vector of the obstacle tracking window sliding, obtain the velocity of the window in the x, y direction, and judge the position of the obstacle tracking window in the next frame of image according to the speed, and compare the judged position with the next Compare the obstacle tracking window position actually calculated in one frame, if the difference between the two is less than the threshold, go to step 5, otherwise, return to step 1 to recalculate; Step 5, using the depth threat model to calculate the threat depth value of the obstacle to the UAV; Step 6, converting the obstacle geometry information calculated in step 3 and the obstacle velocity information calculated in step 4 from pixel coordinates to world coordinates, and correcting them using the motion parameters of the drone; Step 7. Send the obstacle position information and the geometry and speed information obtained in step 6 to the flight control computer of the UAV. The flight control computer controls the UAV to make real-time avoidance actions according to the above information. 2. according to claim 1, based on the UAV obstacle avoidance method based on binocular vision and optical flow fusion, it is characterized in that, the specific process of utilizing the binocular camera to obtain the image on the direction of advancement of the UAV described in step 1 is: Install the binocular camera on the nose of the UAV, obtain the internal and external parameter matrix and distortion parameters of the binocular camera through the calibration method, use the binocular camera to obtain the image in the direction of the UAV, and according to the internal and external parameter matrix and distortion The parameters are used to correct the image to obtain two images with no distortion and line alignment. 3. according to claim 1, the UAV obstacle avoidance method based on binocular vision and optical flow fusion, is characterized in that, the specific process of described step 2 is: calculate the up and down of each pixel on the image after the grayscale transformation , left, right, upper left, lower left, upper right, and lower right, and accumulate the energy functions in 8 directions, find the parallax value to minimize the accumulated energy function, and determine the depth value information of each pixel according to the parallax value. 4. according to claim 1, based on the UAV obstacle avoidance method based on binocular vision and optical flow fusion, it is characterized in that, before the outline with the largest closed area in the obstacle area is used as the obstacle outline in step 3, use speckle The filter filters the depth map that distinguishes the obstacle area and the background area to remove noise. 5. according to claim 1, the UAV obstacle avoidance method based on binocular vision and optical flow fusion is characterized in that, in step 3, the depth value in the depth map is divided into depth values belonging to obstacles or backgrounds The two types of specific processes are: Set the segmentation threshold D _h , classify the depth value greater than or equal to D _h as the depth value belonging to the obstacle, classify the depth value less than D _h as the depth value belonging to the background, and maximize the distance between the obstacle and the background Solving for the variance of D _h , the variance The calculation formula is: ${\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; d</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; \&omega;</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; \&mu;</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; \&mu;</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; \&omega;</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; \&mu;</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; \&mu;</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; ,</span>$ Among them, ω ₀ and ω ₁ are respectively the probability that the depth value is divided into obstacles and background depth values by D _h , and μ ₀ and μ ₁ are the mean values of the depth values belonging to obstacles and background respectively: $\begin{array}{cc}{\mathrm{<span\; class="notranslate">\; \&omega;</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; h</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}<span\; class="notranslate">\; (</span>\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; i</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}{\mathrm{<span\; class="notranslate">\; K</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; )</span>& {\mathrm{<span\; class="notranslate">\; \&mu;</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; =</span>\frac{\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; h</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}{\mathrm{<span\; class="notranslate">\; D.</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}\frac{{\mathrm{<span\; class="notranslate">\; K</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}}{\mathrm{<span\; class="notranslate">\; t</span>}}}{{\mathrm{<span\; class="notranslate">\; \&omega;</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}}\end{array}$ $\begin{array}{cc}{\mathrm{<span\; class="notranslate">\; \&omega;</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; h</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; t</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}<span\; class="notranslate">\; (</span>\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; i</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}{\mathrm{<span\; class="notranslate">\; K</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; )</span>& {\mathrm{<span\; class="notranslate">\; \&mu;</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; =</span>\frac{\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; h</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; t</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}{\mathrm{<span\; class="notranslate">\; D.</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}\frac{{\mathrm{<span\; class="notranslate">\; K</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}}{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; -</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; i</span>}}{\overset{\mathrm{<span\; class="notranslate">\; h</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}{\mathrm{<span\; class="notranslate">\; D.</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}\frac{{\mathrm{<span\; class="notranslate">\; K</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}}{\mathrm{<span\; class="notranslate">\; t</span>}}}{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; \&omega;</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}}\end{array}$ Among them, D _i is a discrete credible depth layer, i=1,...,t, t is the number of depth layers, D ₁ ,...,D _h are depth layers belonging to obstacles, D _h+1 ,...,D _t is the depth layer belonging to the background, K _j is the number of depth values in each depth layer, j=1,...,t. 6. according to claim 1, the UAV obstacle avoidance method based on binocular vision and optical flow fusion, is characterized in that, the specific process of described step 5 is: Set the depth value set belonging to obstacles as D _K ={d ₁ ,d ₂ ,...,d _K }, d ₁ ,d ₂ ,...,d _K are all depth values, and K is the depth value belonging to obstacles number; D ₁ ,…,D _t are discrete credible depth layers, t is the number of depth layers, K ₁ ,…,K _t are the number of depth values in each depth layer, if there is at least one 1≤j≤t, then the threat depth value of the obstacle to the UAV is: Among them, D _min is greater than K j of K _j corresponds to the minimum depth layer in the depth layer, and K _min is the number of depth values in D _min ; if all K ₁ ,...,K _t are less than Then the threat depth value of the obstacle to the UAV is: 7. The UAV obstacle avoidance system based on the fusion of binocular vision and optical flow is characterized in that it includes an image acquisition module, an image processing module, an inertial measurement module, and a GNSS module mounted on the UAV. The UAV includes a flight control module Computer; the image acquisition module acquires the image synchronously incoming image processing module on the advancing direction of the unmanned aerial vehicle, and the image processing module includes a CPU module and a GPU module, calculates respectively the geometry, speed, position information of obstacles, GNSS module, inertial measurement The module performs real-time positioning and attitude measurement on the UAV. The inertial measurement module also corrects the speed information calculated by the image processing module. The flight control computer fuses the information sent by the image processing module and controls the UAV to avoid Avoidance maneuvers for forward obstacles. 8. The UAV obstacle avoidance system based on binocular vision and optical flow fusion according to claim 1, characterized in that, the system also includes an ultrasonic module, and the ultrasonic module includes four ultrasonic sensors, which are respectively installed on the UAV's In the four directions of front, back, left and right, it is used to detect obstacles in four directions.