CN106981073A - A kind of ground moving object method for real time tracking and system based on unmanned plane - Google Patents

Abstract

The invention discloses a method and system for real-time tracking of ground moving targets based on an unmanned aerial vehicle. The target detection and recognition module of the ground control station is started to process the image sequence returned by the camera, and the size and size of the target rectangular frame on the display screen of the ground station are obtained. Center coordinates; start the target tracking module, use the algorithm fusion strategy to track the target, if the tracking is effective, output the target positioning result to the tracking command generation module; if the target is not located, start the target search module, search for the target and output the target positioning The results are sent to the tracking command generation module; according to the requirement that the target image needs to be positioned at the center of the display screen of the ground station, the tracking command generation module generates the position and attitude adjustment commands of the UAV, and uploads them to the UAV flight control system through the wireless transmission device. The pose is adjusted in real time. The invention has high matching efficiency, is easy to realize, can effectively carry out target identification, and avoids the influence of background noise.

Description

A method and system for real-time tracking of ground moving targets based on UAV

technical field

The invention belongs to the field of unmanned aerial vehicle navigation and computer vision, and in particular relates to a method for automatically detecting and tracking a target by using an unmanned aerial vehicle.

Background technique

UAVs have the advantages of high maneuverability, high resolution, good concealment, and flexible operation. Therefore, it has a huge advantage in the field of target reconnaissance and tracking, and has a larger surveillance range than traditional fixed cameras. It is mainly used in day and night air reconnaissance, traffic surveillance, military surveying and mapping and other fields. Utilizing the video sensor carried by the UAV to track and analyze the ground moving target has great practical significance in civil and military applications.

First of all, for most video surveillance systems, an area requiring special attention is monitored when the camera is stationary. The background is static, but the moving target as the foreground is moving. In this case, the target detection only needs to do the background difference method to achieve good results. However, in many cases, such as the target detection and tracking under the camera with the UAV as the carrier, the background of the image sequence taken by it is often changing and not fixed. In this case, the detection and tracking of the target to be tracked Tracking appears to be extremely difficult.

Secondly, the tracking of a single target does not mean that there is only a single moving object in the UAV's field of view, but that there are multiple moving objects in the scene, which interferes with the detection and tracking of the real target of interest. effective identification of targets. There is also background noise, for example, the extracted target is incomplete or has a hole in the center due to the influence of shadow or illumination, etc. In these cases, it often makes the detection and recognition of the target more difficult.

The terms used in the present invention are explained as follows:

UAV: It is an unmanned aircraft controlled by radio remote control equipment and its own program control device, including unmanned fixed-wing aircraft, unmanned helicopters and multi-rotor UAVs.

Wireless transmission equipment: a communication equipment using the MAVlink protocol, and the communication frequency band is generally 2.4G.

Shi-Tomasi corner point: An image feature point detection method, which represents the local features of the image, and is robust to image brightness changes, blur changes, rotation changes, and viewing angle changes.

FRI: Neighborhood image centered on the corner point, the size of which is 30×30 square area in the present invention.

Bhattacharyya coefficient: A value that measures the similarity between the target model and the candidate model region. The smaller the value, the greater the regional similarity; otherwise, the greater the regional similarity.

Contents of the invention

The present invention aims to provide a method and system for real-time tracking of a ground moving target based on an unmanned aerial vehicle, so as to solve the problem of difficult target detection and recognition in the prior art.

In order to solve the above-mentioned technical problems, the technical solution adopted in the present invention is: a method for real-time tracking of ground moving targets based on unmanned aerial vehicles, comprising the following steps:

1) The drone patrols according to the predetermined flight trajectory, transmits the captured image sequence to the ground control station, and detects the target of interest under the drone's field of view;

2) Extracting the size and center position information of the two-dimensional image rectangular frame of the above-mentioned target of interest;

3) Using the size and center position information of the rectangular frame of the two-dimensional image, fusing the output data of the mean shift algorithm and the Kalman filter algorithm, and outputting the final target positioning result in the form of data weighting.

After step 3), according to the target positioning result, adjust the flight mode of the UAV so that the moving target is located in the central area of the display screen of the ground station.

The specific implementation process of step 2) includes:

1) Extract the Shi-Tomasi corner point sets of two adjacent frames of the image sequence captured by the UAV;

2) Construct synthetic base descriptors for the Shi-Tomasi corner sets of the two frames of images respectively;

3) Perform feature matching on the Shi-Tomasi corner set with synthetic base descriptors, and obtain image corner matching pairs of two adjacent frames;

4) For the corner point matching pair obtained in step 3), utilize the RANSAC method to estimate the background motion transformation matrix, and perform image background motion compensation;

5) performing a frame difference operation on the motion-compensated adjacent frame images to obtain a frame difference image, and binarizing the frame difference image;

6) Do a morphological filtering operation on the frame difference image, separate and extract target information, and obtain the size and center position information of the target rectangular frame.

The specific generation process of the synthetic base descriptors for all corners of two adjacent frames of images includes:

1) Binarize each feature point neighborhood image FRI in two adjacent frames of images, and calculate the average gray value of the feature point neighborhood image FRI, when the pixel value in the feature point neighborhood image FRI is greater than The average gray value, the pixel value is set to 1; otherwise, set to 0;

2) Divide all 30×30 feature point neighborhood images FRI in adjacent two frames of images into 6×6 sub-regions with a size of 5×5, and synthesize the base image as a square composed of 5×5 black and white elements; The number of black pixels in the synthetic base image is half of the pixels in the FRI sub-region, and the number of synthetic base images is Wherein, N is the number of pixels in the FRI sub-region; K is the number of black pixels in the synthetic base image;

3) For any feature point neighborhood image FRI in step 2), compare all subregions of the feature point neighborhood image FRI with the synthetic base image set in order from left to right and from top to bottom, each A 9-dimensional vector is generated for each sub-region, and the respective 9-dimensional vectors of 36 sub-regions are combined to form a 324-dimensional synthetic base descriptor.

The 9-dimensional vector generation method of a sub-region of the feature point neighborhood image FRI is as follows: the comparison value of a sub-region and a composite base image in the composite base image set is the same number of black pixels at the same pixel point, The composite base image set is compared in order from left to right and from top to bottom, and then a sub-region is compared with all the composite base images in the composite base image set one by one according to the above comparison rules and order, and 9 integer values to form a 9-dimensional vector.

The specific steps of target information separation and extraction include:

a) Traversing the filtered frame difference image of each frame, the order of traversal is from top to bottom, from left to right;

b) If a pixel satisfies: the pixel value after binarization is 1 and there is no number, assign a new number to the pixel;

c) Traversing the eight neighborhoods of pixels assigned new numbers, according to the conditions in step b), giving new numbers to pixels in the eight neighborhoods that meet the conditions, and the new numbers are the same as the pixel numbers assigned new numbers Same; For the pixels in the eight domains that do not meet the conditions, return to step b);

d) After traversing and numbering all the pixel points with a pixel value of 1 in the frame difference image, the operation ends.

The method for determining the rectangular frame includes: after each frame of the filtered frame difference image is scanned, those with a pixel point of 1 have a number, and those with the same number are the same object, and they are connected together to form a moving object. For m moving objects, for the first moving object, the method of obtaining the rectangular frame is as follows: start traversing from the first marked pixel point until the last marked pixel point is traversed, and compare the x coordinates and y points in the marked pixel points The minimum and maximum values of the coordinates are saved and recorded as x _min , y _min , x _max , y _max , with (x _min , y _min ), (x _max , y _max ) two points as the opposite corners of the rectangular frame, Draw a rectangle.

The present invention also provides a system for real-time tracking of ground moving targets, including:

Unmanned aerial vehicles, used for patrolling according to a predetermined flight trajectory, transmitting the captured image sequence to the ground control station;

Wireless transmission equipment: provide a communication method for data transmission between the UAV and the ground control station;

The ground control station is used to detect the target of interest under the field of view of the drone, extract the size and center position information of the two-dimensional image rectangular frame of the target of interest, and use the size and center position information of the two-dimensional image rectangular frame, The output data of the mean shift algorithm and the Kalman filter algorithm are fused, and the final target positioning result is output in the form of data weighting.

Correspondingly, the system further includes: a tracking instruction generation module, configured to adjust the flight mode of the drone according to the target positioning result, so that the moving target is located in the central area of the display screen of the ground station.

The ground control station includes:

The detection and identification module is used to detect the target of interest under the field of view of the drone, and extract the size and center position information of the two-dimensional image rectangular frame of the target of interest;

The target tracking module uses the size and center position information of the rectangular frame of the two-dimensional image, fuses the output data of the mean shift algorithm and the Kalman filter algorithm, and outputs the final target positioning result in the form of data weighting.

The target search module, when a tracked target is lost, this module uses a sequential search method to relocate the target.

The tracking instruction generation module generates corresponding tracking instructions according to the imaging area of the tracking target in the display screen of the ground station, so that the target is located at the center of the display screen.

Compared with the prior art, the present invention has the beneficial effects that: the detection and tracking process of the target in the present invention does not require manual participation in the whole process, the synthetic base descriptor used is used for feature point matching, and the scale rotation, illumination, and blur changes It has robustness and high matching efficiency, and the generation of synthetic base descriptors does not involve floating-point operations. It is friendly to the hardware platform for image processing, easy to implement, and can effectively perform target recognition and avoid the influence of background noise.

Description of drawings

Figure 1 is a structural composition diagram of the UAV system;

Fig. 2 is the flow chart of the method for estimating the background motion parameters based on the synthetic base descriptor of the unmanned aerial vehicle system;

Figure 3 is a diagram of target information separation and extraction;

Figure 4(a) Synthetic base image collection; Figure 4(b) Binarized FRI; Figure 4(c) Comparison value of the first sub-region of FRI with the first synthetic base image; Figure 4(d) FRI compare values of the first subregion with the second synthetic base image;

Fig. 5 is a flow chart of moving target separation and information extraction;

Figure 6 is a flow chart of the algorithm fusion and search strategy of the UAV system;

Fig. 7 is a flow chart of the UAV system search sequence hierarchy strategy;

Fig. 8 is a schematic diagram of sub-regions of the display screen of the ground station of the UAV system;

Fig. 9 is an arbitrary upper and lower frame image of the UAV video sequence;

Figure 10 is a corner matching image based on synthetic base descriptors;

Fig. 11 is the frame difference detection result image;

Figure 12 is a target detection image after morphological filtering;

Figure 13 is an image of target separation and information extraction.

detailed description

Figure 1 is a composition diagram of the UAV system, which includes UAV, camera, wireless transmission equipment and ground control station. As the carrier of the camera, the UAV expands the shooting range of the camera. The wireless transmission equipment provides a communication method for the download of the image sequence collected by the UAV and the upload of the flight control command of the ground station; the ground control station includes four modules, namely the target detection and recognition module, the target tracking module, the target search module, and the tracking command Generate modules.

The specific implementation method of UAV system tracking is as follows:

1. The UAV uses the pre-planned flight path to patrol according to the flight area specified by the user. The camera downloads the image sequence captured by the wireless transmission device to the target detection and recognition module of the ground control station for processing, and obtains the The imaging position and the size of the rectangular frame on the display screen of the ground station. Any two adjacent frames of the image sequence captured by the UAV are shown in Figure 9.

2. Start the target detection and recognition module of the drone, detect the target of interest in the field of view of the drone, and extract the size and center position information of the rectangular frame of the target on the display screen. The target detection and recognition module is divided into two processes. Background Motion Parameter Estimation and Object Information Separation and Extraction Based on Synthetic Base Descriptors. The first process will introduce the specific implementation method below, and Fig. 2 is a kind of flow chart of the background motion parameter estimation method based on synthetic base descriptor:

1) Extract the feature points of the starting frame. Since the Shi-Tomasi corner points are highly efficient, this feature point is used. Set the starting frame as X, and define an autocorrelation function F at pixel s as follows:

where δs represents the displacement, and W represents the wide window centered on S

Carrying out first-order Taylor expansion on X(s+δs), the above formula can be rewritten as follows:

Among them, △X is the first order derivative function of the image, and Λ is the precision matrix. The feature point extraction standard is that the minimum value of the precision matrix eigenvalue is greater than a constant, namely:

Q(s)=min{λ ₁ ,λ ₂ }>K (3)

Where K is the empirical threshold, generally between 0.05 and 0.5.

2) For the binarization of the corner neighborhood, it is generally more reasonable to take a square neighborhood of 30×30 feature points, which can take both complexity and accuracy into consideration. Next, the descriptor is generated, and the FRI is binarized. The average gray value of the neighborhood of feature points needs to be calculated. The formula for calculating the average gray value of FRI is as follows:

In the formula, p is the number of pixels of FRI, here is 900; I(x, y) is the pixel gray value of a certain point in FRI.

Then, when the value of the pixel in the neighborhood of the feature point is greater than g, the value of the pixel is set to 1; when the value of the pixel in the neighborhood of the feature point is smaller than g, the value of the pixel is set to 0. Through this process, the binarized FRI can be obtained, which can retain the structural information in the neighborhood of key points, and lay the foundation for the next step of feature point descriptor generation.

3) To construct a corner descriptor, first divide the 30×30 FRI into 6×6 sub-regions of 5×5, in order to compare the corresponding elements of the FRI sub-regions with the synthetic base image, the size of a synthetic base image is equal to The subregions of FRI are equal. The composite base image is a square area composed of black and white elements, and the number of composite base images can be determined by the following composite base function.

In the formula, N is the number of pixels in the sub-region; K is the number of black pixels in the synthetic base image; M is the number of SBIs, which can uniquely represent a feature point.

In order to improve the real-time performance of the algorithm, it is of course hoped that the number of synthetic base images should be as small as possible. When K is half of N, the function has a minimum value. The result of K is a decimal, and the rounding operation is performed by adding 1. For example, a 30×30 FRI is divided into 6×6 5×5 sub-regions, then N is 13, and the number of synthetic base images is 13ln(25/13) or 9; a 30×30 FRI is divided into two 15×15 , then N is 450, and the number of synthetic base images is 113ln(225/113) or 78. Take the 5×5 sub-region example in Figure 4(a) to Figure 4(d) as a specific description of the algorithm:

Figure 4(a) is the synthetic base image set, which is composed of 9 synthetic base images. In each synthetic base image area, 13 pixels are black, and the rest are white. These 13 black points are distributed in a pseudo-random manner. 5×5 area, but it must be ensured that the distribution patterns of each synthetic base image are different from each other. Figure 4(b) shows the binarized FRI and divides it into 36 5×5 sub-regions. From left to right and from top to bottom, compare the first sub-region with each synthetic base image. The comparison rule is to see that the two have the same number of black points at the same pixel, so that each sub-region Each region will generate a 9-dimensional vector, which is the descriptor of the subregion, and the range of each component is (0, 13).

According to the comparison sequence above, the descriptions of the remaining 35 sub-regions are obtained. Finally, the descriptors of 36 sub-regions are combined to form a 324-dimensional descriptor. Among them, Figure 4(c) is the descriptor obtained by comparing the first sub-region and the first synthetic base image, with a value of 6; Figure 4(d) is the comparison between the first sub-region and the second synthetic base image The descriptor of is 7.

4) Corner matching based on synthetic base descriptors. The success of the matching of feature points means that the "distance" between the two feature points is the shortest. The most commonly used methods to measure this distance are Euclidean distance, Mahalanobis distance, etc., but the complexity of its calculation is high-dimensional vector unacceptable. Based on this, the L1 norm is used to measure the "distance" of feature points. In order to illustrate the matching process of the feature point set, it is assumed that there are m feature points in the current frame of the video sequence, and n feature points in the next frame, then the L1 norm to measure the distance between the feature points in the upper and lower frames is as follows:

x _i represents the i-th synthetic base descriptor of the current frame, y _j represents the j-th synthetic base descriptor of the next frame image, and w represents the dimension of the descriptor, including 324 components.

The principle of synthetic base descriptor calculation is shown in Figure 5. Each row represents a corner point descriptor, and then the distance is calculated using the L1 norm. In Figure 5, the distance between corner point 1 and corner point 2 is 3. From the previous step, the distance between any feature points in the two images can be obtained. In order to reduce the probability of false matching, a cross-matching method is used: calculate the L1 of the i-th corner point in the current frame and all corner points in the next frame Norm distance d, a total of n distance values are obtained, and the minimum distance is selected as the candidate matching point, which is recorded as y _j ; according to the above method, the distance between the jth corner point of the next frame and all the corner points of the previous frame is calculated , a total of m distance values are obtained, and the minimum value obtained is marked as t. If t=j, it can be determined that x _i and y _j are a pair of feature points that match correctly, otherwise it is considered that the match is wrong. As shown in Figure 10, the corner matching graph of the aerial image is obtained by the cross-matching method.

5) Use the RANSAC algorithm to exclude the corner points (external points) on the moving object, and then estimate the background transformation matrix. Estimate the motion parameters of the background, and hope that the corner matching pairs come from the background corner group as much as possible. For the corner matching pairs in the previous step, it is necessary to use the RANSAC algorithm to eliminate the error interference of the moving target corner matching pairs, so that the calculated The background motion compensation parameters are more precise. Since the image transformation form adopted is an eight-parameter projective transformation model, at least four sets of matching pairs are required to solve the background transformation matrix, and the eight-parameter projective transformation model is as follows;

The algorithm process of the RANSAC algorithm to calculate the background motion compensation matrix is as follows:

a) First define all matching point pairs of two images as the overall sample D, randomly select four groups of matching points as a sample data J _i , and calculate the background parameter model H(J) according to the sample data.

b) From the instance H(J _i ) calculated in the previous step, determine the set of matching points whose geometric distance between H(J i ) and H(J _i ) in the population D is less than the threshold d, and record it as S(H(J _i ) ), called the consistent set of instance H(J _i ).

c) Calculate another consistent set S(H(J _k )) from a) and b) in two steps. If S(H(J _i ))>S(H(J _k )), keep the consistent set S(H (J _i )); otherwise, keep the consistent set S(H(J _k )).

d) After K times of random sampling, select the matching pair in the consistent set with the largest number as the correct matching pair, that is, the background corner point group.

e) Calculate the background motion transformation matrix H by using the least squares method from the determined background corner point group.

Among them, the determination of d and k parameters is calculated as formula (8) and (9) respectively:

d=‖x _i -Hx _i ‖ (8)

In the formula, x _i is a data point of the overall sample; w is the probability of a good sample (inlier point).

The second process of target detection and recognition, the process of target information separation and extraction is shown in Figure 3, and the specific implementation method is as follows:

1) In order to calculate the frame difference image, because there are multiple moving objects in the drone's field of view, a frame-before-frame-after-frame difference method is used to detect all moving objects. The calculation formula is as follows:

Where X _t-2 , X _t-1 , and X _t are any consecutive three frames of the video sequence; and Both are background transformation matrices; E _t-1 is the frame difference subtracted image. The aerial image of the UAV is processed through this step, as shown in Figure 11.

2) Binarization of the frame difference image, using an appropriate threshold to binarize the image obtained in step S301.

3) Morphological filtering operation. The binarized image obtained in step 302 is filtered by morphological operations, which will make the segmentation effect of each moving object more obvious. The morphological operation process is as follows:

a) Perform image erosion on it to remove isolated noise points.

b) Then perform image expansion on it, which is to expand the edge of the target, fill in the missing pits, and make the outline smoother.

After mathematical morphological processing, the detection results are fuller and the target area is more obvious, which is more conducive to the segmentation and information extraction of various moving objects. Fig. 12 is an aerial image after morphological filtering.

4) Separation and extraction of target information. In order to separate multiple moving objects in each frame, it is first necessary to connect and associate each moving object, mark each moving object in each frame as a different number, and finally select the same area come out. To achieve the above purpose, the sequential marking method is generally used. This method can complete the marking and separation of moving objects. Usually, each frame is scanned from top to bottom and left to right. The pixel template used in this method is 3*3 in size, and the specific steps are as follows:

a) Perform pixel traversal for each frame, and the order of traversal is from top to bottom and from left to right.

b) If a pixel satisfies two conditions: the pixel value after binarization is 1 and there is no number, assign a new number to the pixel.

c) Traverse the eight neighbors of the pixel found in b), repeat the condition in b), and give the same number.

d) When the condition in c) is not satisfied, repeat the operation of b).

e) When all the points in the image with a pixel value of 1 have been traversed and numbered, the operation ends.

After each frame is scanned, the pixels with a pixel of 1 are numbered, and those with the same number are objects, and they are connected together to form a moving object. Assuming there are m objects, now take the first moving object as an example, and the rectangular frame is acquired The method is as follows: sequentially traverse from the first marked pixel point to the last marked pixel point, and save the minimum and maximum values of the x-coordinate and y-coordinate in the marked pixel point as x _min , y _min , x min _max , y _max , and then you can draw a rectangular frame. Usually two points (x _min , y _min ) and (x _max , y _max ) are used as the opposite corners of the rectangular frame to draw a rectangular frame. The methods for obtaining the rectangular frames of other moving objects are the same as above. The effect of any two adjacent frames of the UAV image sequence after this step is shown in Figure 13.

3. Start the target tracking module, and input the position and size information of the tracking target rectangular frame obtained in the previous step into the two tracking algorithms of the tracking module. The specific operation process of this step is as follows:

1) Assuming that the target motion model obeys the uniform velocity model, the Kalman filter outputs the positioning result, which is recorded as the first target true value y _kf .

The Kalman filter uses the transition model to predict the current state from the previously estimated state, and updates the current state with the current state as follows, where

Then use the Kalman filter gain K to calculate the current state true value b(t):

Assuming that the current movement model of the moving target is a uniform movement, it is sufficient to set A and M according to the model. Among them, A is the state transition matrix ω _t to control the transition model error, M is the measurement matrix, and ε _t represents the measurement error. where Vω and _Vε are the covariances of _ωt and _{εt ,} respectively. In our application, we assign the size and position of the bounding box of the detected object as the state variable b(t), initializing the Kalman filter.

2) Using the mean shift tracking algorithm, the position of the target template has been given by the target detection and recognition module, so the target positioning result can be output, which is recorded as the second target true value y _ms . The specific process of the mean shift algorithm is very mature, so it is not repeated here.

3) Use the weighting and data fusion method to output the positioning result of the target when it is not lost. , if the target is lost, enable the search module to relocate the target result.

From the first target true value y _kf output in the first step, and the second target true value y _ms output in the second step, the following strategy is used for weighted fusion of data, and the Bhattacharyya coefficient is used to measure the target model and candidate area (second target true value), when the similarity is greater than 0.8, it is considered that the second target true value is completely credible; when the similarity is greater than 0.5 and less than 0.8, the second target true value is not completely trusted, and the data weighted fusion operation is performed; When the similarity is less than 0.5, it is considered that the target has been occluded or the target state has changed. If the target is considered to be occluded or the target state has changed, it can be considered that the target is lost, and the target search module needs to be activated to relocate the target; the data fusion of the above three cases The way can be judged by formulas (13), (14) and (15):

ρ<0.5, y=NULL (15)

In the formula, ρ is the similarity; d is the empirical threshold; y _ms , y _kf are the target values of the mean shift algorithm and the Kalman filter algorithm respectively.

It can be seen from the above that when the output value is NULL, the fusion strategy algorithm believes that the target is lost due to occlusion and other reasons, and the UAV system will automatically switch from the tracking module to the target search module to reposition the target on the ground station display area.

4) As shown in Figure 4, the search sequence flow chart is shown. When the tracking target is lost, the target search module is started. This module uses a sequential search method, which is divided into two levels. It is more targeted to the cause of the target loss and improves the search efficiency. higher.

In the first layer, the equidistant search of front and rear frame differences, y _k+1 = y _k + Δy, where Δy = y _k - y _k-1 .

a) Assuming that the currently processed image sequence is the Kth frame, y _k is the center position of the target at time K, and the target center of the tracking default image sequence is y ₀ , y ₁ , ..., y _k-1 , y _k , y _{k+ 1} ,….

b) Use the frame difference equidistance formula to calculate the center position of the K+1th frame according to the image position point of the Kth frame, and then use the position to take the same size as the rectangular box output by the target detection and recognition module as the candidate target, and calculate its target Then calculate the similarity with the target template. If the similarity is greater than the set threshold of 0.75, choose to trust the candidate template and find the target; otherwise, don’t trust it and enter the second-level search strategy.

The second layer, the local/global search strategy, searches locally first, that is, in the partition where the target was lost in the previous frame, the particle filter method is used to search again. Specifically, if the target is in the imaging field of the camera If the sixth area is lost, spray N particles evenly in this area first, and relocate to the target; if the target cannot be found within K frames, use the sub-area particle filter method, and use particles in areas 1-9 respectively In the filter tracking method, each area will filter out a tracking result, and then use a weighted fusion of the results of each area, and finally obtain the position of the target.

4. According to the target positioning result output in the previous step, enable the tracking command generation module and adjust the flying mode of the UAV so that the moving target is located in the center area of the image. As shown in Figure 5, the sub-regions of the image are numbered. Use this sub-region to activate the tracking command generation module, send commands to the flight control system of the UAV through the wireless transmission module, and adjust the flight mode so that the target imaging area at the current moment moves toward the central area (section five regions) to move. Specifically, the adjustment method of the tracking instruction generation module is as follows:

Area 5: The center area of the image. If the center point of the target is located in this area, the flight attitude of the UAV remains unchanged and no tracking command is generated.

Zone 1: If the center point of the target is located in this area, the tracking command module generates a left front flight mode to control the flight attitude of the UAV so that the center point of the target image is located in the center area of the image.

Zone 2: If the center point of the target is located in this area, the tracking instruction module generates a forward flight mode to control the flight attitude of the UAV so that the center point of the target image is located in the center area of the image.

Area 3: If the center point of the target is located in this area, the tracking command module generates a right-front flight mode to control the flight attitude of the UAV so that the center point of the target image is located in the center area of the image.

Area 4: If the center point of the target is located in this area, the tracking command module generates a flight mode to the left, and controls the flying attitude of the UAV so that the center point of the target image is located in the center area of the image.

Area 6: If the center point of the target is located in this area, the tracking instruction module will generate a right flight mode to control the flying attitude of the UAV so that the center point of the target image is located in the center area of the image.

Area 7: If the center point of the target is located in this area, the tracking instruction module generates a left rear flight pattern to control the flight attitude of the UAV so that the center point of the target image is located in the center area of the image.

Area 8: If the center point of the target is located in this area, the tracking instruction module generates a backward flight mode to control the flight attitude of the UAV so that the center point of the target image is located in the center area of the image.

Area 9: If the center point of the target is located in this area, the tracking instruction module will generate the lower right flight mode to control the flight attitude of the UAV so that the center point of the target image is located in the center area of the image.

Claims (10)

1. a kind of ground moving object method for real time tracking based on unmanned plane, it is characterised in that comprise the following steps：

1) unmanned plane is gone on patrol according to predetermined flight path, and ground control will be transferred to the image sequence that ground visual field is shot System station, detects the interesting target of unmanned aerial vehicle vision off field；

2) the two dimensional image rectangle frame size and center location information of above-mentioned interesting target are extracted；

3) the two dimensional image rectangle frame size and center location information are utilized, fusion mean shift algorithm and Kalman filtering are calculated The output data of method, final goal positioning result is exported using the form of data weighting.

2. the ground moving object method for real time tracking according to claim 1 based on unmanned plane, it is characterised in that step 3) after, according to the target positioning result, unmanned plane during flying pattern is adjusted, moving target is located at earth station's display screen center Region.

3. the ground moving object method for real time tracking according to claim 1 based on unmanned plane, it is characterised in that step 2) the process that implements includes：

1) the Shi-Tomasi angle point set of adjacent two frame for the image sequence that unmanned plane is shot is extracted respectively；

2) the Shi-Tomasi angle points set to two field pictures constructs synthesis base description respectively；

3) characteristic matching is carried out to the Shi-Tomasi angle points set with synthesis base description, obtains the image angle of adjacent two frame Point matching pair；

4) to step 3) obtain corners Matching pair, estimate background motion transformation matrix using RANSAC methods, and schemed As Background Motion Compensation；

5) make frame difference operation to the adjacent two field picture after motion compensation, obtain frame difference image, and by frame difference image binaryzation；

6) make morphologic filtering operation to frame difference image, carry out target information separation and extraction, obtain the size of target rectangle frame With center location information.

4. the ground moving object method for real time tracking according to claim 3 based on unmanned plane, it is characterised in that adjacent The specific generating process of all angle point synthesis base description of two field pictures includes：

1) binary conversion treatment is carried out to each characteristic point neighborhood image FRI in adjacent two field pictures, and calculates feature vertex neighborhood Image FRI average gray value, when the pixel point value in characteristic point neighborhood image FRI is more than average gray value, the then pixel Value is set to 1；Otherwise, set to 0；

2) it is 5 × 5 the characteristic point neighborhood image FRI of all 30 × 30 sizes in adjacent two field pictures to be divided into 6 × 6 sizes Subregion, synthesis basic image is the square of 5 × 5 black and white elements composition；The synthesis basic image black pixel point number For the half of FRI subregion pixels, the number of basic image is synthesizedWherein, N is the number of pixels of FRI subregions；K For the number of black picture element in synthesis basic image；

3) for step 2) in any one characteristic point neighborhood image FRI, by this feature vertex neighborhood image FRI all subregions With order from left to right, from top to bottom with synthesis basic image set be compared, each sub-regions all generate one 9 tie up to Amount, combines respective 9 dimensional vector of 36 sub-regions, eventually forms synthesis base description of one 324 dimension.

5. the ground moving object method for real time tracking according to claim 4 based on unmanned plane, it is characterised in that described The characteristic point neighborhood image FRI dimensional vector generation method of a sub-regions 9 is：One sub-regions are with synthesizing one in basic image set It is individual synthesis basic image fiducial value for both at same pixel black picture element identical numbers, synthesis basic image set by than Compared with order for from left to right, from top to bottom, then a sub-regions are according to above-mentioned comparison rule and comparative sequence, with synthesis All synthesis basic images are compared one by one in basic image set, obtain 9 integer values, constitute 9 dimensional vectors.

6. the ground moving object method for real time tracking according to claim 3 based on unmanned plane, it is characterised in that target The specific steps that information is separated and extracted include：

A) each filtered frame difference image of frame is traveled through, the order of traversal is from top to bottom, from left to right；

If b) pixel is met：Pixel value after binaryzation is 1 and not numbered, then new numbering is assigned to the pixel；

C) traversal imparts the eight neighborhood of the pixel of new numbering, according to the condition in step b), gives 8 neighborhoods of the condition of satisfaction The numbering of interior pixel newly, and the new numbering is identical with imparting the pixel number of new numbering；For being unsatisfactory for condition Pixel in eight fields, return to step b)；

D) when all pixels value in frame difference image has been traveled through for 1 pixel and after all numbers of volume, operation terminates.

7. the ground moving object method for real time tracking according to claim 6 based on unmanned plane, it is characterised in that described The determination method of rectangle frame includes：After each filtered frame difference image of frame is scanned through, pixel is 1 numbering that has, volume Number identical is then same object, links together and just constitutes moving object, it is assumed that have m moving object, for first Moving object, rectangle frame acquisition methods are as follows：Begun stepping through successively from first labeled pixel, until having traveled through last One labeled pixel, the minimum value of x coordinate and y-coordinate in labeled pixel is preserved with maximum, is designated as x_min, y_min, x_max, y_max, with (x_min,y_min),(x_max,y_max) 2 points of angle steel joints as rectangle frame, draw rectangle frame.

8. a kind of system of ground moving object real-time tracking, it is characterised in that including：

Unmanned plane, for being gone on patrol according to predetermined flight path, ground control station is transferred to by the image sequence of shooting；

Ground control station, for detecting the interesting target of unmanned aerial vehicle vision off field, extracts the two dimensional image square of interesting target Shape frame size and center location information, and utilize the two dimensional image rectangle frame size and center location information, fusion average drift The output data of algorithm and Kalman filtering algorithm is moved, target positioning result is obtained using the form output of data weighting is final.

9. the system of ground moving object real-time tracking according to claim 8, it is characterised in that also include：

Trace command generation module, for according to the target positioning result, adjusting unmanned plane during flying pattern, makes moving target position In earth station's display screen central area.

10. the system of ground moving object real-time tracking according to claim 8, it is characterised in that

The ground control station includes：

Detection and identification module, for detecting the interesting target of unmanned aerial vehicle vision off field, and extract the two dimension of interesting target Image rectangle frame size and center location information；

Target tracking module, using the two dimensional image rectangle frame size and center location information, fusion mean shift algorithm and The output data of Kalman filtering algorithm, target positioning result is obtained using the form output of data weighting is final.；

Target search module, when losing tracking target, the module repositions target using a kind of sequence search method；

Trace command generation module, according to imaging region of the tracking target in earth station's display screen, the corresponding tracking of generation refers to Order, so that target is located at display screen center.